Second-expand only the prerequisites of the targets being built.
Defer second-expanding the prerequisites of targets until we need
to decide if they should be built.
* NEWS: Mention the change in behavior.
* doc/make.texi (Secondary Expansion): Document the new behavior.
* src/filedef.h (struct file): Add flag snapped.
(expand_deps): Declare a function to second expand the
prerequisites of a target.
* src/file.c (rehash_file): Merge flag snapped.
(expand_deps): Remove qualifier static. Check flag snapped.
(snap_deps): Remove the loop which performed second expansion for all
targets.
* src/remake.c (update_file_1): Second expand the prerequisites of
the considered target.
* tests/scripts/features/se_explicit: Add tests.
* tests/scripts/features/se_implicit: Ditto.
* tests/scripts/features/se_statpat: Ditto.
Savannah issues such as SV 57242 and SV 62397 show how passing
references to closed file descriptors via the --jobserver-auth option
in MAKEFLAGS can lead to problematic outcomes.
When computing the child environment for a non-recursive shell, add
an extra option to MAKEFLAGS to disable the file descriptors for the
jobserver.
Unfortunately this doesn't modify the value of the make variable
MAKEFLAGS, it only modifies the value of the sub-shell environment
variable MAKEFLAGS. This can lead to confusion if the user is not
considering the distinction.
* src/makeint.h: Publish the jobserver-auth value. Add a global
definition of the name of the command line option.
* src/os.h (jobserver_get_invalid_auth): New function to return a
string invalidating the jobserver-auth option.
* src/w32/w32os.c (jobserver_get_invaid_auth): Implement it. On
Windows we use a semaphore so there's no need to invalidate.
* src/posixos.c (jobserver_parse_auth): If we parse the invalid
auth string, don't set up the jobserver.
(jobserver_get_invalid_auth): Return an invalid option.
* src/variable.h (target_environment): Specify if the target
environment is for a recursive shell or non-recursive shell.
* src/variable.c (target_environment): Move checking for MAKELEVEL
into the loop rather than doing it at the end.
Along with this, check for MAKEFLAGS and MFLAGS, and update them
based on whether we're invoking a recursive or non-recursive child,
and also on whether it's necessary to invalidate the jobserver.
* src/function.c (func_shell_base): Shell functions can never be
recursive to pass 0 to target_environment().
* src/job.c (start_job_command): Specify whether the child is
recursive when calling target_environment().
* src/main.c: Export jobserver_auth. sync_mutex doesn't need to
be exported. Use the global definition for the option name.
* tests/scripts/variables/MAKEFLAGS: Add tests for $MAKEFLAGS.
Export all variables, including exported makefile variables, when
invoking a shell for the $(shell ...) function. If we detect a
recursive variable expansion, silently ignore that variable and do
not export it. We do print a debug message.
* NEWS: Announce the potential backward-incompatibility.
* doc/make.texi (Shell Function): Document the export behavior.
* src/main.c (main): Add "shell-export" to .FEATURES.
* src/job.h: New function to free struct childbase.
* src/job.c (free_childbase): Implement it; call from free_child.
* src/function.c (func_shell_base): Use target_environment() to
obtain the proper environment for the shell function.
Use free_childbase() to free memory.
(windows32_openpipe): Don't reset the environment: the caller
already provided a proper PATH variable in envp.
* src/variable.c (target_environment): If we detect a recursive
expansion and we're called from func_shell, ignore the variable.
(sync_Path_environment): Simplify and reduce memory allocation.
* tests/scripts/functions/shell: Add tests for this.
Introduce non-deterministic ordering into goal and prerequisite
traversal to help tease out inconsistent failures that may happen
when running in parallel build mode.
Introduce second order into each dependency chain:
1. Existing order is syntactic order reachable via 'dep->next'
2. New order is shuffled order stored as 'dep->shuf' in each 'dep'
When updating goals and prerequisites and '--shuffle' is provided,
use the shuffled order to walk the graph. When automatic variable
are set always use the syntactic order of parameters.
* Makefile.am: Add new src/shuffle.c and src/shuffle.h file.
* build_w32.bat: Ditto.
* builddos.bat: Ditto.
* makefile.com: Ditto.
* po/POTFILES.in: Ditto.
* doc/make.texi: Add documentation for --shuffle.
* doc/make.1: Ditto.
* src/dep.h (DEP): Add the shuf pointer.
* src/filedef.h (struct file): Add was_shuffled flag.
* src/main.c: (shuffle_mode): Global flag for the shuffle mode.
(usage): Add the --shuffle option.
(switches): Ditto.
(main): Set shuffle_mode based on the command line parameter.
Reshuffle prerequisites if requested.
* src/remake.c (update_goal_chain): Walk the shuffled list if enabled.
(update_file_1): Ditto.
* src/shuffle.h: Provide an interface for shuffling prerequisites.
* src/shuffle.c: Implement option parsing and prerequisite shuffling.
* tests/scripts/options/shuffle: Test shuffle option and modes.
Further experimentation shows that my previous thinking that there
was a problem using this was woefully mistaken. The value generated
by the kernel shows runn*ABLE* processes not runn*ING* processes.
* NEWS: Announce the change.
Numbers can come from $(words ...), automatic variables such as
$(MAKELEVEL), from environment variables, or from shell output such as
through $(shell expr ...). The $(intcmp ...) function allows
conditional evaluation controlled by numerical variables.
* NEWS: Announce this feature.
* doc/make.texi (Functions for Conditionals): Document 'intcmp'.
* src/function.c (func_intcmp): Create the 'intcmp' built-in function.
* tests/scripts/functions/intcmp: Test the 'intcmp' built-in function.
* NEWS: Announce the change.
* doc/make.texi (Implicit Rule Search): Update the definition of
"ought to exist" and add the extra step for compatibility mode.
When MAKEFLAGS is set in a makefile, reparse it immediately rather
than waiting until after all makefiles have been read and parsed.
This change doesn't actually fix the SV bug referenced because, even
though we do reparse MAKEFLAGS, we don't handle the -r or -R options
immediately. Doing this will require more effort.
* NEWS: Announce the change.
* src/makeint.h: Publish reset_switches() and decode_env_switches()
from main.c
* src/main.c (main): Don't call construct_include_path(); it will be
invoked decode_switches().
Preserve the old values of builtin_rules, builtin_variables, and
job_slots before we read makefiles since they can be changed now.
(reset_switches): Publish (remove static). Set the initial value of
the stringlist list to NULL.
(decode_switches): Call construct_include_path() after decoding.
(decode_env_switches): Publish (remove static).
(define_makeflags): Set the MAKEFLAGS variable for special handling.
* src/read.c (eval_makefile): Check for empty include_directories.
(construct_include_path): Clear any old value of .INCLUDE_DIRS before
appending new values. Free the previous include_directories.
* src/variable.c (lookup_special_var): When MAKEFLAGS is set, first
reset the switches then re-parse the variable.
* tests/run_make_tests.pl: Memo-ize some default variable values.
* tests/scripts/options/dash-r: Create tests for setting -r and -R.
* tests/scripts/variables/MAKEFLAGS: Test that resetting -I from
within the makefile takes effect immediately.
Accept a "-" directory value to the -I option to clear the set of
directories to be searched up to that point, including the default
directories.
* NEWS: Announce the change.
* doc/make.texi (Summary of Options): Add documentation.
* src/read.c (construct_include_path): Check for '-' and if found,
clear the list of directories to be searched.
* tests/scripts/options/dash-I: Add tests for -I-.
* tests/scripts/variables/INCLUDE_DIRS: Add tests for -I-.
Support a new special target, .NOTINTERMEDIATE. Any file or pattern
prerequisite of this target will never be considered intermediate.
This differs from .SECONDARY in that .SECONDARY files won't be deleted
but they will still not be built if they are missing.
.NOTINTERMEDIATE files are treated the same way as a target which is
explicitly mentioned in the makefile. This is mostly useful with
patterns; obviously mentioning a target explicitly here is enough in
and of itself to make something not intermediate.
Some adjustments made by psmith@gnu.org
* NEWS: Announce the new feature.
* doc/make.texi (Special Targets): Document .NOTINTERMEDIATE.
(Chained Rules): Describe how to use .NOTINTERMEDIATE.
* src/main.c (main): Add "notintermediate" to the .FEATURES variable.
* src/filedef.h (struct file): Add "notintermediate" flag.
* src/file.c (no_intermediates): Mark global .NOTINTERMEDIATE.
(snap_file): Support .NOTINTERMEDIATE special target. Throw an error
if the same target is marked both .NOTINTERMEDIATE and .SECONDARY or
.INTERMEDIATE.
(rehash_file): Merge intermediate, notintermediate, secondary flags.
(remove_intermediates): Check notintermediate flag before removing.
(print_file):
* src/implicit.c (pattern_search): Set notintermediate based on the
pattern.
* tests/scripts/targets/NOTINTERMEDIATE: Add a new test suite.
Previously only trivial (single-letter) options were available in
$(MAKEFLAGS) when it is examined from inside a makefile (the full
value was set before expanding recipes).
Ensure that all options (but, not command line variable overrides!)
are visible in the $(MAKEFLAGS) variable. In order to do this
reset the default values of options, particularly options which are
lists, before re-reading MAKEFLAGS after makefiles have been read.
Otherwise we'll get duplicate values for options such as -I.
Unfortunately there are complications here as well: sometimes (for
jobserver options in particular) we modify the values of these
options while running: we must not reset these modifications.
* NEWS: Announce this change
* src/main.c (main): Call new reset_switches() before re-parsing
MAKEFLAGS.
(reset_switches): Reset the value of non-special options to their
defaults.
(define_makeflags): Add non-special options into MAKEFLAGS even
if 'all' is not set.
Add a new function $(let ...) which allows lexically scoped variables.
* NEWS: Add information on this feature.
* doc/make.texi (Let Function): Document the 'let' function.
* src/function.c (func_let): Create the 'let' built-in function.
* tests/scripts/functions/let: Test the 'let' built-in function.
Add debug options to print recipes even if they would otherwise be
silent, and to print the reason that a target was considered out of
date.
Modify --trace to simply be a shorthand for --debug=print,why.
* NEWS: Announce changes.
* doc/make.texi (Summary of Options): Document the new options.
* doc/make.1: Ditto.
* src/debug.h: Add new flags DB_PRINT and DB_WHY.
* src/makeint.h: Remove the trace_flag variable.
* src/job.c (start_job_command): Check debug flags not trace_flag.
(new_job): Ditto.
* src/main.c (trace_flag): Make a static variable for switches.
(decode_debug_flags): Set DB_PRINT and DB_WHY if trace_flag is set.
* tests/scripts/variables/GNUMAKEFLAGS: Update known-good messages.
* tests/scripts/variables/MAKEFLAGS: Ditto.
Each time we invoke a command it's possible that it will change the
filesystem in ways that were not described by the target. If that
happens but we have cached previous directory contents then we may
make decisions or report results based on obsolete information.
Keep a count of how many commands we've invoked, and remember the
current command count every time we load the contents of a directory.
If we request the directory and the current command count has changed
we know the cache is outdated so reload from scratch.
* NEWS: Announce the change.
* src/makeint.h (command_count): Create a global counter.
* src/main.c (command_count): Ditto.
* src/job.c (reap_children): Increment the counter on job completion.
* src/function.c (func_file): Increment if we write a file.
* src/dir.c (clear_directory_contents): Clear the current contents of
a cached directory.
(struct directory_contents): Remember the counter value.
(struct directory): Remember the counter value for non-existing dirs.
(find_directory): If we have a cached directory and the count hasn't
changed then return it. Else, clear the previous contents and re-read
from scratch.
* tests/scripts/features/dircache: Add tests of the directory cache.
Rewrite the environment variable algorithm to correctly inherit
export settings from parent variable sets. The new algorithm
for computing the table of environment variables is:
- Start with the most local variable set and proceed to global.
- If the variable already exists in the table and we don't know
its export status, update it with the current variable's status.
- If the variable is not in the table and it's not global, add it
regardless of its status so if it's unexported we remember that.
- If the variable is not in the table and is global, check its
export status and don't add it if we won't export it.
Then when generating the environment variables, check the export
status of each variable in case it was a target-specific variable
and we have determined it should not be exported.
Rework SHELL handling to check at the end whether we added it or
not and if we didn't, add the value from the environment.
* NEWS: Announce support for target-specific "unexport"."
* doc/make.texi (Target-specific): Document the support.
* src/variable.h (enum variable_export): Make into a global type.
* src/read.c (struct vmodifiers): Use enum variable_export rather
than individual booleans.
(parse_var_assignment): Parse the "unexport" keyword.
(eval): Remember the vmodifier value in the variable.
(record_target_var): Ditto.
* src/variable.c (should_export): Check if the variable should be
exported.
(target_environment): Implement the above algorithm.
* tests/scripts/features/export: Test export/unexport with variable
assignments on the same line.
* tests/scripts/features/targetvars: Add a comprehensive suite of
tests for different types of target-specific export / unexport.
* tests/scripts/variables/SHELL: Update the comment.
GNU make must recognize some special targets as they are defined.
Because of the way targets are defined, we were not recognizing these
special targets until we were handling the NEXT statement. However
that's too late for some special targets such as .POSIX etc. which can
change the behavior of make during parsing.
Check for special targets earlier, as soon as we've finished parsing
the target introduction line (before we've even parsed the recipe).
* NEWS: Mention the change.
* src/read.c (check_specials): New function to look for special
targets. Move checks from eval() and record_files() to this new
function.
(eval): Call check_specials() after we've completed parsing the target
introduction line. Move default goal detection to check_specials().
(record_files): Move handling of .POSIX, .SECONDEXPANSION, and
.ONESHELL to check_specials().
* tests/scripts/misc/bs-nl: Remove workaround for late .POSIX issue.
* tests/scripts/targets/POSIX: Add a comment.
Ensure that makefiles are rebuilt in the order in which make first
considered them, and document this behavior in the manual.
* NEWS: Add a note about the new behavior
* doc/make.text (How make Processes a Makefile): Document it.
* main.c (main): Inverse the list of makefile goals.
* read.c (read_all_makefiles): Add default makefiles to the list at
the front in reverse order, the same way other makefiles are added.
* tests/scripts/features/include: Add tests to verify rebuild order.
Previously if --no-print-directory was seen anywhere even once
(environment, command line, etc.) it would always take precedence
over any --print-directory option. Change this so that the last
seen option (which will be the command line, if present there) takes
precedence.
* NEWS: Mark this change in behavior.
* src/makeint.h (print_directory): A new variable to control printing.
* src/output.c (output_dump): Use the new variable.
(output_start): Ditto.
* src/main.c: Add a new variable print_directory. Use -1 for
print_directory_flag so we know of the option was seen or not. Add a
new default_print_directory_flag set to -1 to keep options from being
added.
(switches): Use flag_off for --no-print-directory, rather than a
separate inhibit_print_directory_flag.
(main): If print_directory_flag was set by the user, use that for
print_directory. If not, compute the print_directory value based on
-s, -C, and sub-makes as before.
* tests/scripts/variables/GNUMAKEFLAGS: -w is not added automatically
* tests/scripts/options/print-directory: Add tests for overriding
print-directory options.
POSIX says that suffix rules cannot have prerequisites, but after
making this change we observed a number of makefiles "in the wild"
that were relying on this behavior and failed.
For .POSIX: makefiles, obey POSIX. Otherwise preserve the old
behavior. However, generate a warning so users know this is a
problem. In a future version we will change all behavior to be
POSIX-conforming.
* NEWS: describe the change
* src/rule.c (convert_to_pattern): If posix_pedantic don't make a
pattern rule if prereqs exist. Otherwise show a warning.
* tests/scripts/features/suffixrules: Add tests for the new behavior
including .POSIX vs. non-.POSIX.
Initial implementation by Christof Warlich <cwarlich@gmx.de>
* NEWS: Announce the new feature.
* doc/make.texi (Other Special Variables): Document .EXTRA_PREREQS.
* src/dep.h (struct dep): New flag to note extra prereq deps.
* src/filedef.h (expand_extra_prereqs): Declare a function to expand
the value of .EXTRA_PREREQS.
* src/file.c (expand_extra_prereqs): Given a struct variable lookup
of .EXTRA_PREREQS, convert it into a list of deps and for each one
make sure it has a struct file and has the new flag set.
(snap_file): A new function invoked by hash_map that will perform
per-file operations: set up second expansion, intermediate, and also
.EXTRA_PREREQS. Manage circular dependencies by ignoring them.
(snap_deps): Defer per-file operations until the end. Look up the
global .EXTRA_PREREQS and pass it along to snap_file for each file.
* src/implicit.c (struct patdeps): Remember the extra prereqs flag.
(pattern_search): Transfer extra prereqs flag settings into the
matched pattern rule.
* src/rule.h (snap_implicit_rules): Rename count_implicit_rules to
snap_implicit_rules since we now do more than count.
* src/rule.c (snap_implicit_rules): As we walk through all the pattern
rules, add in any global .EXTRA_PREREQS to the dep list. Ensure we
take them into account for the max number of prereqs and name length.
* src/main.c (main): Add extra-prereqs to .FEATURES.
Call the renamed snap_implicit_rules.
* tests/scripts/variables/EXTRA_PREREQS: Add tests.
This patch allows "grouped targets" using the &: syntax:
tgt1 tgt2 ... tgtn &: pre1 pre2 ...
recipe
When the &: separator is used (in single or double colon forms), all
the targets are understood to be built by a single invocation of the
recipe. This is accomplished by piggy-backing on the already-existing
pattern rule feature, using the file's "also_make" list.
* NEWS: Add information about grouped targets.
* doc/make.texi (Multiple Targets): Add information on grouped targets.
(Pattern Intro): Refer to the new section to discuss multiple patterns.
* src/main.c (main): Add "grouped-targets" to .FEATURES
* src/read.c (make_word_type): Add new types for &: and &::.
(eval): Recognize the &: and &:: separator and remember when used.
(record_files): Accept an indicator of whether the rule is grouped.
If so, update also_make for each file to depend on the other files.
(get_next_mword): Recognize the &: and &:: word types.
* tests/scripts/features/grouped_targets: New test script.
* AUTHORS: Add Kaz Kylheku
* NEWS: Update with a backward-compatibility warning.
* src/rule.c (convert_to_pattern): If a suffix rule has dependencies,
do not create a pattern rule for it. According to the manual suffix
rules with prerequisites are treated as normal targets.
* tests/scrips/features/suffixrules: Create some regression tests for
.SUFFIXES and suffix rules.
Move content from glob/* and config/* into standard GNU directory
locations lib/* and m4/*.
Install the gnulib bootstrap script and its configuration file, and
create a bootstrap.bat file for Windows. Update the README.git file
with new requirements and instructions for building from Git.
At this point we only install the alloca, getloadavg, and FDL modules
from gnulib. We keep our old glob/fnmatch implementation since the
gnulib versions require significant amounts of infrastructure which
doesn't exist on Windows yet. Further work is required here.
Due to a problem with gnulib's version of getloadavg, we need to bump
the minimum required version of automake to 1.16.1 unfortunately.
* README.git: Update instructions
* NEWS: Move developer news to a separate section
* configure.ac: Update for use with gnulib modules
* bootstrap: Bootstrap from Git workspace (import from gnulib)
* bootstrap.conf: Bootstrap configuration for GNU make
* bootstrap.bat: Bootstrap from Git workspace for Windows
* gl/modules/make-glob: Support our local fnmatch/glob implementation
* config/acinclude.m4: Move to m4/
* config/dospaths.m4: Move to m4/
* glob/fnmatch.c: Move to lib/
* glob/fnmatch.h.in: Move to lib/
* glob/glob.c: Move to lib/
* glob/glob.h.in: Move to lib/
* Makefile.am: Update for new directories
* build.template: Update for new directories
* build_w32.bat: Update for new directories
* builddos.bat: Update for new directories
* maintMakefile: Update for new directories
* makefile.com: Update for new directories
* mk/Amiga.mk: Update for new directories
* mk/Posix.mk.in: Update for new directories
* mk/VMS.mk: Update for new directories
* mk/Windows32.mk: Update for new directories
* mk/msdosdjgpp.mk: Update for new directories
* po/LINGUAS: One language per line (needed by gnulib)
* INSTALL: Remove (obtained from gnulib)
* src/alloca.c: Remove (obtained from gnulib)
* src/getloadavg.c: Remove (obtained from gnulib)
* po/Makevars: Remove (created by bootstrap)
* config/*: Remove leftover files
* glob/*: Remove leftover files
Move the source code (other than glob) into the "src" subdirectory.
Update all scripting and recommendations to support this change.
* *.c, *.h, w32/*: Move to src/
* configure.ac, Makefile.am, maintMakefile: Locate new source files.
* Basic.mk.template, mk/*: Update for new source file locations.
* NEWS, README.DOS.template: Update for new locations.
* build.template, build_w32.bat, builddos.bat: Ditto.
* po/POTFILES.in: Ditto
* tests/run_make_tests.pl, tests/scripts/features/load*: Ditto.
* make.1: Move to doc.
* mk/VMS.mk: Add support for building on VMS (hopefully).
* makefile.vms, prepare_w32.bat: Remove.
* SCOPTIONS: Update to define HAVE_CONFIG_H
Over time the non-standard build and install systems (nmake files,
smake files, Visual Studio project files, etc.) have atrophied and
maintaining them is not worth the effort, for such a simple utility
as make. Remove all the non-standard build tool support and unify
OS-specific build rules under a basic set of (GNU make) makefiles.
Preserve the existing bootstrapping scripts (for POSIX, Windows,
and MS-DOS). Also the existing VMS build scripts are left unchanged:
I don't have enough experience with VMS to venture into this area.
Perhaps one of the VMS maintainers might like to determine whether
conversion would be appropriate.
Rather than create libraries for w32 and glob (non-POSIX), simply
link the object files directly to remove the complexity.
* NEWS: Update with user-facing notes.
* Makefile.am: Clean up to use the latest automake best practices.
Build Windows code directly from the root makefile to avoid recursion.
* README.Amiga, README.DOS.template, README.W32.template: Updated.
* INSTALL: Point readers at the README.git file.
* maintMakefile: Remove obsolete files. Create Basic.mk file.
* Basic.mk.template, mk/*.mk: Create basic GNU make-based makefiles.
* build_w32.bat: Copy Basic.mk to Makefile
* configure.ac: We no longer need AM_PROG_AR.
* dosbuild.bat: Rename to builddos.bat. Incorporate configure.bat.
* Makefile.DOS.template: Remove.
* NMakefile.template, w32/subproc/NMakefile: Remove.
* SMakefile.template, glob/SMakefile, glob/SCOPTIONS, make.lnk: Remove.
* configure.bat, glob/configure.bat: Remove.
* w32/Makefile.am: Remove.
* make_msvc_net2003.sln, make_msvc_net2003.vcproj: Remove.
* doc/make.texi (Appending): Document this behavior.
* variable.c (do_variable_definition): Only add a space if the variable
value is not empty.
* tests/scripts/variables/flavors: Test this behavior.
