Fixes an issue seen in the Linux kernel build system, reported by
Masahiro Yamada <yamada.masahiro@socionext.com>. Fix suggested
on the bug-make mailing list by Mike Shal <marfey@gmail.com>.
* tests/scripts/features/parallelism: Add a test to verify this.
There's no need to frighten users away from capabilities which are
perfectly reasonable and definitely not going away, by calling them
obsolete or discouraged in the manual.
* NEWS: Update with a backward-compatibility warning.
* src/rule.c (convert_to_pattern): If a suffix rule has dependencies,
do not create a pattern rule for it. According to the manual suffix
rules with prerequisites are treated as normal targets.
* tests/scrips/features/suffixrules: Create some regression tests for
.SUFFIXES and suffix rules.
If multiple pattern rules have the same pattern as also-make targets
and we attempt to run them at the same time, we might downgrade the
command state from 'running' to 'deps_running'; this will prevent
that also_make from being considered complete causing make to wait
forever for it to finish.
Ensure that set_command_state never downgrades the state of a target.
* src/file.c (set_command_state): Don't downgrade command_state.
* src/filedef.h (struct file): Document the order prerequisite.
* test/scripts/features/patternrules: Test the behavior.
Commit bc9d72beb0 "Resolve issues discovered by static code
analysis." added range checks on archive member name length. However,
on non-AIX systems it also checked BSD-style long names against the
short name limits and and checked the *offset* for GNU-style long
names against the short name limits. This caused valid long names to
be rejected.
* Record the size of the GNU name map and validate offsets against it
* Ensure that the last entry in the name map is null-terminated
* Apply a maximum length of INT_MAX for element sizes
Reported-by: Philipp Wolski <philipp.wolski@kisters.de>
* src/function.c (func_shell_base): Use error() instead of recreating
the error output.
* src/job.c (exec_command): Show more standard error messages.
* src/load.c (unload_file): Fix whitespace in the error message.
* tests/scripts/features/errors: Add tests for starting non-
existent commands and new error message formats.
* tests/scripts/features/output-sync: New error message formats.
* tests/scripts/functions/shell: Ditto.
If we failed to fork() we were essentially exiting make immediately
without respect to ignore flags, etc. On one hand that makes sense
because if you can't fork you're in real trouble, but it doesn't
work so well on systems where we don't fork at all. Instead, treat
a fork error like any other error by delaying the handling until
the next call to reap_children(). Any child with a PID of -1 is
considered to have died before starting so check these first without
waiting for them.
* src/commands.c (fatal_error_signal): Don't kill children that
never started.
* src/function.c (func_shell_base): Handle cleanup properly if the
child doesn't start.
* src/job.c (reap_children): Check for children that died before
starting and handle them without waiting for the PID.
(start_job_command): Free memory when the child doesn't start.
(start_waiting_job): Don't manage children who never started.
(child_execute_job): If the fork fails return PID -1.
* src/vmsjobs.c: Check for children that never started.
* tests/run_make_tests.pl: Parse config.status to get all options.
When using GNU make on a system with glibc glob a pattern ending in
a slash is also matching regular files, but only in subdirectories:
$ mkdir -p dir/subdir
$ cd dir
$ touch file1 subdir/file2
$ echo 'test:; @echo $(wildcard */ */*/)' | make -f -
subdir/ subdir/file2
$ echo 'test: */ */*/; @echo "$?" != */ */*/' | make -f -
subdir/ subdir/file2 != subdir/ */*/
It happens because in the gl->gl_readdir callback supplied to glob(),
dirent->d_type is set to DT_UNKNOWN, and the glob() implementation
in glibc assumes that such a directory entry *cannot* possibly be a
regular file.
Pass the actual d_type down to glob(); this is the right thing to do
even if glibc is fixed, because it saves an extra stat() syscall for
each dirent.
Copyright-paperwork-exempt: yes
* maintMakefile: Omit generated headers from Basic.mk prerequisites.
* Basic.mk.template: Rework commands to use with $(call ...) macros.
* mk/Amiga.mk: Ditto.
* mk/VMS.mk: Ditto.
* mk/Windows32.mk: Ditto, plus P2W to convert POSIX to Windows paths.
* mk/msdosdjgpp.mk: Ditto.
* tests/scripts/features/jobserver: Windows doesn't use pipes
* tests/scripts/functions/shell: Don't test kill -2 on Windows
* tests/scripts/misc/bs-nl: Windows doesn't handle single quotes
* tests/scripts/misc/general3: Ditto.
Move content from glob/* and config/* into standard GNU directory
locations lib/* and m4/*.
Install the gnulib bootstrap script and its configuration file, and
create a bootstrap.bat file for Windows. Update the README.git file
with new requirements and instructions for building from Git.
At this point we only install the alloca, getloadavg, and FDL modules
from gnulib. We keep our old glob/fnmatch implementation since the
gnulib versions require significant amounts of infrastructure which
doesn't exist on Windows yet. Further work is required here.
Due to a problem with gnulib's version of getloadavg, we need to bump
the minimum required version of automake to 1.16.1 unfortunately.
* README.git: Update instructions
* NEWS: Move developer news to a separate section
* configure.ac: Update for use with gnulib modules
* bootstrap: Bootstrap from Git workspace (import from gnulib)
* bootstrap.conf: Bootstrap configuration for GNU make
* bootstrap.bat: Bootstrap from Git workspace for Windows
* gl/modules/make-glob: Support our local fnmatch/glob implementation
* config/acinclude.m4: Move to m4/
* config/dospaths.m4: Move to m4/
* glob/fnmatch.c: Move to lib/
* glob/fnmatch.h.in: Move to lib/
* glob/glob.c: Move to lib/
* glob/glob.h.in: Move to lib/
* Makefile.am: Update for new directories
* build.template: Update for new directories
* build_w32.bat: Update for new directories
* builddos.bat: Update for new directories
* maintMakefile: Update for new directories
* makefile.com: Update for new directories
* mk/Amiga.mk: Update for new directories
* mk/Posix.mk.in: Update for new directories
* mk/VMS.mk: Update for new directories
* mk/Windows32.mk: Update for new directories
* mk/msdosdjgpp.mk: Update for new directories
* po/LINGUAS: One language per line (needed by gnulib)
* INSTALL: Remove (obtained from gnulib)
* src/alloca.c: Remove (obtained from gnulib)
* src/getloadavg.c: Remove (obtained from gnulib)
* po/Makevars: Remove (created by bootstrap)
* config/*: Remove leftover files
* glob/*: Remove leftover files
Move the source code (other than glob) into the "src" subdirectory.
Update all scripting and recommendations to support this change.
* *.c, *.h, w32/*: Move to src/
* configure.ac, Makefile.am, maintMakefile: Locate new source files.
* Basic.mk.template, mk/*: Update for new source file locations.
* NEWS, README.DOS.template: Update for new locations.
* build.template, build_w32.bat, builddos.bat: Ditto.
* po/POTFILES.in: Ditto
* tests/run_make_tests.pl, tests/scripts/features/load*: Ditto.
* make.1: Move to doc.
* mk/VMS.mk: Add support for building on VMS (hopefully).
* makefile.vms, prepare_w32.bat: Remove.
* SCOPTIONS: Update to define HAVE_CONFIG_H
Over time the non-standard build and install systems (nmake files,
smake files, Visual Studio project files, etc.) have atrophied and
maintaining them is not worth the effort, for such a simple utility
as make. Remove all the non-standard build tool support and unify
OS-specific build rules under a basic set of (GNU make) makefiles.
Preserve the existing bootstrapping scripts (for POSIX, Windows,
and MS-DOS). Also the existing VMS build scripts are left unchanged:
I don't have enough experience with VMS to venture into this area.
Perhaps one of the VMS maintainers might like to determine whether
conversion would be appropriate.
Rather than create libraries for w32 and glob (non-POSIX), simply
link the object files directly to remove the complexity.
* NEWS: Update with user-facing notes.
* Makefile.am: Clean up to use the latest automake best practices.
Build Windows code directly from the root makefile to avoid recursion.
* README.Amiga, README.DOS.template, README.W32.template: Updated.
* INSTALL: Point readers at the README.git file.
* maintMakefile: Remove obsolete files. Create Basic.mk file.
* Basic.mk.template, mk/*.mk: Create basic GNU make-based makefiles.
* build_w32.bat: Copy Basic.mk to Makefile
* configure.ac: We no longer need AM_PROG_AR.
* dosbuild.bat: Rename to builddos.bat. Incorporate configure.bat.
* Makefile.DOS.template: Remove.
* NMakefile.template, w32/subproc/NMakefile: Remove.
* SMakefile.template, glob/SMakefile, glob/SCOPTIONS, make.lnk: Remove.
* configure.bat, glob/configure.bat: Remove.
* w32/Makefile.am: Remove.
* make_msvc_net2003.sln, make_msvc_net2003.vcproj: Remove.
Compare against '$' directly rather than using MAP_VARIABLE.
This saves ~10% for find_map_function, which is the top hotspot in
QEMU's no-op build. The build is sped up overall by about 1.5%
more (from 11.1s to 10.95s).
* read.c (find_map_function): Do not compare against singleton sets.
Use the stopchar map to quickly jump over everything that is
not an open/close brace, an open/close parenthesis or a comma.
This saves 1% on QEMU's noop build (from 11.23s to 11.1s).
* function.c (find_next_argument, handle_function): Check
with STOP_SET before comparing against individual characters.
* main.c (initialize_stopchar_map): Initialize MAP_VARSEP
mappings in stopchar_map.
* makeint.h (MAP_VARSEP): New.
This is about twice as fast as the current hash, and removes the
need for double hashing (improving locality of reference). The
hash function is based on Bob Jenkins' design, slightly adapted
wherever Make needs to hash NUL-terminated strings. The old hash
function is kept for case-insensitive hashing.
This saves 8.5% on QEMU's no-op build (from 12.87s to 11.78s).
* configure.ac: Check endianness.
* hash.c (rol32, jhash_mix, jhash_final, JHASH_INITVAL,
sum_get_unaligned_32, jhash): New.
* hash.h (STRING_HASH_1, STRING_N_HASH_1): Use jhash.
(STRING_HASH_2, STRING_N_HASH_2): Return a dummy value.
(STRING_N_COMPARE, return_STRING_N_COMPARE): Prefer memcmp to strncmp.
In most cases, find_char_unquote has a single stopchar. In that
case we can look for it using strchr's optimized implementation.
This saves 3.5% on QEMU's noop build (from 11.78s to 11.37s).
* read.c (find_char_unquote): Rename to find_map_unquote. Replace
with an implementation optimized for the case where the stopchar
is a singleton. Adjust all callers.
collapse_continuations is already using strchr to speed up the
common case of no backslash-newline sequence, but on modern
processors it is faster to scan the string twice with
strchr+memmove (or strlen+memmove) than to move bytes manually.
Saves about 1.5% on QEMU's no-op build (from 11.37s to 11.23s).
* misc.c (collapse_continuations): Rewrite the scanning of LINE.
It has been proposed that glibc glob start using gl_lstat,
which the API allows it to do. GNU 'make' should not get in
the way of this. See:
https://sourceware.org/ml/libc-alpha/2017-09/msg00409.html
* dir.c (local_lstat): New function, like local_stat.
(dir_setup_glob): Use it to initialize gl_lstat too, as the API
requires.
