Commit graph

2529 commits

Author SHA1 Message Date
Paul Smith
80ca781adf * po/LINGUAS: Add Portuguese translation 2019-05-12 16:28:26 -04:00
Paul Smith
4a31ca461f Add developer customizations
* .ccls: Configure the ccls LSP server
* .dir-locals.el: Reset some parameters for ccls and lsp-mode
* .gitignore: Update for GTAGS and ccls
* maintMakefile: Don't search hidden directories for sources
2019-05-12 16:28:26 -04:00
Paul Smith
214865ed5c * src/arscan.c (ar_member_touch): [SV 54533] Stop \0 in archive headers 2018-09-16 01:10:57 -04:00
Paul Smith
5d6508a475 Change output_write() to writebuf(), add readbuf() helper.
* src/misc.c (writebuf, readbuf): Create helper functions that will
reliably write a buffer to a file descriptor in the face of EINTR
causing short writes, and read from a file descriptor into a buffer
in the face of EINTR causing short reads.
* src/makeint.h: Declare these functions.
* src/output.c: Remove output_write() and replace with writebuf().
(_outputs, out_of_memory): Call writebuf(), not output_write().
* src/arscan.c (ar_scan): Call readbuf() instead of read(2).
(ar_member_touch): Remove duplicate header write, call writebuf()
instead of output_write(), and readbuf() instead of read(2).
2018-09-16 01:04:10 -04:00
Paul Smith
ac11ec5497 * doc/make.texi (Remaking Makefiles): [SV 52273] Note MAKE_RESTARTS 2018-09-15 16:12:44 -04:00
Paul Smith
d603b2e6ac * src/main.c (main): Set jobserver permissions before re-execing
Fixes an issue seen in the Linux kernel build system, reported by
Masahiro Yamada <yamada.masahiro@socionext.com>.  Fix suggested
on the bug-make mailing list by Mike Shal <marfey@gmail.com>.
* tests/scripts/features/parallelism: Add a test to verify this.
2018-09-15 15:08:47 -04:00
Paul Smith
8fd614e238 * doc/make.texi: [SV 48970] Clarify the value of $? 2018-09-15 15:08:47 -04:00
Paul Smith
92032bc16a * doc/make.text: [SV 54360] Weaken "obsolete" language in the manual.
There's no need to frighten users away from capabilities which are
perfectly reasonable and definitely not going away, by calling them
obsolete or discouraged in the manual.
2018-09-15 15:08:47 -04:00
Paul Smith
67e6d81256 [SV 40657] Don't create pattern rules for suffix rules with deps.
* NEWS: Update with a backward-compatibility warning.
* src/rule.c (convert_to_pattern): If a suffix rule has dependencies,
do not create a pattern rule for it.  According to the manual suffix
rules with prerequisites are treated as normal targets.
* tests/scrips/features/suffixrules: Create some regression tests for
.SUFFIXES and suffix rules.
2018-09-15 15:08:29 -04:00
Paul Smith
a1bb739165 * NEWS: Update for the latest changes. 2018-08-04 19:20:58 -04:00
Paul Smith
05769ca009 [SV 54233] Preserve higher command_state values on also_make targets.
If multiple pattern rules have the same pattern as also-make targets
and we attempt to run them at the same time, we might downgrade the
command state from 'running' to 'deps_running'; this will prevent
that also_make from being considered complete causing make to wait
forever for it to finish.

Ensure that set_command_state never downgrades the state of a target.

* src/file.c (set_command_state): Don't downgrade command_state.
* src/filedef.h (struct file): Document the order prerequisite.
* test/scripts/features/patternrules: Test the behavior.
2018-08-04 19:07:59 -04:00
Paul Smith
e2234bc1ab * src/job.c (reap_children): Fix inverted win/lose message. 2018-08-04 19:05:29 -04:00
Paul Smith
f9ba22e029 * tests/scripts/functions/wildcard: [SV 52018] Test dangling symlink. 2018-08-04 17:35:10 -04:00
Bernhard M. Wiedemann
eedea52afb * src/read.c (parse_file_seq): [SV 52076] Sort wildcard results. 2018-08-04 17:21:15 -04:00
Paul Smith
a9975756d0 * tests/scripts/features/archives: [SV 54395] Test long archive names. 2018-08-04 17:02:25 -04:00
Ben Hutchings
a37fa47e32 * src/arscan.c (ar_scan): [SV 54395] Allow long names in archives.
Commit bc9d72beb0 "Resolve issues discovered by static code
analysis." added range checks on archive member name length.  However,
on non-AIX systems it also checked BSD-style long names against the
short name limits and and checked the *offset* for GNU-style long
names against the short name limits.  This caused valid long names to
be rejected.

* Record the size of the GNU name map and validate offsets against it
* Ensure that the last entry in the name map is null-terminated
* Apply a maximum length of INT_MAX for element sizes

Reported-by: Philipp Wolski <philipp.wolski@kisters.de>
2018-08-04 17:02:21 -04:00
Paul Smith
9ff4d6af92 * lib/glob.c (glob_in_dir): [SV 53465] Allow symlinks to directories.
Fix from Rich Felker <bugdal@aerifal.cx> on the musl mailing list.
* tests/scripts/functions/wildcard: Create a regression test for this.
2018-08-04 14:26:10 -04:00
Paul Smith
a370268739 * configure.ac: Add --disable-posix-spawn option
* maintMakefile: Add a test for the option
* src/job.c: Change HAVE_* preprocessor checks to USE_POSIX_SPAWN
2018-08-04 12:37:19 -04:00
Paul Smith
1129df27b8 * src/job.c (child_execute_job): Clean up posix_spawn invocation 2018-08-04 12:37:19 -04:00
Aron Barath
749a54d7a4 * job.c (child_execute_job): Prefer posix_spawn() over fork()/exec() 2018-08-04 12:37:19 -04:00
Paul Smith
fa937343f5 Clean up errors for invalid commands and add regression tests.
* src/function.c (func_shell_base): Use error() instead of recreating
the error output.
* src/job.c (exec_command): Show more standard error messages.
* src/load.c (unload_file): Fix whitespace in the error message.
* tests/scripts/features/errors: Add tests for starting non-
existent commands and new error message formats.
* tests/scripts/features/output-sync: New error message formats.
* tests/scripts/functions/shell: Ditto.
2018-08-04 12:37:19 -04:00
Paul Smith
3112c87993 Queue failed fork() (etc.) to be handled like any other failed job.
If we failed to fork() we were essentially exiting make immediately
without respect to ignore flags, etc.  On one hand that makes sense
because if you can't fork you're in real trouble, but it doesn't
work so well on systems where we don't fork at all.  Instead, treat
a fork error like any other error by delaying the handling until
the next call to reap_children().  Any child with a PID of -1 is
considered to have died before starting so check these first without
waiting for them.

* src/commands.c (fatal_error_signal): Don't kill children that
never started.
* src/function.c (func_shell_base): Handle cleanup properly if the
child doesn't start.
* src/job.c (reap_children): Check for children that died before
starting and handle them without waiting for the PID.
(start_job_command): Free memory when the child doesn't start.
(start_waiting_job): Don't manage children who never started.
(child_execute_job): If the fork fails return PID -1.
* src/vmsjobs.c: Check for children that never started.
* tests/run_make_tests.pl: Parse config.status to get all options.
2018-08-04 12:18:39 -04:00
Aron Barath
8bde7db16f * configure.ac: Check for posix_spawnattr_setsigmask 2018-08-02 20:59:02 -04:00
Aron Barath
5d985c2851 * configure.ac: Check for spawn.h and posix_spawn() 2018-08-02 20:59:02 -04:00
Aron Barath
3f194b69e3 * src/makeint.h: Use pid_t to store PIDs, of int.
* src/commands.c (getpid): Ditto.
* src/job.h (*): Ditto.
* src/job.c (*): Ditto.
* src/main.c (main): Ditto.
* src/remote-cstms.c (start_remote_job): Ditto.
* src/remote-stub.c (start_remote_job): Ditto.
2018-08-02 20:59:02 -04:00
spagoveanu@gmail.com
b7acb10e86 * src/dir.c: Preserve glob d_type field
When using GNU make on a system with glibc glob a pattern ending in
a slash is also matching regular files, but only in subdirectories:

$ mkdir -p dir/subdir
$ cd dir
$ touch file1 subdir/file2
$ echo 'test:; @echo $(wildcard */ */*/)' | make -f -
subdir/ subdir/file2
$ echo 'test: */ */*/; @echo "$?" != */ */*/' | make -f -
subdir/ subdir/file2 != subdir/ */*/

It happens because in the gl->gl_readdir callback supplied to glob(),
dirent->d_type is set to DT_UNKNOWN, and the glob() implementation
in glibc assumes that such a directory entry *cannot* possibly be a
regular file.

Pass the actual d_type down to glob(); this is the right thing to do
even if glibc is fixed, because it saves an extra stat() syscall for
each dirent.

Copyright-paperwork-exempt: yes
2018-08-02 20:59:02 -04:00
Paul Smith
b36a8ec92a * src/read.c(unescape_char): Use C comments not C++ comments.
* src/posixos.c(set_blocking): Ditto.
* src/w32/subproc/sub_proc.c(process_init): Ditto
2018-08-02 20:59:02 -04:00
Paul Smith
23554ee658 Enhance the Basic.mk environment to work with Gnulib
* maintMakefile: Omit generated headers from Basic.mk prerequisites.
* Basic.mk.template: Rework commands to use with $(call ...) macros.
* mk/Amiga.mk: Ditto.
* mk/VMS.mk: Ditto.
* mk/Windows32.mk: Ditto, plus P2W to convert POSIX to Windows paths.
* mk/msdosdjgpp.mk: Ditto.
2018-08-02 20:55:34 -04:00
Paul Smith
0ce880e5ee Resolve most of the Windows Visual Studio warnings.
* Convert integer types to size_t where necessary.
* Align other integral types to avoid casts and type warnings.
2018-07-02 07:54:07 -04:00
Paul Smith
c808f10d08 Update regression tests for Windows.
* tests/scripts/features/jobserver: Windows doesn't use pipes
* tests/scripts/functions/shell: Don't test kill -2 on Windows
* tests/scripts/misc/bs-nl: Windows doesn't handle single quotes
* tests/scripts/misc/general3: Ditto.
2018-07-02 07:54:07 -04:00
Paul Smith
1dfd55ca36 * maintMakefile: Preserve comments during compilation 2018-07-02 07:54:07 -04:00
Paul Smith
0bc699a761 * function.c(abspath): Use memcpy() to avoid GCC 8.1 warnings 2018-07-02 07:54:07 -04:00
Paul Smith
63b42fa235 Convert GNU make to use the gnulib portability library
Move content from glob/* and config/* into standard GNU directory
locations lib/* and m4/*.

Install the gnulib bootstrap script and its configuration file, and
create a bootstrap.bat file for Windows.  Update the README.git file
with new requirements and instructions for building from Git.

At this point we only install the alloca, getloadavg, and FDL modules
from gnulib.  We keep our old glob/fnmatch implementation since the
gnulib versions require significant amounts of infrastructure which
doesn't exist on Windows yet.  Further work is required here.

Due to a problem with gnulib's version of getloadavg, we need to bump
the minimum required version of automake to 1.16.1 unfortunately.

* README.git: Update instructions
* NEWS: Move developer news to a separate section
* configure.ac: Update for use with gnulib modules

* bootstrap: Bootstrap from Git workspace (import from gnulib)
* bootstrap.conf: Bootstrap configuration for GNU make
* bootstrap.bat: Bootstrap from Git workspace for Windows
* gl/modules/make-glob: Support our local fnmatch/glob implementation

* config/acinclude.m4: Move to m4/
* config/dospaths.m4: Move to m4/
* glob/fnmatch.c: Move to lib/
* glob/fnmatch.h.in: Move to lib/
* glob/glob.c: Move to lib/
* glob/glob.h.in: Move to lib/

* Makefile.am: Update for new directories
* build.template: Update for new directories
* build_w32.bat: Update for new directories
* builddos.bat: Update for new directories
* maintMakefile: Update for new directories
* makefile.com: Update for new directories
* mk/Amiga.mk: Update for new directories
* mk/Posix.mk.in: Update for new directories
* mk/VMS.mk: Update for new directories
* mk/Windows32.mk: Update for new directories
* mk/msdosdjgpp.mk: Update for new directories

* po/LINGUAS: One language per line (needed by gnulib)

* INSTALL: Remove (obtained from gnulib)
* src/alloca.c: Remove (obtained from gnulib)
* src/getloadavg.c: Remove (obtained from gnulib)
* po/Makevars: Remove (created by bootstrap)
* config/*: Remove leftover files
* glob/*: Remove leftover files
2018-07-02 07:54:07 -04:00
Paul Smith
a7b0dfddac * Makefile.ami, glob/Makefile.ami: Remove obsolete files 2018-07-01 13:25:16 -04:00
Ola Olsson
73ac4526b9 * doc/make.texi: Updated language 2018-07-01 12:23:09 -04:00
Paul Smith
afbecbd9dc * all: Update Copyright statements for 2018 2018-07-01 12:22:04 -04:00
Paul Smith
192469952f * scripts/copyright-update: Maintainer's script for copyright mgmt 2018-07-01 12:22:03 -04:00
Paul Smith
8a731d1b2c * build_w32.bat: Support Visual Studio 17. 2018-02-10 15:34:53 -05:00
Paul Smith
48c8a116a9 * configure.ac: Support GLIBC glob interface version 2 2017-11-19 15:17:47 -05:00
Paul Smith
fb779d2f1e Rework directory structure to use GNU-recommended "src" directory.
Move the source code (other than glob) into the "src" subdirectory.
Update all scripting and recommendations to support this change.

* *.c, *.h, w32/*: Move to src/
* configure.ac, Makefile.am, maintMakefile: Locate new source files.
* Basic.mk.template, mk/*: Update for new source file locations.
* NEWS, README.DOS.template: Update for new locations.
* build.template, build_w32.bat, builddos.bat: Ditto.
* po/POTFILES.in: Ditto
* tests/run_make_tests.pl, tests/scripts/features/load*: Ditto.
* make.1: Move to doc.
* mk/VMS.mk: Add support for building on VMS (hopefully).
* makefile.vms, prepare_w32.bat: Remove.
* SCOPTIONS: Update to define HAVE_CONFIG_H
2017-11-19 14:07:22 -05:00
Paul Smith
1af314465e * job.c: Add "command" as a known shell built-in.
This is not a POSIX shell built-in but it's common in UNIX shells.
Reported by Nick Bowler <nbowler@draconx.ca>.
2017-11-18 09:31:31 -05:00
Paul Smith
b80837c553 * tests/scripts/features/output-sync: Revert bad change in 0c5a9f9b92 2017-11-18 09:31:31 -05:00
Paul Smith
aa44e66c8f Remove unsupported build facilities.
Over time the non-standard build and install systems (nmake files,
smake files, Visual Studio project files, etc.) have atrophied and
maintaining them is not worth the effort, for such a simple utility
as make.  Remove all the non-standard build tool support and unify
OS-specific build rules under a basic set of (GNU make) makefiles.

Preserve the existing bootstrapping scripts (for POSIX, Windows,
and MS-DOS).  Also the existing VMS build scripts are left unchanged:
I don't have enough experience with VMS to venture into this area.
Perhaps one of the VMS maintainers might like to determine whether
conversion would be appropriate.

Rather than create libraries for w32 and glob (non-POSIX), simply
link the object files directly to remove the complexity.

* NEWS: Update with user-facing notes.
* Makefile.am: Clean up to use the latest automake best practices.
Build Windows code directly from the root makefile to avoid recursion.
* README.Amiga, README.DOS.template, README.W32.template: Updated.
* INSTALL: Point readers at the README.git file.
* maintMakefile: Remove obsolete files.  Create Basic.mk file.
* Basic.mk.template, mk/*.mk: Create basic GNU make-based makefiles.
* build_w32.bat: Copy Basic.mk to Makefile
* configure.ac: We no longer need AM_PROG_AR.
* dosbuild.bat: Rename to builddos.bat.  Incorporate configure.bat.
* Makefile.DOS.template: Remove.
* NMakefile.template, w32/subproc/NMakefile: Remove.
* SMakefile.template, glob/SMakefile, glob/SCOPTIONS, make.lnk: Remove.
* configure.bat, glob/configure.bat: Remove.
* w32/Makefile.am: Remove.
* make_msvc_net2003.sln, make_msvc_net2003.vcproj: Remove.
2017-11-18 09:31:31 -05:00
Paul Smith
a7e0dd98e4 * Makefile.am: Add jhelp.pl to remote test setup. 2017-11-11 13:12:28 -05:00
Paolo Bonzini
0c44ca2683 Do not use STOP_SET for singleton compares.
Compare against '$' directly rather than using MAP_VARIABLE.
This saves ~10% for find_map_function, which is the top hotspot in
QEMU's no-op build.  The build is sped up overall by about 1.5%
more (from 11.1s to 10.95s).

* read.c (find_map_function): Do not compare against singleton sets.
2017-11-11 10:08:30 -05:00
Paolo Bonzini
b280989ab7 Speedup parsing of functions.
Use the stopchar map to quickly jump over everything that is
not an open/close brace, an open/close parenthesis or a comma.

This saves 1% on QEMU's noop build (from 11.23s to 11.1s).

* function.c (find_next_argument, handle_function): Check
with STOP_SET before comparing against individual characters.
* main.c (initialize_stopchar_map): Initialize MAP_VARSEP
mappings in stopchar_map.
* makeint.h (MAP_VARSEP): New.
2017-11-11 10:08:30 -05:00
Paolo Bonzini
9860296198 Remove MAP_PERCENT as strchr is faster.
* read.c (find_percent_cached): Use strchr instead of STOP_SET
to find % or nul.
* makeint.h (MAP_PERCENT): Remove.
* main.c (initialize_stopchar_map): Remove.
2017-11-11 10:08:30 -05:00
Paolo Bonzini
4fd56724ad Use Jenkins hash.
This is about twice as fast as the current hash, and removes the
need for double hashing (improving locality of reference).  The
hash function is based on Bob Jenkins' design, slightly adapted
wherever Make needs to hash NUL-terminated strings.  The old hash
function is kept for case-insensitive hashing.

This saves 8.5% on QEMU's no-op build (from 12.87s to 11.78s).

* configure.ac: Check endianness.
* hash.c (rol32, jhash_mix, jhash_final, JHASH_INITVAL,
sum_get_unaligned_32, jhash): New.
* hash.h (STRING_HASH_1, STRING_N_HASH_1): Use jhash.
(STRING_HASH_2, STRING_N_HASH_2): Return a dummy value.
(STRING_N_COMPARE, return_STRING_N_COMPARE): Prefer memcmp to strncmp.
2017-11-11 10:08:30 -05:00
Paolo Bonzini
0ca31088dc Use strchr for simple case of find_char_unquote.
In most cases, find_char_unquote has a single stopchar.  In that
case we can look for it using strchr's optimized implementation.

This saves 3.5% on QEMU's noop build (from 11.78s to 11.37s).

* read.c (find_char_unquote): Rename to find_map_unquote.  Replace
with an implementation optimized for the case where the stopchar
is a singleton.  Adjust all callers.
2017-11-11 09:38:51 -05:00
Paolo Bonzini
ef7a1b7d6e Use strchr/memmove in collapse_continuations.
collapse_continuations is already using strchr to speed up the
common case of no backslash-newline sequence, but on modern
processors it is faster to scan the string twice with
strchr+memmove (or strlen+memmove) than to move bytes manually.

Saves about 1.5% on QEMU's no-op build (from 11.37s to 11.23s).

* misc.c (collapse_continuations): Rewrite the scanning of LINE.
2017-11-11 09:37:55 -05:00