Commit graph

2490 commits

Author SHA1 Message Date
Paul Smith
fb779d2f1e Rework directory structure to use GNU-recommended "src" directory.
Move the source code (other than glob) into the "src" subdirectory.
Update all scripting and recommendations to support this change.

* *.c, *.h, w32/*: Move to src/
* configure.ac, Makefile.am, maintMakefile: Locate new source files.
* Basic.mk.template, mk/*: Update for new source file locations.
* NEWS, README.DOS.template: Update for new locations.
* build.template, build_w32.bat, builddos.bat: Ditto.
* po/POTFILES.in: Ditto
* tests/run_make_tests.pl, tests/scripts/features/load*: Ditto.
* make.1: Move to doc.
* mk/VMS.mk: Add support for building on VMS (hopefully).
* makefile.vms, prepare_w32.bat: Remove.
* SCOPTIONS: Update to define HAVE_CONFIG_H
2017-11-19 14:07:22 -05:00
Paul Smith
1af314465e * job.c: Add "command" as a known shell built-in.
This is not a POSIX shell built-in but it's common in UNIX shells.
Reported by Nick Bowler <nbowler@draconx.ca>.
2017-11-18 09:31:31 -05:00
Paul Smith
b80837c553 * tests/scripts/features/output-sync: Revert bad change in 0c5a9f9b92 2017-11-18 09:31:31 -05:00
Paul Smith
aa44e66c8f Remove unsupported build facilities.
Over time the non-standard build and install systems (nmake files,
smake files, Visual Studio project files, etc.) have atrophied and
maintaining them is not worth the effort, for such a simple utility
as make.  Remove all the non-standard build tool support and unify
OS-specific build rules under a basic set of (GNU make) makefiles.

Preserve the existing bootstrapping scripts (for POSIX, Windows,
and MS-DOS).  Also the existing VMS build scripts are left unchanged:
I don't have enough experience with VMS to venture into this area.
Perhaps one of the VMS maintainers might like to determine whether
conversion would be appropriate.

Rather than create libraries for w32 and glob (non-POSIX), simply
link the object files directly to remove the complexity.

* NEWS: Update with user-facing notes.
* Makefile.am: Clean up to use the latest automake best practices.
Build Windows code directly from the root makefile to avoid recursion.
* README.Amiga, README.DOS.template, README.W32.template: Updated.
* INSTALL: Point readers at the README.git file.
* maintMakefile: Remove obsolete files.  Create Basic.mk file.
* Basic.mk.template, mk/*.mk: Create basic GNU make-based makefiles.
* build_w32.bat: Copy Basic.mk to Makefile
* configure.ac: We no longer need AM_PROG_AR.
* dosbuild.bat: Rename to builddos.bat.  Incorporate configure.bat.
* Makefile.DOS.template: Remove.
* NMakefile.template, w32/subproc/NMakefile: Remove.
* SMakefile.template, glob/SMakefile, glob/SCOPTIONS, make.lnk: Remove.
* configure.bat, glob/configure.bat: Remove.
* w32/Makefile.am: Remove.
* make_msvc_net2003.sln, make_msvc_net2003.vcproj: Remove.
2017-11-18 09:31:31 -05:00
Paul Smith
a7e0dd98e4 * Makefile.am: Add jhelp.pl to remote test setup. 2017-11-11 13:12:28 -05:00
Paolo Bonzini
0c44ca2683 Do not use STOP_SET for singleton compares.
Compare against '$' directly rather than using MAP_VARIABLE.
This saves ~10% for find_map_function, which is the top hotspot in
QEMU's no-op build.  The build is sped up overall by about 1.5%
more (from 11.1s to 10.95s).

* read.c (find_map_function): Do not compare against singleton sets.
2017-11-11 10:08:30 -05:00
Paolo Bonzini
b280989ab7 Speedup parsing of functions.
Use the stopchar map to quickly jump over everything that is
not an open/close brace, an open/close parenthesis or a comma.

This saves 1% on QEMU's noop build (from 11.23s to 11.1s).

* function.c (find_next_argument, handle_function): Check
with STOP_SET before comparing against individual characters.
* main.c (initialize_stopchar_map): Initialize MAP_VARSEP
mappings in stopchar_map.
* makeint.h (MAP_VARSEP): New.
2017-11-11 10:08:30 -05:00
Paolo Bonzini
9860296198 Remove MAP_PERCENT as strchr is faster.
* read.c (find_percent_cached): Use strchr instead of STOP_SET
to find % or nul.
* makeint.h (MAP_PERCENT): Remove.
* main.c (initialize_stopchar_map): Remove.
2017-11-11 10:08:30 -05:00
Paolo Bonzini
4fd56724ad Use Jenkins hash.
This is about twice as fast as the current hash, and removes the
need for double hashing (improving locality of reference).  The
hash function is based on Bob Jenkins' design, slightly adapted
wherever Make needs to hash NUL-terminated strings.  The old hash
function is kept for case-insensitive hashing.

This saves 8.5% on QEMU's no-op build (from 12.87s to 11.78s).

* configure.ac: Check endianness.
* hash.c (rol32, jhash_mix, jhash_final, JHASH_INITVAL,
sum_get_unaligned_32, jhash): New.
* hash.h (STRING_HASH_1, STRING_N_HASH_1): Use jhash.
(STRING_HASH_2, STRING_N_HASH_2): Return a dummy value.
(STRING_N_COMPARE, return_STRING_N_COMPARE): Prefer memcmp to strncmp.
2017-11-11 10:08:30 -05:00
Paolo Bonzini
0ca31088dc Use strchr for simple case of find_char_unquote.
In most cases, find_char_unquote has a single stopchar.  In that
case we can look for it using strchr's optimized implementation.

This saves 3.5% on QEMU's noop build (from 11.78s to 11.37s).

* read.c (find_char_unquote): Rename to find_map_unquote.  Replace
with an implementation optimized for the case where the stopchar
is a singleton.  Adjust all callers.
2017-11-11 09:38:51 -05:00
Paolo Bonzini
ef7a1b7d6e Use strchr/memmove in collapse_continuations.
collapse_continuations is already using strchr to speed up the
common case of no backslash-newline sequence, but on modern
processors it is faster to scan the string twice with
strchr+memmove (or strlen+memmove) than to move bytes manually.

Saves about 1.5% on QEMU's no-op build (from 11.37s to 11.23s).

* misc.c (collapse_continuations): Rewrite the scanning of LINE.
2017-11-11 09:37:55 -05:00
Paul Smith
0c5a9f9b92 * main.c (main): [SV 48274] Allow -j in makefile MAKEFLAGS variable.
* tests/jhelp.pl: New file to allow testing parallelism without sleep.
* tests/scripts/features/parallelism: Test this.
* tests/scripts/features/jobserver: Update tests.
* tests/scripts/features/output-sync: Remove useless rm command.
2017-10-31 01:08:57 -04:00
Paul Smith
6264deece3 * job.c (child_error): Modify error message string.
Ensure Emacs compile-mode's next-error doesn't match target failure
messages.  Syntax errors in makefiles are still matched.
2017-10-30 17:07:21 -04:00
Paul Smith
b9c25de370 * function.c (func_if): Check the first character of condition.
Reported by Rob W <robw9739@gmail.com>
2017-10-30 17:07:21 -04:00
Paul Eggert
193f1e81ed glob: Do not assume glibc glob internals.
It has been proposed that glibc glob start using gl_lstat,
which the API allows it to do.  GNU 'make' should not get in
the way of this.  See:
https://sourceware.org/ml/libc-alpha/2017-09/msg00409.html

* dir.c (local_lstat): New function, like local_stat.
(dir_setup_glob): Use it to initialize gl_lstat too, as the API
requires.
2017-10-30 17:07:21 -04:00
Paul Smith
d8728efc80 Compute load from number of running processes.
* job.c (load_too_high): If /proc/loadavg is available, compare the
maximum load against the number of processes currently running.  If
we can't parse /proc/loadavg, use the previous algorithm.

Concept and initial patch from Sven C. Dack <sven.c.dack@sky.com>
2017-10-30 17:07:21 -04:00
Eli Zaretskii
baa57d2cf9 Fix checking existence of directories on MS-Windows
* remake.c (name_mtime) [WINDOWS32]: Emulate Posix behavior of
'stat' with the likes of "foo/" and "foo/.".
2017-09-30 10:56:39 +03:00
Paul Smith
7ef56bc8dc Update copyright statements for 2017. 2017-07-09 21:52:28 -04:00
Paul Smith
ac97214635 Remove uses of unnecessary "register" keyword. 2017-07-09 21:24:41 -04:00
Paul Smith
22fd578693 * variable.c (create_pattern_var): [SV 51266] Create with xcalloc().
Reported by Chris Metcalf <cmetcalf@mellanox.com>
2017-07-09 21:14:15 -04:00
Paul Smith
78b5fec689 [SV 51400] Only unblock fatal signals after child invocation
* job.c (unblock_sigs): Only unblock fatal signals not all signals.
(unblock_all_sigs): Unblock all signals not just fatal signals.
(child_execute_job): Call unblock_all_sigs() in child process.
* job.h: Remove unused function definitions.
* remote-cstms.c (start_remote_job): Call unblock_all_sigs() in
child process.
Reported by Koen Van Hoof <koen.van_hoof@nokia.com>
2017-07-09 21:14:15 -04:00
Paul Smith
75b5268faf * function.c (shell_completed): [SV 51014] Add signal to .SHELLSTATUS
* tests/scripts/functions/shell: Verify that .SHELLSTATUS contains
a non-0 value when the shell exits due to a signal.
2017-07-01 19:40:21 -04:00
Eli Zaretskii
66a9b5c3ba Avoid crashes when SHELL=abcde is specified on the command line
* variable.c (do_variable_definition): If $SHELL was not found,
process "SHELL=foo" as any other variable definition.  This avoids
segfaults when SHELL=foo is specified on the Make command line.
Reported by Orgad Shaneh <orgads@gmail.com>.
2017-06-07 08:32:35 +03:00
Paul Smith
b552b05251 [SV 51159] Use a non-blocking read with pselect to avoid hangs.
* posixos.c (set_blocking): Set blocking on a file descriptor.
(jobserver_setup): Set non-blocking on the jobserver read side.
(jobserver_parse_auth): Ditto.
(jobserver_acquire_all): Set blocking to avoid a busy-wait loop.
(jobserver_acquire): If the non-blocking read() returns without
taking a token then try again.
2017-06-04 18:37:21 -04:00
Paul Smith
5345adf82f [SV 50823] Support filenames containing '$' in MAKEFILE_LIST
* variable.h (enum variable_flavor: Add a new flavor for appended
values that shouldn't be expanded.
* variable.c (do_variable_definition): If given this new flavor,
do not expand the value before appending it.
* read.c (eval_makefile): Use this new flavor for MAKEFILE_LIST
* tests/scripts/variables/MFILE_LIST: Test filenames containing '$'.
2017-06-04 18:37:21 -04:00
Paul Smith
248f70510e * tests/test_driver.pl: Useful error if given an invalid test name. 2017-06-04 18:37:21 -04:00
Paul Smith
b90fabc8d6 * NEWS: Do not insert a space during '+=' if the value is empty.
* doc/make.texi (Appending): Document this behavior.
* variable.c (do_variable_definition): Only add a space if the variable
value is not empty.
* tests/scripts/variables/flavors: Test this behavior.
2017-06-04 18:37:21 -04:00
Enrique Olaizola
816a867ff5 * read.c (read_all_makefiles): [SV 50909] Add MAKEFILES to strcache
Copyright-paperwork-exempt: yes
2017-06-04 18:37:21 -04:00
Paul Smith
8ff529b1cb * po/LINGUAS: Add support for traditional Chinese (zh_TW) 2017-06-04 18:37:21 -04:00
Paul Smith
5acc59c704 Rename output_tmpfile() to a misc function get_tmpfile()
* output.c: Remove output_tmpfile() and umask handling.
* output.h: Ditto.
* misc.c: Add get_tmpfile() and umask handling.
* makeint.h: Ditto.
* function.c: Rename output_tmpfile() to get_tmpfile().
* main.c: Ditto.
* vmsjobs.c: Ditto.
2017-06-04 18:37:21 -04:00
Paul Smith
68be4f74fc [SV 13651] Handle out-of-memory conditions slightly more gracefully.
* makeint.h: Change OUT_OF_MEM() macro to out_of_memory() function.
* output.h, job.h: Move FD_* macros from job.h to output.h.
* output.c (output_write): Write a buffer to an FD directly.
(out_of_memory): Use output_write() to avoid allocating more
memory while writing the error, and call exit() instead of die().
This does mean we can't translate the error string, though.
* misc.c (xmalloc, xcalloc, xrealloc, xstrdup, xstrndup): Call new
out_of_memory() rather than OUT_OF_MEM().
* read.c (parse_file_seq): Ditto.
2017-06-04 18:37:20 -04:00
Paul Smith
2b8e3bb23f Clean up close-on-exec, particularly with jobserver pipes.
* configure.ac: Check sys/file.h and assume fileno() always exists.
* output.h: Move output-specific content from job.h to output.h.
* os.h (fd_inherit, fd_noinherit): New functions manage FD inheritance.
* posixos.c (fd_inherit, fd_noinherit): Implement for POSIX systems.
(jobserver_setup): Force jobserver FDs to not be inherited by default.
(jobserver_pre_child): Enable inheritance in recursive invocations.
(jobserver_post_child): Disable inheritance after recursive invocations.
* w32/w32os.c (fd_inherit, fd_noinherit): Implement for W32 systems.
* job.h (CLOSE_ON_EXEC): Remove macro in deference to new fd_noinherit.
* function.c (func_shell_base): Convert CLOSE_ON_EXEC to fd_noinherit.
* job.c (child_execute_job): Ditto.
* output.c (setup_tmpfile): Ditto.
* read.c (eval_makefile): Ditto, plus remove HAVE_FILENO check.
* w32/include/sub_proc.h: Remove process_noinherit for fd_noinherit.
* w32/subproc/sub_proc.c: Ditto.
2017-06-04 18:37:20 -04:00
Paul Smith
c4f30e6012 [SV 50300] Use CFLAGS value of "-O1" in POSIX mode.
* read.c (record_files): Use "-O1" not "-O" for CFLAGS and FFLAGS.
* tests/scripts/targets/POSIX: Test for the new value.
2017-06-04 18:37:20 -04:00
Paul Smith
fbf71ec25a * configure.ac: [SV 50648] Detect Guile 2.2 packages. 2017-06-04 18:37:20 -04:00
Paul Smith
e20c48e385 * doc/make.texi: [SV 50304] Add missing close parenthesis. 2017-06-04 18:37:20 -04:00
Paul Smith
fda00f88d3 Add test suite support to Windows
* main.c (main): Sanitize program name detection on Windows.
* makeint.h: 'program' is a const string on all platforms now.
* tests/run_make_tests.bat: Windows bat file to invoke tests
* tests/test_driver.pl: Obtain system-specific error messages.
(get_osname): Compute the $port_type here.  Add more $osname checks
for different Windows Perl ports.
(_run_command): Rewrite the timeout capability to work properly
with Windows.  Don't use Perl fork/exec; instead use system(1,...)
which allows a more reliable/proper kill operation.
Also, allow options to be given as a list instead of a string, to
allow more complex quoting of command-line arguments.
* tests/run_make_tests.pl (run_make_with_options): Allow options
to be provided as a list in addition to a simple string.
(set_more_defaults): Write sample makefiles and run make on them
instead of trying to run echo and invoking make with -f-, to avoid
relying on shell and echo to get basic configuration values.  Also
create a $sh_name variable instead of hard-coding /bin/sh.
* tests/scripts/features/archives: Skip on Windows.
* tests/scripts/features/escape: Use list method for passing options.
* tests/scripts/features/include: Use system-specific error messages.
* tests/scripts/features/output-sync: "Command not found" errors
generate very different / odd output on Windows.  This needs to be
addressed but for now disable these tests on Windows.
* tests/scripts/functions/abspath: Disable on Windows.
* tests/scripts/functions/file: Use system-specific error messages.
* tests/scripts/functions/shell: "Command not found" errors generate
very different / odd output on Windows.  This needs to be addressed
but for now disable these tests on Windows.
* tests/scripts/misc/close_stdout: Disable on Windows.
* tests/scripts/options/dash-k: Use system-specific error messages.
* tests/scripts/options/dash-l: Disable on Windows.
* tests/scripts/options/eval: Use list method for passing options.
* tests/scripts/options/general: Skip some non-portable tests.
* tests/scripts/targets/ONESHELL: Skip some non-portable tests.
* tests/scripts/targets/POSIX: Skip some non-portable tests.
* tests/scripts/variables/MAKEFILES: Skip some non-portable tests.
* tests/scripts/variables/SHELL: Use a makefile not -f- for testing.
2017-06-04 18:37:20 -04:00
Enrique Olaizola
d9d4e06084 * tests/run_make_tests.pl: [SV 50902] Find Perl modules 2017-06-04 10:19:57 -04:00
Eli Zaretskii
a95cb30ab5 [SV 50021] Avoid infloop on MS-Windows with short scripts
* job.c (reap_children) [WINDOWS32]: Avoid recursive call to
reap_children when the argument passed to
map_windows32_error_to_string is negative or too large.
2017-01-11 20:43:41 +02:00
Paul Smith
d351c1fef2 * main.c (switches): Add -E as an alias for --eval.
* make.1: Document the -E and --eval options.
* doc/make.texi: Document the -E option.
* tests/scripts/options/eval: Test the -E option and MAKEFILES.
* NEWS: Add information about the new option.
2016-12-28 09:25:51 -05:00
Paul Smith
3daaa4dd3e * main.c (switches): Add --no-silent to undo -s options.
* make.1: Document the new flag.
* doc/make.texi: Document the new flag.  Remove suggestions that the
.SILENT special target is deprecated or should not be used.
* tests/scripts/options/dash-s: Test the -s and --no-silent options.
* NEWS: Add information about the new option.
2016-12-28 09:25:34 -05:00
Martin Dorey
a359e32eb5 * job.c (child_execute_job): [SV 49938] Avoid spurious GCC warning. 2016-12-26 14:09:38 -05:00
Martin Dorey
bee153c762 * main.c (main): [SV 49935] Fix uninitialized variable. 2016-12-26 13:26:44 -05:00
Paul Smith
c5ccc4930c [SV 40236] Handle included file open failures properly.
* read.c (eval_makefile): Set deps->error if we discovered any
error reading makefiles, and set NONEXISTENT_MTIME so we know
it needs to be rebuilt.
* main.c (main): Clean up management of makefile_mtimes.
* tests/scripts/features/include: Add open failure testcases.
2016-12-26 09:01:59 -05:00
Paul Smith
45bf0e3a67 Portability changes for the test suite.
* tests/test_driver.pl: Save error strings for later comparison.
* tests/run_make_tests.pl: Create portable commands for later use.
* tests/*: Use these new variables.
2016-12-26 09:01:59 -05:00
Paul Smith
c6966b3238 [SV 20513] Un-escaped # are not comments in function invocations
* NEWS: Document the change, as a backward-incompatible change.
* main.c (main): Add 'nocomment' to the .FEATURES variable.
* read.c (remove_comments): Skip variable references during remove.
(find_char_unquote): Fix comments for new STOPMAP support.
* tests/scripts/features/escape: Test new escape syntax.
* tests/scripts/functions/guile: Ditto.
* tests/scripts/functions/shell: Ditto.
2016-12-26 09:01:59 -05:00
Paul Smith
0029ea8941 * main.c (main): [SV 40234] Show correct error message. 2016-12-26 09:01:59 -05:00
Christoph Schulz
053d49367c * main.c (switches): [SV 48809] Accept obsolete jobserver flag. 2016-12-26 09:01:59 -05:00
Paul Smith
80861cce43 * read.c (get_next_mword): [SV 49865] Make fallthrough explicit. 2016-12-26 09:01:59 -05:00
Paul Smith
72af7ce8af [SV 49114] Remove support for the NO_FLOAT compile flag.
* makeint.h (max_load_average): Always a double.
* main.c (max_load_average, default_load_average): Always doubles.
(switches): -l option is a "floating" format.
(decode_switches, define_makeflags): Support "floating" format.
* remake.c (f_mtime): Show time skew as double.
* rule.c (print_rule_data_base): Show percentages as double.
* Makefile.ami, README.DOS.template, SMakefile.template: Remove
references to NO_FLOAT.
* SCOPTIONS, glob/SCOPTIONS: Remove NO_FLOAT settings.
2016-12-26 09:01:58 -05:00
Paul Smith
e4a9c9b825 [SV 45477] [SV 49115] Parse DOS/Windows drivespecs correctly.
* read.c (parse_file_seq) [WINDOWS32]: Only consider a colon part of
a path if it's in a valid Windows drivespec.
2016-12-26 08:59:58 -05:00