When the `jj-proc-macros` crate was introduced, it triggered an underlying
bug in `nextest`, which is the test harness we use in the Nix build. This is
upstream Nextest bug 267. The long and short of it is that `rustc` fails to
find needed libraries whenever the proc macros are loaded.
This can easily be worked around however, by setting
`DYLD_FALLBACK_LIBRARY_PATH` to an appropriate value in the devShell and in the
`preCheck` phase of the main expression.
Signed-off-by: Austin Seipp <aseipp@pobox.com>
When running the `nix build`, the `buildRustPackage` function -- which builds
the `jj` crates -- calls `cargo build --release` with flags like `HOST_CXX`
set. This is called the `buildPhase`. Then, it runs the `checkPhase`, which
calls `cargo nextest`, in our case. However, it does not set `HOST_CXX`, for
some reason.
The intent of `buildRustPackage` is that the `buildPhase` and `checkPhase`
have their compilation options fully aligned so that they reuse the local cargo
`target/` cache directory, avoiding a full recompilation. However, the lack
of `HOST_CXX` above among others causes a cache miss, and a bunch of cascading
recompilations. The net impact is that we compile all of the codebase once in
`buildPhase`, then again in `checkPhase`, making the Nix CI build 2x slower on
average than the other Linux runners; 2-3 minutes versus 7 minutes, on average.
Instead, re-introduce a 'check' into the Flake directly, which overrides the
`jujustsu` package, but stubs out the `buildPhase`. This means it will only run
`checkPhase`, which is what we want. Then, modify the CI to run `nix flake check`
again, like it used to in the past.
Unfortunately, this doesn't fix the cache miss when running `nix build`
yourself, it recompiles from scratch in both phases still. That needs to be
fixed in the future, but it's tolerable for now.
This reverts most of 71a3045032.
Signed-off-by: Austin Seipp <aseipp@pobox.com>
This matches the behavior of the actual `nix build` more closely, and might also
help Anton, since he was debugging some recompilation issues on his machine,
where `RUSTFLAGS` might have become inconsistent due to VS Code.
Signed-off-by: Austin Seipp <aseipp@pobox.com>
Summary: Pure QOL improvement, not that this was too awful as it stood. But this
ultimately makes all builds faster, effectively for free; on my Linux machine,
debug link time for jj-cli goes from 3s to 1s.
This was originally submitted as part of #1864, and backed out as part of #1982,
because the default `mold` package did not have the proper "wrapper script"
that allowed it to find the right external libraries. Using the `mold-wrapped`
package however fixes this.
On top of that, let's go ahead and make `mold` the default linker in the Nix
package/flake as well; this should improve the `nix build` speed since a lot of
tests need to be linked.
Signed-off-by: Austin Seipp <aseipp@pobox.com>
Recognize signature metadata from git commit objects, implement
a basic version of that for the native backend.
Extract the signed data (a commit binary repr without the signature) to
be verified later.
Summary: Without `devShell` providing the needed Darwin-specific inputs, `cargo
build` does not work inside a `nix develop` or `direnv` environment; libgit2 in
particular fails on being able to find the Security framework.
The actual `nix build` invocation however *does* work because we correctly
include those dependencies in the package `buildInputs`. So just factor them
out, and use them in both places.
Signed-off-by: Austin Seipp <aseipp@pobox.com>
Change-Id: I484bf381ca31c29c4c39fb6d184bdd21
Summary: Apparently this was broken. Maybe I broke it. Maybe something upstream
changed and caused a regression. But without it, we get the stable `rustfmt` in
the `nix develop` shell environment, not the nightly version. Fix it.
Signed-off-by: Austin Seipp <aseipp@pobox.com>
Change-Id: I602ed8e5691c4d48f8db575d62624955
Summary: Since 066032b6e6 was merged, the `nix flake check` build no longer
overrides the 'cargo test' profile explicitly, to save disk space. The CI seems
to be in a better spot. This will stem the tide for a while hopefully.
However, with that change in place, the `nix flake check` build was
essentially a redundant, nearly-identical copy of a normal `nix build` with no
differentiating features, except: `RUST_BACKTRACE` is set to 1.
Delete all this code, and remove it from the CI matrix, and instead just export
`RUST_BACKTRACE` on the `checkPhase` of the normal `nix build` instead, which is
functionally equivalent.
Also does some minor, no-functional-change touchups to `flake.nix` while I was
there (whitespace, etc.)
Signed-off-by: Austin Seipp <aseipp@pobox.com>
Change-Id: I87336b16e2a0b973343ecbde8ffd7b8f
Summary: The Nix CI has been failing recently due to (what I assume is) disk
space issues. But only the `flake check` step is failing. Right now, `nix flake
check` runs the Cargo tests with the debug profile to help get more debug info,
which is even heftier in terms of debug info than the normal 'test' profile. For
reference, a single build of 'cargo test' in a clean working copy results in a
15 gigabyte `target/` directory.
Turn off the debug profile for `nix flake check`, which should hopefully stem
the bleeding a bit. I believe the 'test' profile should still have enough
symbols for backtraces, so panics should still be useful.
Signed-off-by: Austin Seipp <aseipp@pobox.com>
Change-Id: Idde10ac15847a1ad1e6f4e48a2497eca
Summary: By default, and to be explicit, the local flake turns off default
features. But, 70f6e0a turned off watchman support for the Nix build by
accident. Enable default features so this doesn't happen again.
Signed-off-by: Austin Seipp <aseipp@pobox.com>
Change-Id: Ib7dd935eceab328fa5b0333b10368377
This should allow Nix users to build the documentation in
`nix develop` following the instructions in `contributing.md`.
It feels a little weird to add a package manager and ask
people to use it inside nix, but I think it's simpler
than having two sets of instructions and dependencies
to maintain.
Summary: mold apparently breaks linking to several external libraries, meaning
that the `jj` binary is useless as it can't load e.g. `libgit2.so` -- but
somehow I didn't catch it. Backout until a solution can be pinpointed. We can at
least give off a sigh of relief knowing that our link times aren't too bad yet.
Signed-off-by: Austin Seipp <aseipp@pobox.com>
Change-Id: I6e1851c339ddc72d32c201a44a7ddc49
Summary: Let's be more aggressive about tracking the latest stable Rust release.
There's little benefit to being conservative so early on, especially when no
users seem to have faced any issue with upgrading, or strictly required an old
Rust version.
Right now, just lagging Rust by 1 major release probably seems fine. We're
targeting 1.71.0 to get ahead of the curve, since 1.72.0 will likely release
sometime before the next `jj` release.
Signed-off-by: Austin Seipp <aseipp@pobox.com>
Change-Id: I4e691b6ba63b5b9023a75ae0a6917672
Summary: Pure QOL improvement, not that this was too awful as it stood. But this
ultimately makes all builds faster, effectively for free; on my Linux machine,
debug link time for jj-cli goes from 3s to 1s.
Signed-off-by: Austin Seipp <aseipp@pobox.com>
Change-Id: I5a0d516363635710733e8bee8f208050
Summary: Nix packages tend to come "full extra pack of batteries"-included, so
adding watchman support makes sense. I'll also want it for my own experiments.
Signed-off-by: Austin Seipp <aseipp@pobox.com>
Change-Id: Ia99757bf7a39814529525efe423d527b
Summary: When building in CI, we just build everything from scratch in the `dev`
profile, so just turn off debuginfo and incremental compilation data. These
aren't reused, but more importantly take up more space and CPU time to generate;
that's more expensive on slower machines and when using The Cloud(TM) to build.
Together, these take a fresh build (`rm -rf target/` ahead of time) from about
40s to 30s on my machine.
Signed-off-by: Austin Seipp <aseipp@pobox.com>
Change-Id: Id6e706634dbaf88f6ccd7f457409ad7c
I wasn't quite happy with `jj support` but I couldn't think of
anything better when I moved the commands from `jj debug` in
e2b4d7058d. Thanks to @ilyagr for suggesting `jj util`.
We need 1.64 to bump `clap` to `4.1`. We don't really need to upgrade
to that, but being on an older version causes minor confusions like
#1393. Rust 1.64 is very close to 6 months old at this point.
The `jj debug` commands are hidden from help and are described as
"Low-level commands not intended for users", but e.g. `jj debug
completion` is intended for users, and should be visible in the help
output.
We don't actually need `jj` itself in the development shell. But having it in
the overlay significantly increases the time it takes to build the shell.
This gets used by `nix develop`, or automatically by `direnv` if you have it
installed.
The binary versions are pinned to the versions recommended by `contributing.md`.
```
>> cargo --version
cargo 1.60.0 (d1fd9fe 2022-03-01)
>> rustc --version
rustc 1.60.0 (7737e0b5c 2022-04-04)
>> cargo fmt --version
rustfmt 1.5.1-nightly (3984bc5 2023-01-17)
>> cargo clippy --version
clippy 0.1.60 (7737e0b 2022-04-04)
```
Otherwise nix 2.8 and newer will give this error message:
```
>> nix --version; nix run github:martinvonz/jj
nix (Nix) 2.11.1
error: attribute 'defaultApp.x86_64-linux' should have type 'derivation'
```
As mentioned in the previous commit, we need to remove the Protobuf
dependency in order to be allowed to import jj into Google's
repo. This commit makes `SimpleOpStore` store its data using Thrift
instead of Protobufs. It also adds automatic upgrade of existing
repos. The upgrade process took 18 s in my repo, which has 22k
operations. The upgraded storage uses practically the same amount of
space. `jj op log` (the full outut) in my repo slowed down from 1.2 s
to 3.4 s. Luckily that's an uncommon operation. I couldn't measure any
difference in `jj status` (loading a single operation).
`pkgconfig` has been a alias to `pkg-config` since 2021-01-18, and has started being an error since 2022-09-24.
This commit makes recent nixpkgs versions work, but might make version older than January 2021 break (which is probably not an issue).
I was able to build a working musl binary with this change, by running
this command:
```
cargo build --release --target x86_64-unknown-linux-musl
```
Thanks to @arxanas for the tip.
