Commit graph

2618 commits

Author SHA1 Message Date
Yuya Nishihara
1572c251ef revset: add positions() iterator to InternalRevset
I just wanted to clean up the callers, but this might also be marginally
faster.
2024-02-22 23:26:29 +09:00
Yuya Nishihara
33c7e18ac8 revset: flip ordering of generic combination iterators
As a general-purpose iterator combinator, ascending order makes more sense.
2024-02-22 23:26:29 +09:00
Yuya Nishihara
22933563e8 revset: extract generic combination iterators
I'm going to add pre-filtering to the 'roots..heads' evaluation path, and
difference_by() will be used there to calculate 'heads ~ roots'.

Union and intersection iterators are slightly changed so that all iterators
prioritize iter1's item.
2024-02-22 23:26:29 +09:00
Julien Vincent
f97e929cbf sign: Skip gpg tests if gpg is not installed
This adds a guard to the gpg signing tests which will skip the test if
`gpg` is not installed on the system.

This is done in order to avoid requiring all collaborators to have setup
all the tools on their local machines that are required to test commit
signing.
2024-02-21 13:22:53 +00:00
Yuya Nishihara
9f05aa8c46 tests: fix fun typo "singing" -> "signing" 2024-02-21 22:04:41 +09:00
Yuya Nishihara
e3d2ff2b75 signing: change default gpg program, add --keyid-format option accordingly
This is the default of Git, and Debian sid doesn't install the gpg2 symlink
by default.

https://github.com/git/git/blob/v2.43.2/gpg-interface.c#L92
https://github.com/martinvonz/jj/pull/3007#discussion_r1496877808
https://packages.debian.org/bookworm/gnupg2
2024-02-21 22:04:41 +09:00
Austin Seipp
6c31bab0d3 fsmonitor: allow core.fsmonitor = "none" to disable
When doing things like testing snapshot performance differences,
this allows you to turn off the monitor, no matter what the enabled
user or repository configuration has, e.g.

    jj st --config-toml='core.fsmonitor="none"'

Signed-off-by: Austin Seipp <aseipp@pobox.com>
2024-02-20 20:19:47 -06:00
Evan Mesterhazy
79518eafce Output better error messages when deriving ContentHash for an enum fails
Consider this code:
```
struct NoContentHash {}

#[derive(ContentHash)]
enum Hashable {
    NoCanHash(NoContentHash),
    Empty,
}
```

Before this commit, it generates an error like this:
```
error[E0277]: the trait bound `NoContentHash: ContentHash` is not satisfied
   --> lib/src/content_hash.rs:150:10
    |
150 | #[derive(ContentHash)]
    |          ^^^^^^^^^^^ the trait `ContentHash` is not implemented for `NoContentHash`
151 | enum Hashable {
152 |     NoCanHash(NoContentHash),
    |     --------- required by a bound introduced by this call
    |
    = help: the following other types implement trait `ContentHash`:
              bool
              i32
              i64
              u8
              u32
              u64
              std::collections::HashMap<K, V>
              BTreeMap<K, V>
            and 35 others

For more information about this error, try `rustc --explain E0277`.
```

After this commit, it generates a better error message:
```
error[E0277]: the trait bound `NoContentHash: ContentHash` is not satisfied
   --> lib/src/content_hash.rs:152:15
    |
152 |     NoCanHash(NoContentHash),
    |               ^^^^^^^^^^^^^ the trait `ContentHash` is not implemented for `NoContentHash`
    |
    = help: the following other types implement trait `ContentHash`:
              bool
              i32
              i64
              u8
              u32
              u64
              std::collections::HashMap<K, V>
              BTreeMap<K, V>
            and 35 others

For more information about this error, try `rustc --explain E0277`.
error: could not compile `jj-lib` (lib) due to 1 previous error
```

It also works for enum variants with named fields:
```
error[E0277]: the trait bound `NoContentHash: ContentHash` is not satisfied
   --> lib/src/content_hash.rs:152:23
    |
152 |     NoCanHash { named: NoContentHash },
    |                       ^^^^^^^^^^^^^ the trait `ContentHash` is not implemented for `NoContentHash`
    |
    = help: the following other types implement trait `ContentHash`:
              bool
              i32
              i64
              u8
              u32
              u64
              std::collections::HashMap<K, V>
              BTreeMap<K, V>
            and 35 others

For more information about this error, try `rustc --explain E0277`.
```
2024-02-20 16:29:25 -05:00
Evan Mesterhazy
e8f324ffde Replace uses of content_hash! with #[derive(ContentHash)]
This is a pure refactor with no behavior changes.

#3054
2024-02-20 14:18:13 -05:00
Evan Mesterhazy
966a5505e2 Add support for deriving ContentHash for Enums
Here's an example of what the derived output looks like for an enum:

```rust
pub enum TreeValue {
    File { id: FileId, executable: bool },
    Symlink(SymlinkId),
    Tree(TreeId),
    GitSubmodule(CommitId),
    Conflict(ConflictId),
}
#[automatically_derived]
impl ::jj_lib::content_hash::ContentHash for TreeValue {
    fn hash(&self, state: &mut impl digest::Update) {
        match self {
            Self::File { id, executable } => {
                state.update(&0u32.to_le_bytes());
                ::jj_lib::content_hash::ContentHash::hash(id, state);
                ::jj_lib::content_hash::ContentHash::hash(executable, state);
            }
            Self::Symlink(field_0) => {
                state.update(&1u32.to_le_bytes());
                ::jj_lib::content_hash::ContentHash::hash(field_0, state);
            }
            Self::Tree(field_0) => {
                state.update(&2u32.to_le_bytes());
                ::jj_lib::content_hash::ContentHash::hash(field_0, state);
            }
            Self::GitSubmodule(field_0) => {
                state.update(&3u32.to_le_bytes());
                ::jj_lib::content_hash::ContentHash::hash(field_0, state);
            }
            Self::Conflict(field_0) => {
                state.update(&4u32.to_le_bytes());
                ::jj_lib::content_hash::ContentHash::hash(field_0, state);
            }
        }
    }
}
```


#3054
2024-02-20 12:59:35 -05:00
Evan Mesterhazy
8e1a6c708f Add support for generics to #[derive(ContentHash)]
#3054
2024-02-20 12:48:25 -05:00
Daehyeok Mun
a9f489ccdf Switch to ignore crate for gitignore handling.
Co-authored-by: Waleed Khan <me@waleedkhan.name>
2024-02-20 09:12:46 -08:00
Evan Mesterhazy
965d6ce4e4 Implement a procedural macro to derive the ContentHash trait for structs
This is a no-op in terms of function, but provides a nicer way to derive the
ContentHash trait for structs using the `#[derive(ContentHash)]` syntax used
for other traits such as `Debug`.

This commit only adds the macro. A subsequent commit will replace uses of
`content_hash!{}` with `#[derive(ContentHash)]`.

The new macro generates nice error messages, just like the old macro:

```
error[E0277]: the trait bound `NotImplemented: content_hash::ContentHash` is not satisfied
   --> lib/src/content_hash.rs:265:16
    |
265 |             z: NotImplemented,
    |                ^^^^^^^^^^^^^^ the trait `content_hash::ContentHash` is not implemented for `NotImplemented`
    |
    = help: the following other types implement trait `content_hash::ContentHash`:
              bool
              i32
              i64
              u8
              u32
              u64
              std::collections::HashMap<K, V>
              BTreeMap<K, V>
            and 38 others
```

This commit does two things to make proc macros re-exported by jj_lib useable
by deps:

1. jj_lib needs to be able refer to itself as `jj_lib` which it does
   by adding an `extern crate self as jj_lib` declaration.

2. jj_lib::content_hash needs to re-export the `digest::Update` type so that
   users of jj_lib can use the `#[derive(ContentHash)]` proc macro without
   directly depending on the digest crate. This is done by re-exporting it
   as `DigestUpdate`.


#3054
2024-02-20 11:29:05 -05:00
Ilya Grigoriev
106483ad6a clippy: run nightly cargo clippy --fix 2024-02-19 23:38:33 -08:00
Martin von Zweigbergk
11c67cf979 op_store: add metadata flag for ops representing working-copy snapshot
It should be useful at least in the presentation layer to know which
operations correspond to working-copy snapshots. They might be
rendered differently in the graph, for example. Or maybe an undo
command wants to warn if you just undid a snapshot operation. This
patch just introduces a field in the metadata to store the
information.
2024-02-19 22:44:38 -08:00
Julien Vincent
23e5fba737 sign: Add SSH backend tests 2024-02-20 00:02:08 +00:00
Julien Vincent
5e24677301 sign: Implement SSH signing backend 2024-02-20 00:02:08 +00:00
Julien Vincent
7c11a61c23 sign: GPG backend tests 2024-02-20 00:02:08 +00:00
Anton Bulakh
0efaef2da9 sign: Implement GPG signing backend
Now it is actually possible to set GPG as the main backend and have jj
"preserving" signatures on rewrites. Just no way to make signatures yet
2024-02-20 00:02:08 +00:00
Martin von Zweigbergk
3f1d75f518 rewrite: default to not simplifying ancestor merges
This means auto-rebase will no longer simplify ancestor merges.
2024-02-19 14:20:18 -08:00
Martin von Zweigbergk
a9d0300b11 rewrite: make simplification of ancestor merges optional
I think the conclusion from #2600 is that at least auto-rebasing
should not simplify merge commits that merge a commit with its
ancestor. Let's start by adding an option for that in the library.
2024-02-19 14:20:18 -08:00
Yuya Nishihara
0c0eb37f2e index: don't store commit ids in sorted lookup table to save disk space
This reduces the index file size. In my linux mirror repo containing 1591524
commits, the initial index file shrank from 122MB to 92MB. In theory, this
makes commit id lookup slow because of additional indirection and cache miss,
but I don't see significant difference. In mid-size repo, this is actually a
bit faster thanks to smaller index reads.

Alternatively, the commit id field could be removed from the CommitGraphEntry,
but doing that would introduce indirect lookup there, and the index disk size
isn't as small as this change.

- jj-0 baseline                         122MB
- jj-1 shrink CommitLookupEntry (this)   92MB
- jj-3 shrink CommitGraphEntry           98MB

Mid-size repo, "log" with default template
```
% hyperfine --sort command --warmup 3 --runs 20 -L bin jj-0,jj-1,jj-2,jj-3 \
  -s "target/release-with-debug/{bin} -R ~/mirrors/linux debug reindex" \
  "target/release-with-debug/{bin} -R ~/mirrors/linux --ignore-working-copy log -r.. -l100 --config-toml='revsets.short-prefixes=\"\"'"
Benchmark 1: target/release-with-debug/jj-0 -R ~/mirrors/linux --ignore-working-copy log -r.. -l100 --config-toml='revsets.short-prefixes=""'
  Time (mean ± σ):     177.7 ms ±  12.9 ms    [User: 96.3 ms, System: 81.5 ms]
  Range (min … max):   156.8 ms … 191.2 ms    20 runs

Benchmark 2: target/release-with-debug/jj-1 -R ~/mirrors/linux --ignore-working-copy log -r.. -l100 --config-toml='revsets.short-prefixes=""'
  Time (mean ± σ):     169.8 ms ±  13.8 ms    [User: 93.3 ms, System: 76.6 ms]
  Range (min … max):   151.1 ms … 191.5 ms    20 runs

Benchmark 4: target/release-with-debug/jj-3 -R ~/mirrors/linux --ignore-working-copy log -r.. -l100 --config-toml='revsets.short-prefixes=""'
  Time (mean ± σ):     170.3 ms ±  13.4 ms    [User: 90.1 ms, System: 79.7 ms]
  Range (min … max):   154.8 ms … 186.2 ms    20 runs

Relative speed comparison
        1.05 ±  0.11  target/release-with-debug/jj-0 -R ~/mirrors/linux --ignore-working-copy log -r.. -l100 --config-toml='revsets.short-prefixes=""'
        1.00          target/release-with-debug/jj-1 -R ~/mirrors/linux --ignore-working-copy log -r.. -l100 --config-toml='revsets.short-prefixes=""'
        1.00 ±  0.11  target/release-with-debug/jj-3 -R ~/mirrors/linux --ignore-working-copy log -r.. -l100 --config-toml='revsets.short-prefixes=""'
```

Small repo, "log" thousands of commits with -T"commit_id.shortest()"
```
% hyperfine --sort command --warmup 3 --runs 100 -L bin jj-0,jj-1,jj-2,jj-3 \
  -s "target/release-with-debug/{bin} -R ~/mirrors/git debug reindex" \
  "target/release-with-debug/{bin} -R ~/mirrors/git --ignore-working-copy log -r.. -l5000 -T'commit_id.shortest()' --config-toml='revsets.short-prefixes=\"\"'"
Benchmark 1: target/release-with-debug/jj-0 -R ~/mirrors/git --ignore-working-copy log -r.. -l5000 -T'commit_id.shortest()' --config-toml='revsets.short-prefixes=""'
  Time (mean ± σ):     179.3 ms ±  12.8 ms    [User: 149.7 ms, System: 29.6 ms]
  Range (min … max):   155.2 ms … 191.0 ms    100 runs

Benchmark 2: target/release-with-debug/jj-1 -R ~/mirrors/git --ignore-working-copy log -r.. -l5000 -T'commit_id.shortest()' --config-toml='revsets.short-prefixes=""'
  Time (mean ± σ):     179.1 ms ±  13.7 ms    [User: 148.5 ms, System: 30.5 ms]
  Range (min … max):   157.2 ms … 196.7 ms    100 runs

Benchmark 4: target/release-with-debug/jj-3 -R ~/mirrors/git --ignore-working-copy log -r.. -l5000 -T'commit_id.shortest()' --config-toml='revsets.short-prefixes=""'
  Time (mean ± σ):     178.2 ms ±  13.6 ms    [User: 148.7 ms, System: 29.6 ms]
  Range (min … max):   156.5 ms … 191.7 ms    100 runs

Relative speed comparison
        1.01 ±  0.11  target/release-with-debug/jj-0 -R ~/mirrors/git --ignore-working-copy log -r.. -l5000 -T'commit_id.shortest()' --config-toml='revsets.short-prefixes=""'
        1.01 ±  0.11  target/release-with-debug/jj-1 -R ~/mirrors/git --ignore-working-copy log -r.. -l5000 -T'commit_id.shortest()' --config-toml='revsets.short-prefixes=""'
        1.01 ±  0.11  target/release-with-debug/jj-3 -R ~/mirrors/git --ignore-working-copy log -r.. -l5000 -T'commit_id.shortest()' --config-toml='revsets.short-prefixes=""'
```
2024-02-19 11:36:45 +09:00
Vladimir Petrzhikovskii
06d67f02d8 cli: list new remote branches during git fetch 2024-02-18 17:36:01 +01:00
Yuya Nishihara
a1b16c5583 index: build reachable change ids set lazily
Instead of abstracting RevWalk over borrowed/Arc-ed index types, I decided to
implement bitset-based ancestor traversal. It's simpler and probably faster so
long as the set isn't sparse.

"jj log" without working copy snapshot:
```
% hyperfine --sort command --warmup 3 --runs 20 -L bin jj-0,jj-1,jj-2 \
  -s "target/release-with-debug/{bin} -R ~/mirrors/linux debug reindex" \
  "target/release-with-debug/{bin} -R ~/mirrors/linux \
   --ignore-working-copy log -r.. -l100 --config-toml='revsets.short-prefixes=\"\"'"
Benchmark 2: target/release-with-debug/jj-1 -R ~/mirrors/linux --ignore-working-copy log -r.. -l100 --config-toml='revsets.short-prefixes=""'
  Time (mean ± σ):     271.3 ms ±   9.9 ms    [User: 183.8 ms, System: 87.7 ms]
  Range (min … max):   250.5 ms … 282.7 ms    20 runs

Benchmark 3: target/release-with-debug/jj-2 -R ~/mirrors/linux --ignore-working-copy log -r.. -l100 --config-toml='revsets.short-prefixes=""'
  Time (mean ± σ):     177.5 ms ±  12.6 ms    [User: 94.6 ms, System: 82.9 ms]
  Range (min … max):   154.4 ms … 188.7 ms    20 runs

Relative speed comparison
        1.53 ±  0.12  target/release-with-debug/jj-1 -R ~/mirrors/linux --ignore-working-copy log -r.. -l100 --config-toml='revsets.short-prefixes=""'
        1.00          target/release-with-debug/jj-2 -R ~/mirrors/linux --ignore-working-copy log -r.. -l100 --config-toml='revsets.short-prefixes=""'
```

"jj status" with working copy snapshot (watchman enabled):
```
% hyperfine --sort command --warmup 3 --runs 20 -L bin jj-0,jj-1,jj-2 \
  -s "target/release-with-debug/{bin} -R ~/mirrors/linux debug reindex" \
  "target/release-with-debug/{bin} -R ~/mirrors/linux \
   status --config-toml='revsets.short-prefixes=\"\"'"
Benchmark 2: target/release-with-debug/jj-1 -R ~/mirrors/linux status --config-toml='revsets.short-prefixes=""'
  Time (mean ± σ):     318.6 ms ±  12.6 ms    [User: 219.1 ms, System: 94.1 ms]
  Range (min … max):   294.2 ms … 333.0 ms    20 runs

Benchmark 3: target/release-with-debug/jj-2 -R ~/mirrors/linux status --config-toml='revsets.short-prefixes=""'
  Time (mean ± σ):     214.7 ms ±  15.0 ms    [User: 117.4 ms, System: 96.1 ms]
  Range (min … max):   198.4 ms … 243.3 ms    20 runs

Relative speed comparison
        1.48 ±  0.12  target/release-with-debug/jj-1 -R ~/mirrors/linux status --config-toml='revsets.short-prefixes=""'
        1.00          target/release-with-debug/jj-2 -R ~/mirrors/linux status --config-toml='revsets.short-prefixes=""'
```
2024-02-19 00:54:43 +09:00
Yuya Nishihara
adcb01ef95 index: move RevWalk tests to inner module
The main tests module is getting bigger, and these tests are very specific
to the RevWalk* implementations.
2024-02-19 00:54:43 +09:00
Yuya Nishihara
924a5fc842 index: inline entry size calculation
There aren't many callers now, and using self.commit_id_length might help
compiler remove redundant bounds checking in CommitLookupEntry.
2024-02-19 00:47:46 +09:00
Yuya Nishihara
d5c75da4f5 index: precompute base data offsets
These offsets are getting messier, so let's calculate them in one place. This
will probably help compiler optimization.
2024-02-19 00:47:46 +09:00
Yuya Nishihara
3c7aa75b9b index: switch to persistent change id index
The shortest change id prefix will become a few digits longer, but I think
that's acceptable. Entries included in the "revsets.short-prefixes" set are
unaffected.

The reachable set is calculated eagerly, but this is still faster as we no
longer need to sort the reachable entries by change id. The lazy version will
save another ~100ms in mid-size repos.

"jj log" without working copy snapshot:
```
% hyperfine --sort command --warmup 3 --runs 20 -L bin jj-0,jj-1,jj-2 \
  -s "target/release-with-debug/{bin} -R ~/mirrors/linux debug reindex" \
  "target/release-with-debug/{bin} -R ~/mirrors/linux \
   --ignore-working-copy log -r.. -l100 --config-toml='revsets.short-prefixes=\"\"'"
Benchmark 1: target/release-with-debug/jj-0 -R ~/mirrors/linux --ignore-working-copy log -r.. -l100 --config-toml='revsets.short-prefixes=""'
  Time (mean ± σ):     353.6 ms ±  11.9 ms    [User: 266.7 ms, System: 87.0 ms]
  Range (min … max):   329.0 ms … 365.6 ms    20 runs

Benchmark 2: target/release-with-debug/jj-1 -R ~/mirrors/linux --ignore-working-copy log -r.. -l100 --config-toml='revsets.short-prefixes=""'
  Time (mean ± σ):     271.3 ms ±   9.9 ms    [User: 183.8 ms, System: 87.7 ms]
  Range (min … max):   250.5 ms … 282.7 ms    20 runs

Relative speed comparison
        1.99 ±  0.16  target/release-with-debug/jj-0 -R ~/mirrors/linux --ignore-working-copy log -r.. -l100 --config-toml='revsets.short-prefixes=""'
        1.53 ±  0.12  target/release-with-debug/jj-1 -R ~/mirrors/linux --ignore-working-copy log -r.. -l100 --config-toml='revsets.short-prefixes=""'
```

"jj status" with working copy snapshot (watchman enabled):
```
% hyperfine --sort command --warmup 3 --runs 20 -L bin jj-0,jj-1,jj-2 \
  -s "target/release-with-debug/{bin} -R ~/mirrors/linux debug reindex" \
  "target/release-with-debug/{bin} -R ~/mirrors/linux \
   status --config-toml='revsets.short-prefixes=\"\"'"
Benchmark 1: target/release-with-debug/jj-0 -R ~/mirrors/linux status --config-toml='revsets.short-prefixes=""'
  Time (mean ± σ):     396.6 ms ±  10.1 ms    [User: 300.7 ms, System: 94.0 ms]
  Range (min … max):   373.6 ms … 408.0 ms    20 runs

Benchmark 2: target/release-with-debug/jj-1 -R ~/mirrors/linux status --config-toml='revsets.short-prefixes=""'
  Time (mean ± σ):     318.6 ms ±  12.6 ms    [User: 219.1 ms, System: 94.1 ms]
  Range (min … max):   294.2 ms … 333.0 ms    20 runs

Relative speed comparison
        1.85 ±  0.14  target/release-with-debug/jj-0 -R ~/mirrors/linux status --config-toml='revsets.short-prefixes=""'
        1.48 ±  0.12  target/release-with-debug/jj-1 -R ~/mirrors/linux status --config-toml='revsets.short-prefixes=""'
```
2024-02-18 09:44:57 +09:00
Yuya Nishihara
5f3a31300b index: implement index-level change id lookup methods
These methods are basically the same as the commit_id versions, but
resolve_change_id_prefix() is a bit more involved as we need to gather matches
from multiple segments.
2024-02-18 09:44:57 +09:00
Yuya Nishihara
f73e590837 index: implement segment-level change id lookup methods
In resolve_change_id_prefix(), I've implemented two different ways of
collecting the overflow items. I don't think they impact the performance,
but we can switch to the alternative method as needed.
2024-02-18 09:44:57 +09:00
Yuya Nishihara
8cdf6d752c index: move change ids to sstable, build change-id-to-pos lookup table
This basically means that the change ids are interned. We'll implement binary
search over the sorted change ids table. The table could be sorted differently
for better cache locality, but it is in lexicographical order for simplicity.
With my testing, the cost of the id lookup isn't dominant.

Unlike the parent entries, the size of the per-id overflow items isn't saved.
That's s because the number of the same-change-id commits is either 1 or many.
It doesn't make sense to allocate 8 bytes for each change id. Instead, we'll
pay extra indirection cost to determine the size.
2024-02-18 09:44:57 +09:00
Yuya Nishihara
9974a46327 index: clarify parent entries are global positions
I'm going to add change id overflow table whose elements are of LocalPosition
type. Let's make sure that the serialization code would break if we changed
the underlying data type.
2024-02-18 09:44:57 +09:00
Thomas Castiglione
aaa5d6bc4f working_copy: add Send supertrait
If WorkingCopy: Send, then Workspace is Send, which is useful for long-running
servers. All existing impls are Send already, so this is just a marker.
2024-02-17 15:13:25 +08:00
Yuya Nishihara
5eea88d26a tests: fix concurrent git read/write test to retry on ref lock contention
Apparently, gix has 100ms timeout. Since this test tries to create contended
situation, it's possible that the ref lock can't be acquired. I've added
upper bound to the retry loop at b37293fa68 "tests: add upper bound to
test_concurrent_read_write_commit() loop", so ignoring arbitrary errors
should be okay.

The problem can be reproduced on my Linux machine by inserting 10ms sleep() to
gix and increasing the concurrency.

Fixes #3069
2024-02-17 15:09:27 +09:00
Yuya Nishihara
ce295f8bc2 op_store: remove unneeded repr(u8) from RemoteRefState
It no longer makes sense after e1fd402d39 "Fix the ContentHash implementations
for std::Option, MergedTreeId, and RemoteRefState."
2024-02-17 02:13:44 +09:00
Yuya Nishihara
718d080e7a index: make reindexing message less scary 2024-02-17 01:45:23 +09:00
Evan Mesterhazy
a80d0183a2 Implement ContentHash for u32 and u64
This is for completeness and to avoid accidents such as someone calling
`ContentHash::hash(1234u32.to_le_bytes())` and expecting it to hash properly as
a u32 instead of a 4 byte slice, which produces a different hash due to hashing
the length of the slice before its contents.
2024-02-16 10:23:39 -05:00
Evan Mesterhazy
e1fd402d39 Fix the ContentHash implementations for std::Option, MergedTreeId, and RemoteRefState
The `ContentHash` documentation specifies that implementations for enums should
hash the ordinal number of the variant contained in the enum as a 32-bit
little-endian number and then hash the contents of the variant, if any.

The current implementations for `std::Option`, `MergedTreeId`, and
`RemoteRefState` are non-conformant since they hash the ordinal number as a u8
with platform specific endianness.


Fixes #3051
2024-02-16 09:27:32 -05:00
Yuya Nishihara
903f18acfd index: extract helper functions for id lookup in mutable table
Similar to the previous commit, these functions will be reused by the change id
lookup methods. The return value isn't cloned because resolve_id_prefix() will
return (key, value) pair, and the current caller doesn't need a cloned value.
2024-02-16 11:12:53 +09:00
Yuya Nishihara
000cb41c7e index: extract helper struct for post processing binary search result
This code will be shared among commit id and change id lookup functions.
2024-02-16 11:12:53 +09:00
Yuya Nishihara
6fa660d9a8 index: extract inner binary search function
The callback returns Ordering instead of &[u8] due to lifetime difficulty.
2024-02-16 11:12:53 +09:00
Yuya Nishihara
2e64bf83fd index: pass bytes prefix to binary search function
This helps extract common binary search helper to be used by change id index.
2024-02-14 23:34:47 +09:00
Yuya Nishihara
91a68b950d index: adjust binary search function to conform to std behavior
This removes redundant case from resolve_neighbor_commit_ids(). The returned
position should never be lower than the prefix id.

The implementation is basically a copy of slice::binary_search_by(). We still
use (low + high) / 2 as the size wouldn't exceed 2^31.

https://github.com/rust-lang/rust/blob/1.76.0/library/core/src/slice/mod.rs#L2825
2024-02-14 23:34:47 +09:00
Yuya Nishihara
e2c8a8fabd index: fix change id resolution test to not depend on deterministic order
Since IdIndex sorts the entries by using .sort_unstable_by_key(), the order of
the same-key elements is undefined. Perhaps, it's stable for short arrays, and
the test passes because of that.
2024-02-14 23:22:23 +09:00
Yuya Nishihara
8b1dfa7157 index: compact parent encoding, inline up to two parents
This saves 4 more bytes per entry, and more importantly, most commit parents
can be resolved with no indirection to the overflow table.

IIRC, Git always inlines the first parent, but that wouldn't be useful in jj
since jj diffs merge commit against the auto-merge parent. The first merge
parent is nothing special.

I'll use a similar encoding in change id sstable, where only one position
will be inlined (to optimize for imported commits.)

Benchmark number measuring the cost of change id index building:
```
% hyperfine --sort command --warmup 3 --runs 20 -L bin jj-0,jj-1 \
  -s "target/release-with-debug/{bin} -R ~/mirrors/linux \
      --ignore-working-copy debug reindex" \
  "target/release-with-debug/{bin} -R ~/mirrors/linux \
    --ignore-working-copy log -r@ --config-toml='revsets.short-prefixes=\"\"'"
Benchmark 1: target/release-with-debug/jj-0 -R ~/mirrors/linux --ignore-working-copy log -r@ --config-toml='revsets.short-prefixes=""'
  Time (mean ± σ):     342.9 ms ±  14.5 ms    [User: 202.4 ms, System: 140.6 ms]
  Range (min … max):   326.6 ms … 360.6 ms    20 runs

Benchmark 2: target/release-with-debug/jj-1 -R ~/mirrors/linux --ignore-working-copy log -r@ --config-toml='revsets.short-prefixes=""'
  Time (mean ± σ):     325.0 ms ±  13.6 ms    [User: 196.2 ms, System: 128.8 ms]
  Range (min … max):   311.6 ms … 343.2 ms    20 runs

Relative speed comparison
        1.06 ±  0.06  target/release-with-debug/jj-0 -R ~/mirrors/linux --ignore-working-copy log -r@ --config-toml='revsets.short-prefixes=""'
        1.00          target/release-with-debug/jj-1 -R ~/mirrors/linux --ignore-working-copy log -r@ --config-toml='revsets.short-prefixes=""'
```
2024-02-14 22:33:48 +09:00
Yuya Nishihara
89928ffdd8 index: remove local-global pos round trip from entry_by_id() 2024-02-14 22:33:48 +09:00
Yuya Nishihara
249449ff1a index: store local position in lookup table 2024-02-14 22:33:48 +09:00
Yuya Nishihara
1d11cffcfa index: use local position in segment-local operations
I'm going to change the index format to store local positions in the lookup
table. That's not super important, but I think it makes sense because the
lookup table should never contain inter-segment links.

The mutable segment now stores local positions in its lookup map. The readonly
segment will be updated later.
2024-02-14 22:33:48 +09:00
Yuya Nishihara
3e43108abb index: remove unused flag field from readonly index segment
This is remainder of fdb861b957 "backend: remove unused Commit::is_pruned."
As I'm going to change the index format, let's remove unused fields, too.
2024-02-14 22:33:48 +09:00
Yuya Nishihara
26528091e6 revset: drop now unused is_legacy flag from dag ranges 2024-02-14 10:04:56 +09:00