This basically means that the change ids are interned. We'll implement binary
search over the sorted change ids table. The table could be sorted differently
for better cache locality, but it is in lexicographical order for simplicity.
With my testing, the cost of the id lookup isn't dominant.
Unlike the parent entries, the size of the per-id overflow items isn't saved.
That's s because the number of the same-change-id commits is either 1 or many.
It doesn't make sense to allocate 8 bytes for each change id. Instead, we'll
pay extra indirection cost to determine the size.
I'm going to add change id overflow table whose elements are of LocalPosition
type. Let's make sure that the serialization code would break if we changed
the underlying data type.
When adding a new workspace, I would expect that it inherits the
patterns from the workspace I ran the command in. We currently don't
do that. That's quite annoying when your repo has very many files
(like at Google).
`graphlog::Edge` is used somewhat inconsistently. I've replaced `Edge::Present`
with two distinct `Edge::Direct` and `Edge::Indirect` which simplifies the
construction of the enum.
Apparently, gix has 100ms timeout. Since this test tries to create contended
situation, it's possible that the ref lock can't be acquired. I've added
upper bound to the retry loop at b37293fa68 "tests: add upper bound to
test_concurrent_read_write_commit() loop", so ignoring arbitrary errors
should be okay.
The problem can be reproduced on my Linux machine by inserting 10ms sleep() to
gix and increasing the concurrency.
Fixes#3069
This is for completeness and to avoid accidents such as someone calling
`ContentHash::hash(1234u32.to_le_bytes())` and expecting it to hash properly as
a u32 instead of a 4 byte slice, which produces a different hash due to hashing
the length of the slice before its contents.
The `ContentHash` documentation specifies that implementations for enums should
hash the ordinal number of the variant contained in the enum as a 32-bit
little-endian number and then hash the contents of the variant, if any.
The current implementations for `std::Option`, `MergedTreeId`, and
`RemoteRefState` are non-conformant since they hash the ordinal number as a u8
with platform specific endianness.
Fixes#3051
Similar to the previous commit, these functions will be reused by the change id
lookup methods. The return value isn't cloned because resolve_id_prefix() will
return (key, value) pair, and the current caller doesn't need a cloned value.
This is recommended by the insta documentation.
See: https://docs.rs/insta/latest/insta/#optional-faster-runs.
Running on an M1 MacBook Pro
Before:
```
________________________________________________________
Executed in 31.10 secs fish external
usr time 118.87 secs 107.00 micros 118.87 secs
sys time 46.19 secs 847.00 micros 46.19 secs
```
After:
```
________________________________________________________
Executed in 29.73 secs fish external
usr time 119.58 secs 0.13 millis 119.58 secs
sys time 46.27 secs 1.74 millis 46.27 secs
```
This doesn't seem like a huge change, and there's noise in the measurements,
but I don't think it can hurt and insta recommends it.
This bundles all Jujutsu related media (the talk, its slides and the three articles we were featured in)
into a section.
Also update the news section to mention the recent deprecations and the three articles we recently
were featured in.
Thanks to everyone who wrote one.
This removes redundant case from resolve_neighbor_commit_ids(). The returned
position should never be lower than the prefix id.
The implementation is basically a copy of slice::binary_search_by(). We still
use (low + high) / 2 as the size wouldn't exceed 2^31.
https://github.com/rust-lang/rust/blob/1.76.0/library/core/src/slice/mod.rs#L2825
Since IdIndex sorts the entries by using .sort_unstable_by_key(), the order of
the same-key elements is undefined. Perhaps, it's stable for short arrays, and
the test passes because of that.
This saves 4 more bytes per entry, and more importantly, most commit parents
can be resolved with no indirection to the overflow table.
IIRC, Git always inlines the first parent, but that wouldn't be useful in jj
since jj diffs merge commit against the auto-merge parent. The first merge
parent is nothing special.
I'll use a similar encoding in change id sstable, where only one position
will be inlined (to optimize for imported commits.)
Benchmark number measuring the cost of change id index building:
```
% hyperfine --sort command --warmup 3 --runs 20 -L bin jj-0,jj-1 \
-s "target/release-with-debug/{bin} -R ~/mirrors/linux \
--ignore-working-copy debug reindex" \
"target/release-with-debug/{bin} -R ~/mirrors/linux \
--ignore-working-copy log -r@ --config-toml='revsets.short-prefixes=\"\"'"
Benchmark 1: target/release-with-debug/jj-0 -R ~/mirrors/linux --ignore-working-copy log -r@ --config-toml='revsets.short-prefixes=""'
Time (mean ± σ): 342.9 ms ± 14.5 ms [User: 202.4 ms, System: 140.6 ms]
Range (min … max): 326.6 ms … 360.6 ms 20 runs
Benchmark 2: target/release-with-debug/jj-1 -R ~/mirrors/linux --ignore-working-copy log -r@ --config-toml='revsets.short-prefixes=""'
Time (mean ± σ): 325.0 ms ± 13.6 ms [User: 196.2 ms, System: 128.8 ms]
Range (min … max): 311.6 ms … 343.2 ms 20 runs
Relative speed comparison
1.06 ± 0.06 target/release-with-debug/jj-0 -R ~/mirrors/linux --ignore-working-copy log -r@ --config-toml='revsets.short-prefixes=""'
1.00 target/release-with-debug/jj-1 -R ~/mirrors/linux --ignore-working-copy log -r@ --config-toml='revsets.short-prefixes=""'
```
I'm going to change the index format to store local positions in the lookup
table. That's not super important, but I think it makes sense because the
lookup table should never contain inter-segment links.
The mutable segment now stores local positions in its lookup map. The readonly
segment will be updated later.
This unblocks removal of 'is_legacy: bool' fields.
Note that all legacy dag range expressions can't be accepted by the new grammar.
For example, 'x:y()' is parsed as ('x:y', error) because 'x:y' is a valid string
pattern expression, and '(' isn't an infix operator. The old compat_dag_range_op
is NOT removed as it can still translate 'x():y' or 'x:(y)' to a better error,
and we might make the string pattern syntax stricter #2101.
The legacy parsing rules are turned into compatibility errors. The x:y rule
is temporarily enabled when parsing string patterns. It's weird, but we can't
isolate the parsing function because a string pattern may be defined in an
alias.
This adds a config to render a synthetic node with a "(elided
revisions)" description for elided segments of the graph.
I didn't add any templating support for the elided nodes because I'm
not sure how we would want that to work. In particular, I don't know
what `commit_id` and most other keywords should return for elided
revisions.
Users who edit non-head commits usually expect `jj next/prev` to
continue to edit the next/previous commit, so let's make that the
default behavior. This should not confuse users who don't edit
non-head commits since they will simply not be in this state. My main
concern is that doing `jj next; jj prev` will now usually take you
back to the previous commit, but not if you started on the parent of a
head commit.
The main goal is to avoid having a symlink in our source tree. Currently, there
is no good way to work with the `jj` repo with `jj` on Windows. Currently `jj`
just crashes with symlinks. This is being worked on, see e.g. #2939, but it will
always depend on whether Developer Mode is enabled in Windows or whether
symlinks are materialized as text files with symlinks. Finally, MkDocs has
trouble following symlinks on Windows, so building docs wouldn't work there.
Another advantage is that, previously, we were lucky that MkDocs treats `insta`
header in `cli-reference@.md.snap` as a Markdown header and follows symlinks at
all. Now, we no longer depend on that.
It's not ideal to print the error there, but using stderr should be slightly
better. It could be a tracing message, but tracing won't be displayed by
default.
I'm going to introduce breaking changes in index format. Some of them will
affect the file size, so version number or signature won't be needed. However,
I think it's safer to detect the format change as early as possible.
I have no idea if embedded version number is the best way. Because segment
files are looked up through the operation links, the version number could be
stored there and/or the "segments" directory could be versioned. If we want to
support multiple format versions and clients, it might be better to split the
tables into data chunks (e.g. graph entries, commit id table, change id table),
and add per-chunk version/type tag. I choose the per-file version just because
it's simple and would be non-controversial.
As I'm going to introduce format change pretty soon, this patch doesn't
implement data migration. The existing index files will be deleted and new
files will be created from scratch.
Planned index format changes include:
1. remove unused "flags" field
2. inline commit parents up to two
3. add sorted change ids table
Since the operation log has a root operation, we don't need to create
the repo-initialization operation in order to create a valid
`ReadonlyRepo` instance. I think it's conceptually simpler to create
the instance at the root operation id and then add the initial
operation using the usual `Transaction` API. That's what this patch
does.
Doing that also brought two issues to light:
1. The empty view object doesn't have the root commit as head.
2. The initialized `OpHeadsStore` doesn't have the root operation as
head.
Both of those seem somewhat reasonable, but maybe we should change
them. For now, I just made the initial repo (before the initial
operation) have a single op head (to compensate for (2)). It might be
worth addressing both issues so the repo is in a better state before
we create the initial operation. Until we do, we probably shouldn't
drop the initial operation.
