This converts `TreeDiffIterator::tree()` and
`TreeDiffIterator::single_tree()` into associated functions and passes
in the `&MergedTree` into the former. This prepares for fixing #2165,
and it removes the need for the `TreeDiffIterator::store` field.
This is what I proposed in #2095. @ is now an operator to concatenate symbols.
Unlike the other operators, lhs/rhs of @ is not a target of alias substitution.
'x' in 'x@y' doesn't look like a named variable, though it's technically
possible to allow definition of an alias expanded to a symbol of specific remote
or vice versa. This will probably apply to the kind:pattern syntax, where
aliases are expanded due to the current implementation restriction. I've added
a TODO comment about that.
I'm going to change the parsing rule of name@remote, and @ will no longer be
included in a symbol identifier. I could add a separate test for remote symbols,
but I think it's better to write tests that cover both "x"@"y" and "x@y" paths.
`itertools::interleave()` does exactly what we want for
`Merge::iter()`. I had just not thought to look for it
before. Hopefully it's not noticeably slow.
An alternative name for it would be `arity()`, but `num_sides()`
probably more clearly says that it's not about the number of removes
or the total number of terms.
We were using `current_tree()` only for an assertion where we were
walking its entries. Now that `MergedTree` supports that, we can
replace `current_tree()` by `current_merged_tree()`.
There's more work needed before the working copy can fully work with
tree-level conflicts. We still need to be able to store multiple tree
ids in the `tree_state` file, and we need to be able to create
multiple trees instead of writing conflict objects to the backend.
To support tree-level conflicts, we're going to need to update the
working copy from one `MergedTree` to another. We're going need to
store multiple tree ids in the `tree_state` file. This patch gets us
closer to that by getting the diff from `MergedTree`s`, even though we
assume that they are legacy trees for now, so we can write to the
single-tree `tree_state` file.
If we're going to be able to replace most instances of `Tree` by
`MergedTree`, we'll need to be able to diff two `MergedTree`s. This
implements support for that. The implementation copies a lot from the
diff iterator we have for `Tree`. I suspect we should be able to reuse
some of the code by introducing some traits that can then be
implemented by both `Tree` and `MergedTree`. I've left a TODO about
that.
When we do an update between two `MergedTree` instances, we'll get
diffs between two `Merge<Option<TreeValue>>`. This commit prepares for
that by changing the type of the `before` and `after` arguments we
pass into the closure in `update()`.
I think it's a little easier to follow if we don't update the stats in
the large callback. It also reduces the risk of forgetting to update
the stats in some case (like in the exec-bit-optimization case I just
removed).
When updating the working copy from one tree to another, if only the
executable bit has changed between the two trees, we set the
executable bit on the file without touching its contents. The
optimization probably gets used quite rarely. Maybe it's even so
rarely that it's a pessimization overall. Perhaps its value lies more
in that we avoid updating the file's mtime unnecessarily. Either way,
I'm about to change this code to use `Merge<Option<TreeValue>>` and
that will make this block more complex. I don't think it's worth the
complexity even it provides some small benefit sometimes.
Many of the `TreeBuilder` users have an `Option<TreeValue>` and call
either `set()` or `remove()` or the builder depending on whether the
value is present. Let's centralize this logic in a new
`TreeBuilder::set_or_remove()`.
One of the error types that I later created embedded `BackendError`, but `clippy` complained that the size of the type was too large. This helps address that.
The VS Code "Better TOML" plugin (which I think most of our VS Code developers use?) doesn't support the `x.y = z` syntax at the top level, even though it's valid TOML.
This is also useful if we ever want to add additional properties in different sub-crates (although unlikely for the near future).
When the main `TreeState::snapshot()` thread doesn't receive any
updated tree entries over the channel, it correctly doesn't write a
new tree. However, it also doesn't write the working copy state file
(`.jj/working_copy/tree_state`). This resulted in performance
regression in 3f97a6da78. From that commit, repeated snapshotting
would have to re-read all files from disk because it didn't remember
the updated mtime from the previous time.
This patch fixes the bug by also writing the file if there were any
new file states.
This doesn't seem to make any difference right now, but it will if we
write the state file when there are mtime-only changes, which we
currently don't do.
`revset::parse()` already has a `RevsetWorkspaceContext` argument, so
I think it makes sense to put that and the other context arguments
into a larger `RevsetParseContext` object.
We resolve file paths into repo-relative paths while parsing the
revset expression, so I think it's consistent to also resolve which
workspace "@" refers to while parsing it. That means we won't need the
workspace context both while parsing and while resolving symbols.
In order to break things like `author("martinvonz@")` (thanks to @yuja
for catching this), I also changed the parsing of working-copy
expressions so they are not allowed to be
quoted. `author(martinvonz@)` will therefore be an error now. That
seems like a small improvement anyway, since we have recently talked
about making `root` and `[workspace]@` not parsed as other symbols.
Per discussion in #2107, I believe "exact" is preferred.
We can also change the default to exact match, but it doesn't always make
sense. Exact match would be useful for branches(), but not for description().
We could define default per predicate function, but I'm pretty sure I cannot
remember which one is which.
git-branchless calls it a substring, so let's do the same.
FWIW, I copied literal:_ from Mercurial, but it's exact:_ in git-branchless.
I have no idea which one is preferred. Since this feature isn't released, we
can freely change it if exact:_ makes more sense.
https://github.com/arxanas/git-branchless/wiki/Reference:-Revsets#patterns
This commit replaces the functions `UserSettings::user_name_placeholder()`` and
`UserSettings::user_email_placeholder()` with `const` `&str`s to emphasize that
the placeholder strings must not be changed to support commits without
names or email addresses made before this change.
The code for getting the current tree object was repeated a few times
over. I'm going to soon make it return a `MergedTree` and I don't want
to repeat that code (it's more complicated than the current code).
The syntax is slightly different from Mercurial. In Mercurial, a pattern must
be quoted like "<kind>:<needle>". In JJ, <kind> is a separate parsing node, and
it must not appear in a quoted string. This allows us to report unknown prefix
as an error.
There's another subtle behavior difference. In Mercurial, branch(unknown) is
an error, whereas our branches(literal:unknown) is resolved to an empty set.
I think erroring out doesn't make sense for JJ since branches() by default
performs substring matching, so its behavior is more like a filter.
The parser abuses DAG range syntax for now. It can be rewritten once we remove
the deprecated x:y range syntax.
Add type annotation to `vec` to avoid the following build error if you
additionally import `bstr`:
```
~/jj> cargo test
Compiling jj-lib v0.8.0 (/home/aspotashev/jj/lib)
warning: unused import: `bstr`
--> lib/src/default_index_store.rs:30:5
|
30 | use bstr;
| ^^^^
|
= note: `#[warn(unused_imports)]` on by default
error[E0282]: type annotations needed
--> lib/src/default_index_store.rs:564:14
|
564 | .as_mut()
| ^^^^^^
565 | .write_u32::<LittleEndian>(parent_overflow.len() as u32)
| --------- type must be known at this point
|
help: try using a fully qualified path to specify the expected types
|
563 | <[u8] as AsMut<T>>::as_mut(&mut buf[parent_overflow_offset..parent_overflow_offset + 4])
| +++++++++++++++++++++++++++++++ ~
For more information about this error, try `rustc --explain E0282`.
warning: `jj-lib` (lib) generated 1 warning
error: could not compile `jj-lib` (lib) due to previous error; 1 warning emitted
```
Reason to support `bstr` being imported: in a Bazel environment where
crates are imported with certain features enabled, jj-lib may pull in
bstr as part of the following dependency chain:
jj-lib -> insta -> similar -> bstr.
We now have all the pieces in place to read the current tree as a
`MergedTree` when snapshotting the working copy. For now, it's still
always a legacy tree. We'll need to update the working copy state file
to support storing multiple trees before we can create a `MergedTree`
with multiple sides here.
For tree-level conflicts, we're going to be getting
`Merge<Option<TreeValue>>` from the current tree and produce a new
such value if contents changes on disk. This commit gets us a little
closer to that by passing in a value of that type into
`write_path_to_store()`.
This seems to have a small but measurable performance
impact. Snapshotting the working copy in the git repo with all files
`touch`ed went from 2.36 s to 2.43 s (3%). I think that's okay,
especially since most files' mtimes rarely change, and we only pay the
price when it has.
If the value at a path hasn't changed, there's no need to send it over
the channel and have the receiver add it to `TreeBuilder`. I couldn't
measure any performance impact.
Now we should no longer send `TreeValue::Conflict` variants over the
tree entry channel.
When writing tree-level conflicts, we're going to be writing multiple
tree (maybe using some new `MergedTreeBuilder`), so we'll need the
full `Merge<Option<TreeValue>>` object. This gets us closer to that by
sending such objects over the channel and having the receiver write
the conflict object.
Note that we still sometimes send `TreeValue::Conflict` variants over
the channel. That only happens if they're unchanged.
When writing tree-level conflicts, we won't pass `TreeValue::Conflict`
over the `tree_entries` channel. Instead, we're going to pass possibly
unresolved `Merge<Option<TreeValue>>` instances. This commit prepares
for that by changing the type even though we'll only pass
`Merge::normal()` over the channel at this point.
I did this partly to see what the performance impact is. I tested that
by touching all files in the git.git repo to force the trees (and
files) to be rewritten. There was no measurable impact at all
(best-of-10 time was 2.44 s before and 2.40 s after, but I assume that
was a fluke).
This basically means that heads in a filtered graph appear in reverse
chronological order. Before, "jj log -r 'tags()'" in linux-stable repo would
look randomly sorted once you ran "jj debug reindex" in it.
With this change, indexing is more like breadth-first search, and BFS is
known to be bad at rendering nice graph (because branches run in parallel.)
However, we have a post process to group topological branches, so we don't
have this problem. For serialization formats like Mercurial's revlog iirc,
BFS leads to bad compression ratio, but our index isn't that kind of data.
Reindexing gets slightly slower, but I think this is negligible.
(in Git repository)
% hyperfine --warmup 3 --runs 10 "jj debug reindex --ignore-working-copy"
(original)
Time (mean ± σ): 1.521 s ± 0.027 s [User: 1.307 s, System: 0.211 s]
Range (min … max): 1.486 s … 1.573 s 10 runs
(new)
Time (mean ± σ): 1.568 s ± 0.027 s [User: 1.368 s, System: 0.197 s]
Range (min … max): 1.531 s … 1.625 s 10 runs
Another idea is to sort heads chronologically and run DFS-based topological
sorting. It's ad-hoc, but worked surprisingly well for my local repositories.
For repositories with lots of long-running branches, this commit will provide
more predictable result than DFS-based one.
With the new `Merge::iter()`, we can simplify the code a bit by
combining that with `zip`.
I'll simplify the last part of `update_from_content()` next.
Implementing `Iterator` and `FromIterator` on `Merge<T>` provides much
more flexibility than the current `map()`, `try_map()`, etc.
`Merge::from_iter()` wouldn't have a way of failing if it's given an
unexpected (even) number of items. I would be fine with having it
panic, but we can't even usefully do that, because
e.g. `Option::from_iter()` will pass us an iterator ends early if the
input interator ends early. For example,
`Merge::resolved(None).iter().collect()` would call
`Merge::from_iter()` with an empty iterator (first item `None`). So, I
instead created a `MergeBuilder` type implementing `FromIterator`, and
let `MergeBuilder::build()` panic if there were an even number of
items.
I re-implemented some existing `Merge` methods using the new
facilities in this commit. Maybe we should remove some of the methods.
This allows us to reorder commits to be indexed in bulk.
The incremental update optimization is applied only for a single head. This
could be tried for multiple heads, but it's unlikely that every head has
a single new commit for each.
This is similar to what mut_repo.add_head() does.
I'm going to adjust the visiting order so the bulk-imported history preserves
chronological order. It might be a small adjustment on the current DFS
approach, or new function based on Kahn's algorithm. Either way, it's important
that both "jj git import" and "jj debug reindex" use the same underlying
function.
Almost the entire method deals with `FileType::Normal`, so we can
reduce indentation and repeated matching on the file type by doing it
early and returning in the non-normal-file cases.
For tree-level conflicts, we're eventually not going to have
`ConflictId`. We'd want to make `write_conflict_to_store()` take a
`Merge<Option<TreeValue>>` and return an updated such value. That
would leave very little logic in the function, so let's just inline it
instead.
`update_from_content()` already writes file content for each term of
an unresolved merge, so it seems consistent for it to also write the
file content for resolved merges. I think this should simplify further
refactoring for tree-level conflicts and for preserving the executable
bit.
Since `update_from_contents()` only works with file contents and not
the executable or other kinds of paths, I think it makes more sense
for it to deal with `FileId`s instead of `TreeValue`s.
There were still many instances of `conflict` left from before we
renamed `Conflict<T>` to `Merge<T>`. I decided to rename many of them
based on the type parameter instead of the container. I think that
made it more readable in many cases.
I think I moved way too many functions onto `Merge<Option<TreeValue>>`
in 82883e648d. This effectively reverts almost all of that
commit. The `Merge<T>` type is simple container and it seems like it
should be at fairly low level in the dependency graph. By moving
functions off of it, we can get rid of the back-depdencies from the
`merge` module to the `conflict` module that I introduced when I moved
`Merge` to the `merge` module. I'm thinking the `conflict` module can
focus on materialized conflicts.
Per discussion in #2009. This behavior isn't affected by e7e49527ef "git:
ensure that remote branches never diverge", but it's subtle enough to write
a test.
