This doesn't provide any benefit yet bit I think we've known for a
while that we want to make the backend write methods async. It's just
not been important to Google because we have the local daemon process
that makes our writes pretty fast. Regardless, this first commit just
changes the API and all callers immediately block for now, so it won't
help even on slow backends.
This helps resolve diverged refs by abandoning both sides:
D ref = [D]
|\
| C C ref = [B - D + C]
| | |
B | B | B ref = [B - D + A]
|/ |/ |
A A A A ref = [A - D + A]
This is closer to the original behavior before 5e8d7f8c "rewrite: update
references after rewriting all commits." References can move to divergent
commits, so they should propagate further if there are more rewrites. See
the inline comment for subtle behavior difference.
We could instead replay parent_mapping in topological order, but we would
still need to flatten abandon records.
"Concurrent" operations are not necessarily actually concurrent, so
"divergent" seems like a better name. And "reconcile" seems like a
better term for merging them, though we also sometimes use "merge".
Like https://github.com/martinvonz/jj/pull/4189, this allows extensions the ability to load the repo in an environment where the local filesystem is not accessible. This change allows such extensions to exist at the CLI layer where jj is invoked as a subprocess, rather than a library (common in testing).
I just choose "clru" because it already exists in our dependency tree thorough
gix. I don't think LRU is the best cache eviction policy for our use case (a
simpler FIFO-based one might be good enough?), but it wouldn't matter for CLI
or GUI use case. I don't see significant performance degradation with "jj log
--stat -n1000".
RwLock is replaced with Mutex since get() is inherently a mutable operation.
If merge-heavy history was abandoned, intermediate parent chains can have tons
of duplicates, and the process explodes soon. Instead, we can skip any parent
ids that have been remapped.
We can no longer detect cycles reliably, but I think that's okay so long as
the function terminates.
Fixes#4352
FileConflict will be changed to not materialize Merge<BString>. I also updated
the revset engine to ignore non-file conflict. It doesn't make sense to grep
conflict description.
The native backend currently errors out if you ask it about copies. So
does the test backend. I think it's better to return an empty stream
of copies so it doesn't prevent other functionality.
Not all callers need this information, but I assumed it's relatively cheap to
look up the source path in the target tree compared to diffing.
This could be represented as Regular(_)|Copied(_, _)|Renamed(_, _), but it's
a bit weird if Copied and Renamed were separate variants. Instead, I decided
to wrap copy metadata in Option.
This patch adds accessor methods as I'm going to change the underlying data
types. Since entry values are consumed separately, these methods are implemented
on CopiesTreeDiffEntryPath, not on *TreeDiffEntry.
I plan to provide a richer version of `TreeDiffEntry` with copy info
(and to make `TreeDiffEntry` itself "poorer"). Most callers want to
know about copies/renames, but at least working copy implementations
probably don't. This patch adds separate `diff_stream()` and
`diff_stream_with_copies()` so we can provide the simpler interface
for callers that don't need copy info.
The support for copy tracing is already simply added to the stream
just before yielding the item, so we can easily implement it as a
stream adapter. That ensures that we use the same logic for the
iterator- and stream-based versions. More importantly, it enables
further cleanups and a simpler interface.
So that more tests can leverage diff::diff() helper.
I also removed the fast path for identical inputs. This function is only used by
tests and benches, and production code usually compares content hashes first.
The tree-level conflicts have worked well in practice and we don't
want to allow users to use legacy trees for new commits. We don't
really support legacy trees very well since 0590f8bece anyway.
I'm going to split color-words diffs to by_line() and by_word() stages.
Perhaps, Diff::default_refinement() can be removed once all non-test callers
are migrated.
I'm thinking of adding some heuristics to render hunks containing lots of
small word changes differently, in a similar manner to the unified diffs. This
patch might help add some pre/post-processing at consumer.
files::diff() is inlined to caller to get around 'self borrowing.
When rebasing a new child commit on top of the moved commit(s), the
order of the new child commit's parent commits is now correctly
preserved if the original parent commit is now a parent of the moved
commit(s).
Closes#3969.
- add support for copy tracking to `diff --stat`
- switch `--summary` to match git's output more closely
- rework `show_diff_summary` signature to be more consistent
Add home directory expansion for SSH key filepaths. This allows the
`signing.key` configuration value to work more universally across both
Linux and macOS without requiring an absolute path.
This moved and renamed the previous `expand_git_path` function to a more
generic location, and the prior use was updated accordingly.
Perhaps, we should also cache merged trees, but this patch saves time until
we implement the bookkeeping. Even if we had a cache, it wouldn't be ideal to
calculate uncached merged trees during revset evaluation.
```
% hyperfine --sort command --warmup 3 --runs 10 -L bin jj-1,jj-2 \
'target/release-with-debug/{bin} -R ~/mirrors/git --ignore-working-copy \
log -r "::@ & file(root:builtin)" --no-graph -n50'
Benchmark 1: target/release-with-debug/jj-1 ..
Time (mean ± σ): 3.512 s ± 0.014 s [User: 3.391 s, System: 0.119 s]
Range (min … max): 3.489 s … 3.528 s 10 runs
Benchmark 2: target/release-with-debug/jj-2 ..
Time (mean ± σ): 1.351 s ± 0.010 s [User: 1.275 s, System: 0.074 s]
Range (min … max): 1.332 s … 1.366 s 10 runs
Relative speed comparison
2.60 ± 0.02 target/release-with-debug/jj-1 ..
1.00 target/release-with-debug/jj-2 ..
```
I'll add conflict resolution there.
This change adds more synchronization points, which is probably bad for
concurrency. However, this module is a revset engine for the default index,
so the store backends are supposed to be fast local disks.
I'll add a public function that resolves file conflicts. This function will
take owned MergedTreeValue, and that's why the extracted function returns
None instead of cloning the passed value.
MergedTreeVal was roughly equivalent to Merge<Option<Cow<_>>. As we've dropped
support for the legacy trees, it can be simplified to Merge<Option<&_>>.
try_resolve_file_conflict() is also updated. It could be a generic function,
but there are only two callers, and the legacy tree one is used only in tests.
For the same reason as 2cb7e91d "merged_tree: do not re-look up non-conflicting
tree values by name." This appears to bring a similar performance improvement.
I assume this change is/will be covered by test_merged_tree.rs. I considered
adding a few unit tests, but constructing Tree object isn't trivial, and the
iterator implementation is relatively straightforward.
- force each diff command to explicitly enable copy tracking
- enable copy tracking in diff_summary
- post-process for diff iterator
- post-process for diff stream
- update changelog
- use a single commit instead of an array of them. This simplifies the
implementation. A higher level api can wrap this when an array of
commits is desired and those semantics are figured out.
- since this API is directly 1-1 on parents, there are no conflicts
- if we introduce a higher level API that handles lists of commits, we
may need to restore the conflict/resolved distinction, but for now
simplify
This allows us to diff trees without fully resolving conflicts:
let from_tree = merge_no_resolve(..);
for (path, (from, to)) in from_tree.diff(to_tree, matcher) {
let from = resolve_conflicts(from);
if from == to {
continue; // resolved file may be identical
...
I originally considered adding a matcher argument to merge() functions, but the
resulting API looked misleading. If merge() took a matcher, callers might expect
unmatched trees and files were omitted, not left unresolved. It's also slower
than diffing unresolved trees because merge(.., matcher) would have to write
partially resolved trees to the store.
Since "ancestor_tree" isn't resolved by itself, this patch has subtle behavior
change. For example, "jj diff -r9eaef582" in the "git" repository is no longer
empty. I think the new behavior is also technically correct, but I'm not pretty
sure.
While measuring file(path) query, I noticed BTreeMap lookup appears in perf.
It actually has a measurable cost if the history is linear and parent trees
don't have to be merged dynamically. For merge-heavy history, the cost of
tree merges is more significant. I'll address that separately.
```
% hyperfine --sort command --warmup 3 --runs 50 -L bin jj-1,jj-2 \
'target/release-with-debug/{bin} -R ~/mirrors/git --ignore-working-copy \
log -r "::trunk() & ~merges() & file(root:builtin)" --no-graph -n100'
Benchmark 1: target/release-with-debug/jj-1 ..
Time (mean ± σ): 239.7 ms ± 7.1 ms [User: 192.1 ms, System: 46.5 ms]
Range (min … max): 222.2 ms … 249.7 ms 50 runs
Benchmark 2: target/release-with-debug/jj-2 ..
Time (mean ± σ): 201.7 ms ± 6.9 ms [User: 153.7 ms, System: 46.6 ms]
Range (min … max): 184.2 ms … 211.1 ms 50 runs
Relative speed comparison
1.19 ± 0.05 target/release-with-debug/jj-1 ..
1.00 target/release-with-debug/jj-2 ..
```
Suppose we add copy information to MergedTree, a MergedTree can be considered
a root tree representation plus global metadata. I think Merge<Tree> is a better
type for sub trees.
I considered making `MergedTree` just a newtype (1-tuple) but I went
with a struct instead because we may want to add copy information in a
separate field in the future.
In order to remove the `MergedTree::Legacy` form, we need to stop
creating such instances. This patch removes the last place we create
them, which is in `Store::get_root_tree()`.
The main practical consequence of this change is that loading legacy
trees gets a lot slower on large repos. However, since the default log
template includes the `conflict` keyword, we ended up scanning all
paths in `jj log` anyway, so I'm not sure many people will notice.
Since "op abandon" just rewrites DAG, it works no matter if the heads are
merged or not. This change will help crash recovery. "op abandon
--at-op=<one-of-the-heads>" can't be used because ancestor operations would be
preserved by the other head.
Suppose a squash node in obslog is analogous to a merge in revisions log, it
makes sense to show diffs from auto-merge (or auto-squash) parents. This
basically means a non-partial squash node no longer shows diffs.
This also fixes missing diffs at the root predecessors if there were.
Author dates and committer dates can be filtered like so:
committer_date(before:"1 hour ago") # more than 1 hour ago
committer_date(after:"1 hour ago") # 1 hour ago or less
A date range can be created by combining revsets. For example, to see any
revisions committed yesterday:
committer_date(after:"yesterday") & committer_date(before:"today")
Creates a DatePattern type that can be created by parsing a string in any
format supported by the chrono-english crate, including:
- 2024-03-25
- 2024-03-25T00:00:00
- 2024-03-25T00:00:00-08:00
- 2 weeks ago
- 5 minutes ago
- yesterday
- yesterday 5pm
- yesterday 10:30
- yesterday 15:30
- tomorrow
A `kind` can be specified to indicate whether the pattern should match dates at
or after (`after`) or strictly before (`before`) the given instant.
chrono-english supports US and UK dialects to disambiguate mm/dd/yy from
dd/mm/yy, but for now we default to US. This should probably be a config
setting.
This enables the creation of Repo objects in environments without standard filesystem support, by allowing the caller to load the store objects however they see fit. This confines interaction with the filesystem to the WorkingCopy abstractions.
This is part of migrating away from legacy trees (with path-level
conflicts). I can't think of any practical impact (we already compare
the tree ids equal).
This basically reverts 20eb9ecec1 "git: don't abandon HEAD commit when it
loses a branch." I think the new behavior is more consistent because the Git
HEAD is equivalent to @- in jj, so it shouldn't be considered a named ref.
Note that we've made old HEAD branch not considered at 92cfffd843 "git: on
external HEAD move, do not abandon old branch."
#4108
If readonly_index() and index() returned Result, it would propagate to many
call sites. That seems bad for API ergonomics. Suppose most "repo" commands
depend on an index, I think it's okay to load index eagerly:
- "jj config" doesn't load repo (nor index)
- "jj workspace root" doesn't load repo (nor index)
- some other mutation commands load index when printing commit summary
- many other commands load index when resolving revset
In order to render description template, we'll need a Commit object that
represents the old state (with new tree and parents) before updating the
commit description. The added functions will help generate an intermediate
Commit object.
Alternatively, we can create an in-memory Commit object with some fake
CommitId. It should be lightweight, but might cause weird issue because the
fake id wouldn't be found in the store.
I think it's okay to write a temporary commit and rely on GC as we do for
merge trees. However, I should note that temporary commits are more likely to
be preserved as they are pinned by no-gc refs until "jj util gc".
This allows us to construct a builder, format description template with an
intermediate commit, then write() a final commit object to the repo.
I originally considered removing mut_repo from CommitBuilder at all, but
rewriter APIs rely on that CommitBuilder has &mut_repo, and splitting them
would make call sites uglier.
The inner builder methods are based on &mut Self instead of Self, because it's
easier to wrap, and users of the inner builder will bind it to a named variable
anyway.
As the doc comment says, it's called only from CommitBuilder. Let's clarify
that. I'm also planning to extract a builder that only writes to the store
(without mutably borrowing a mut_repo.) It will help implement description
template.
It's inconvenient that we have to quote glob patterns as 'glob:"*.rs"'. Suppose
filesets are usually specified in shell, it's better to allow unquoted strings
if possible. This change also means we'll probably abandon #2101 "make the
parsing of string arguments stricter."
Note that we can no longer introduce ? operator or [] subscript syntax in
filesets.
Closes#4053
The text pattern is applied prior to comparison as we do in Mercurial. This
might affect hunk selection, but is much faster than computing diff of full
file contents. For example, the following hunk wouldn't be caught by
diff_contains("a") because the line "b\n" is filtered out:
- a
b
+ a
Closes#2933
