Commit graph

800 commits

Author SHA1 Message Date
James Sully
0946934ca6 revset: Add optional argument n to ancestors() in revset language 2023-09-08 02:50:58 +10:00
Yuya Nishihara
a868b2d9a5 revset: do not lookup unimported tags or remote branches by unqualified name
Since e7e49527ef "git: ensure that remote branches never diverge", the last
known "refs/remotes" ref should be synced with the corresponding remote branch.
So we can always trust the branch@remote expression. We don't need "refs/tags"
lookup either since tags should have been imported by git::import_refs().

FWIW, I'm thinking of reorganizing view.git_refs() map as per-remote views.
It would be nice if we can get rid of revsets and template keywords exposing
low-level Git ref primitives.
2023-09-06 13:42:22 +09:00
Yuya Nishihara
4e6323c0a4 revset: add basic test for tag name resolution 2023-09-06 13:42:22 +09:00
Martin von Zweigbergk
f198a1e5f9 git: skip branches on root commit on export
Git doesn't have a root commit, so we should skip branches pointing to
it on export, just like we do with conflicted branches (which Git also
doesn't support).
2023-09-04 20:08:11 -07:00
Martin von Zweigbergk
ef550a9d6d git: include reason for each failed ref export 2023-09-04 20:08:11 -07:00
Martin von Zweigbergk
f6c24fbcef git: return refs that failed to export in sorted order
Before this patch, the order would depend on the reason we failed to
export a ref, because we would add to the `failed_branches` list in
several different places. What's worse, when the export failed because
the branch was conflicted or had an invalid name (from Git's
perspective), it was non-deterministic because we iterated over a
HashSet. This patch fixes that by sorting at the end.

Note that we still want the `branches_to_update` map to be a
`BTreeMap` so we update branches in deterministic order. Otherwise the
error when trying to export both branches `main` and `main/sub` will
become non-deterministic.
2023-09-04 20:08:11 -07:00
Yuya Nishihara
b0c8e9ef62 revset: add 0-ary "::" and ".." operators as short for "all()" and "~root()"
Suppose "x::y" is the operator that defaults to "root()::visible_heads()"
respectively, "::" is identical to "all()". Since we've just changed the
behavior of "..y", ".." is now "root()..visible_heads()" meaning "~root()".
2023-09-05 10:40:04 +09:00
Yuya Nishihara
6b2ad23f8f revset: evaluate "..y" expression to "root()..y"
This seems useful since the root commit is often uninteresting. It's also
consistent with "x::y" in a way that the left operand defaults to "root()".
2023-09-05 10:40:04 +09:00
Yuya Nishihara
e3c85d6ecc revset: convert root symbol to function
The idea is that we can fully eliminate special symbols that would otherwise
shadow user branches, tags, or change ID prefixes.

Closes #2095
2023-09-04 10:36:30 +09:00
Martin von Zweigbergk
b8f71a4b30 working_copy: in LockedWorkingCopy::drop(), discard unsaved changes
In `LockedWorkingCopy::drop()`, we panic if the caller had not called
`finish()`. IIRC, the idea was both to find bugs where we forgot to
call `finish()` and to prevent continuing with a modified
`WorkingCopy` instance. I don't think the former has been a problem in
practice. It has been a problem in practice to call `discard()` to
avoid the panic, though. To address that, we can make the `Drop`
implementation discard the changes (forcing a reload of the state if
the working copy is accessed again).
2023-09-01 12:25:47 -07:00
Martin von Zweigbergk
5ef0be73c1 merged_tree: allow building trees with variable-arity overrides
When restoring (`jj restore`) a 3-sided conflict from one tree into a
2-sided tree (or a resolved tree), we'll need to extend the size arity
of the target tree to that of the source tree. I had not considered
this case before. This patch relaxes the constraint in
`MergedTreeBuilder` to allow such cases. The additional trees are
based on empty trees with only the larger overrides in them.
2023-09-01 06:09:37 -07:00
Martin von Zweigbergk
9d9b2cd057 tests: leverage create_tree() in a few more tests 2023-08-30 19:58:42 -07:00
Martin von Zweigbergk
26581750fe store: add a empty_merged_tree_id() helper
Many (most?) callers of `Store::empty_tree_id()` really want a
`MergedTreeId`, so let's create a helper for that. It returns the
`Legacy` variant, which is what all current callers used. That should
be all we need since the two variants compare equal these days, and
since trees built based on the legacy variant can get promoted to the
new variant on write if the config is enabled.
2023-08-30 19:58:42 -07:00
Martin von Zweigbergk
bb755ae754 working_copy: use MergedTreeBuilder in test 2023-08-30 06:17:21 -07:00
Martin von Zweigbergk
6b3fa3ee79 working_copy: avoid a nested TreeBuilder in a test
It's just slightly simpler to not create a `TreeBuilder` to create a
`TreeValue`; we can instead write to the outer builder.
2023-08-30 06:17:21 -07:00
Martin von Zweigbergk
7e6930b56f backend: remove last few instances of MergedTreeId::as_legacy_tree_id() 2023-08-30 06:17:21 -07:00
Martin von Zweigbergk
962da1947e tests: make dump_tree() work with merged trees
My goal is to minimize impact on tests when we start using the new
format.
2023-08-30 06:17:21 -07:00
Martin von Zweigbergk
145b0b24d8 commit: drop merged_ prefix from tree() and tree_id()
The old `tree()` and `tree_id()` functions are now gone, so we can use
those names for the new functions.
2023-08-29 08:32:04 -07:00
Martin von Zweigbergk
2d50d8a077 merged_tree: propagate errors from from_legacy_tree() 2023-08-29 08:32:04 -07:00
Martin von Zweigbergk
67832a3940 merged_tree: take store argument to write_tree() instead of new()
The store isn't needed until we write the trees, so I think it makes
more sense to pass it there.
2023-08-29 08:32:04 -07:00
Martin von Zweigbergk
d9ce70c176 tests: make create_tree() return MergedTree
I think most tests want a `MergedTree`, so this makes `create_tree()`
return that. I kept the old function as `create_single_tree()`. That's
now only used in `test_merge_trees` and `test_merged_tree`.

I also consistently imported the functions now, something I've
considered doing for a long time.
2023-08-29 07:01:52 -07:00
Martin von Zweigbergk
e4c6595620 tests: make create_random_tree() return a MergedTreeId 2023-08-29 07:01:52 -07:00
Martin von Zweigbergk
9c77e6aa8c commit_builder: remove last traces of pre-MergedTree API 2023-08-29 07:01:52 -07:00
Martin von Zweigbergk
6e3590f5cd tests: remove last uses of Commit:tree() and delete it 2023-08-29 07:01:52 -07:00
Yuya Nishihara
ce3d28e234 git: do not import refs from remote named "git"
I made it simply fail on explicit fetch/import, and ignored on implicit import.
Since the error mode is predictable and less likely to occur. I don't think it
makes sense to implement warning propagation just for this.

Closes #1690.
2023-08-29 22:50:46 +09:00
Martin von Zweigbergk
1b24b522f6 tree: move diff_summary() to MergedTree 2023-08-28 16:21:44 -07:00
Martin von Zweigbergk
dc06bbc7d1 commit: migrate remaining uses of Commit::tree_id() and delete it 2023-08-28 15:58:34 -07:00
Martin von Zweigbergk
873a6f0674 merged_tree: add a function for merging 3 MergedTrees
With the already existing `MergedTree::resolve()` and all the recent
refactorings into `Merge<T>`, it's now very easy to add support for
3-way merging of `MergedTree` instances.
2023-08-28 15:58:34 -07:00
Martin von Zweigbergk
1674a421ec commit_builder: take MergedTreeId for root id argument 2023-08-28 15:58:34 -07:00
Martin von Zweigbergk
a7e5ea06c0 tests: make test helper for snapshotting working copy return MergedTree 2023-08-27 06:49:45 -07:00
Martin von Zweigbergk
1895a55157 working_copy: make old_checkout argument be MergedTreeId
I think this was the last piece for making the working copy handle
tree-level conflicts.
2023-08-27 06:49:45 -07:00
Martin von Zweigbergk
abf3853717 working_copy: return MergedTreeId on snapshot 2023-08-27 06:49:45 -07:00
Martin von Zweigbergk
88e9933462 working_copy: enable storing multiple tree ids in state file 2023-08-27 06:49:45 -07:00
Martin von Zweigbergk
49e32aa532 merged_tree: teach tree builder to build multiple trees 2023-08-27 06:49:45 -07:00
Martin von Zweigbergk
2dd2e77170 merged_tree: add entries() for iterating over all entries
We already have `entries_matching()`, so this is just a version of
that that doesn't take a matcher.
2023-08-27 06:49:45 -07:00
Martin von Zweigbergk
36674e8f7e merged_tree: make id() return a MergedTreeId
We will rarely want to use the tree id without knowing whether it can
contain `TreeValue::Conflict` values, so let's make the callers check.
2023-08-27 06:49:45 -07:00
Martin von Zweigbergk
598cfcb89b merged_tree: in diff iterator, maintain legacy/modern variant in subtree
As #2165 showed, when diffing two `MergedTree::Legacy` variants (or
one of each variant) and re recurse into a subtree, we need to treat
that as a legacy tree too, so we expand `TreeValue::Conflict`s found
in the diff.
2023-08-26 05:58:54 -07:00
Martin von Zweigbergk
769c248c49 working_copy: show bug when checking out conflict in subdir 2023-08-26 05:58:54 -07:00
Yuya Nishihara
79291a3ca4 revset: add separate name@remote node to discriminate it from quoted one
A local branch named "name@remote" no longer shadows a remote branch "name"
at "remote".
2023-08-26 07:47:12 +09:00
Yuya Nishihara
81dda498e5 test_revset: rewrite resolve_symbol() to go through normal parse/resolve paths
I'm going to change the parsing rule of name@remote, and @ will no longer be
included in a symbol identifier. I could add a separate test for remote symbols,
but I think it's better to write tests that cover both "x"@"y" and "x@y" paths.
2023-08-26 07:47:12 +09:00
Martin von Zweigbergk
f877610792 merge: add Merge::num_sides()
An alternative name for it would be `arity()`, but `num_sides()`
probably more clearly says that it's not about the number of removes
or the total number of terms.
2023-08-25 08:54:49 -07:00
Martin von Zweigbergk
416fa2741c merged_tree: add entry iterator 2023-08-25 07:06:20 -07:00
Martin von Zweigbergk
85bdba5bea working_copy: use MergedTree for diffing in reset() 2023-08-25 07:06:20 -07:00
Martin von Zweigbergk
23509e939e working_copy: get diff from MergedTrees
To support tree-level conflicts, we're going to need to update the
working copy from one `MergedTree` to another. We're going need to
store multiple tree ids in the `tree_state` file. This patch gets us
closer to that by getting the diff from `MergedTree`s`, even though we
assume that they are legacy trees for now, so we can write to the
single-tree `tree_state` file.
2023-08-25 06:40:36 -07:00
Martin von Zweigbergk
d5ceefcd8e merged_tree: add diff iterator
If we're going to be able to replace most instances of `Tree` by
`MergedTree`, we'll need to be able to diff two `MergedTree`s. This
implements support for that. The implementation copies a lot from the
diff iterator we have for `Tree`. I suspect we should be able to reuse
some of the code by introducing some traits that can then be
implemented by both `Tree` and `MergedTree`. I've left a TODO about
that.
2023-08-25 06:40:36 -07:00
Martin von Zweigbergk
e1c0d4fd5f cleanup: replace x[n..n+l] by x[n..][..l]
This avoids repeating the `n` in these expressions. Thanks to
@Dr-Emann for the suggestion.
2023-08-21 22:29:46 -07:00
Martin von Zweigbergk
c43a3067eb revset: pass all context arguments to parse() via an object
`revset::parse()` already has a `RevsetWorkspaceContext` argument, so
I think it makes sense to put that and the other context arguments
into a larger `RevsetParseContext` object.
2023-08-20 21:30:06 -07:00
Martin von Zweigbergk
5f3df4aaea revset: resolve "@" symbol's workspace id earlier (while parsing)
We resolve file paths into repo-relative paths while parsing the
revset expression, so I think it's consistent to also resolve which
workspace "@" refers to while parsing it. That means we won't need the
workspace context both while parsing and while resolving symbols.

In order to break things like `author("martinvonz@")` (thanks to @yuja
for catching this), I also changed the parsing of working-copy
expressions so they are not allowed to be
quoted. `author(martinvonz@)` will therefore be an error now. That
seems like a small improvement anyway, since we have recently talked
about making `root` and `[workspace]@` not parsed as other symbols.
2023-08-20 17:57:18 -07:00
Yuya Nishihara
b6794ca04a revset: rename literal:"" prefix to exact:""
Per discussion in #2107, I believe "exact" is preferred.

We can also change the default to exact match, but it doesn't always make
sense. Exact match would be useful for branches(), but not for description().
We could define default per predicate function, but I'm pretty sure I cannot
remember which one is which.
2023-08-19 11:33:57 +09:00
Emily Fox
3f8ac2198d commits: use empty strings instead of placeholders for missing name or email
This commit replaces the functions `UserSettings::user_name_placeholder()`` and
`UserSettings::user_email_placeholder()` with `const` `&str`s to emphasize that
the placeholder strings must not be changed to support commits without
names or email addresses made before this change.
2023-08-18 17:22:59 -05:00
Benjamin Saunders
4bd05e8285 tests: hack around broken lint 2023-08-17 19:29:38 -07:00
Benjamin Saunders
417035cb20 tests: validate snapshot.max-new-file-size behavior 2023-08-17 19:29:38 -07:00
Benjamin Saunders
54f1d310c4 testutils: propagate snapshot errors 2023-08-17 19:29:38 -07:00
Yuya Nishihara
81f1ae38b3 revset: add literal:"string" pattern syntax
The syntax is slightly different from Mercurial. In Mercurial, a pattern must
be quoted like "<kind>:<needle>". In JJ, <kind> is a separate parsing node, and
it must not appear in a quoted string. This allows us to report unknown prefix
as an error.

There's another subtle behavior difference. In Mercurial, branch(unknown) is
an error, whereas our branches(literal:unknown) is resolved to an empty set.
I think erroring out doesn't make sense for JJ since branches() by default
performs substring matching, so its behavior is more like a filter.

The parser abuses DAG range syntax for now. It can be rewritten once we remove
the deprecated x:y range syntax.
2023-08-17 07:42:12 +09:00
Emily Fox
9ba9ecd708 revset: add function mine() 2023-08-16 11:00:14 -05:00
Martin von Zweigbergk
1d55a404cc merged_tree: add path_value() 2023-08-15 07:56:55 -07:00
Martin von Zweigbergk
2238c87da1 merged_tree: import create_tree() in tests to reduce line wrapping 2023-08-15 07:56:55 -07:00
Martin von Zweigbergk
0b3b62a777 conflicts: remove redundant num_removes argument from parse_conflict()
Merges always have exactly one more "adds" than "removes" these days.
2023-08-13 09:54:16 +00:00
Martin von Zweigbergk
4c46398b1c conflicts: make update_from_content() write resolved content to store
`update_from_content()` already writes file content for each term of
an unresolved merge, so it seems consistent for it to also write the
file content for resolved merges. I think this should simplify further
refactoring for tree-level conflicts and for preserving the executable
bit.
2023-08-11 23:59:44 +00:00
Martin von Zweigbergk
0b85f06e3d conflicts: make update_from_content() work with only FileIds
Since `update_from_contents()` only works with file contents and not
the executable or other kinds of paths, I think it makes more sense
for it to deal with `FileId`s instead of `TreeValue`s.
2023-08-11 23:59:44 +00:00
Martin von Zweigbergk
94c14d454a tests: levarage the materialize_conflict_string() helper in more places 2023-08-11 23:59:44 +00:00
Martin von Zweigbergk
a995c66635 merge: move some methods back to conflicts as free functions
I think I moved way too many functions onto `Merge<Option<TreeValue>>`
in 82883e648d. This effectively reverts almost all of that
commit. The `Merge<T>` type is simple container and it seems like it
should be at fairly low level in the dependency graph. By moving
functions off of it, we can get rid of the back-depdencies from the
`merge` module to the `conflict` module that I introduced when I moved
`Merge` to the `merge` module. I'm thinking the `conflict` module can
focus on materialized conflicts.
2023-08-11 21:11:25 +00:00
Yuya Nishihara
925d54614d revset: remove round-trip conversion from heads() evaluation
This wouldn't matter much in practice, but I think it's better to stick to
low-level index primitives during revset evaluation.
2023-08-12 02:16:29 +09:00
Martin von Zweigbergk
d1dbe6de98 git: propagate errors for missing commits when importing refs 2023-08-11 05:06:36 +00:00
Yuya Nishihara
530547eb9c tests: test that git::import_refs() can update conflicted remote branch
Per discussion in #2009. This behavior isn't affected by e7e49527ef "git:
ensure that remote branches never diverge", but it's subtle enough to write
a test.
2023-08-10 06:27:16 +09:00
Yuya Nishihara
552c71ed36 tests: move commit_transactions() helper to testutils 2023-08-10 06:27:16 +09:00
Martin von Zweigbergk
ef5f97f8d7 conflicts: move Merge<T> to merge module
The `merge` module now seems like the obvious place for this type.
2023-08-06 22:08:09 +00:00
Martin von Zweigbergk
ecc030848d conflicts: rename Conflict<T> to Merge<T>
Since `Conflict<T>` can also represent a non-conflict state (a single
term), `Merge<T>` seems like better name.

Thanks to @ilyagr for the suggestion in
https://github.com/martinvonz/jj/pull/1774#discussion_r1257547709

Sorry about the churn. It would have been better if I thought of this
name before I introduced `Conflict<T>`.
2023-08-06 22:08:09 +00:00
Martin von Zweigbergk
56109bda38 tests: another attempt to fix remaining flaky tests in test_view.rs
This is another commit like 0f3bd7fb03 and 8e7e32710d. I don't
know how I didn't catch all remaining instances last time :(
2023-08-04 23:36:15 +00:00
Martin von Zweigbergk
4a10ea4e3e tests: attempt to fix more flaky tests in test_view.rs
This is the same kind of fix as in 8e7e3271. I should have just fixed
all instances then.
2023-08-04 21:13:30 +00:00
Martin von Zweigbergk
48b1a1c533 working_copy: in ignored directories, visit only already tracked paths
`.gitignores` in ignored directories should be ignored. Before this
commit, we would visit ignored directories like any others if there
were any ignored paths in them.

I've done a lot of preparation for this commit, but There's still a
bit of duplication between the new code and the existing code. I don't
mind improving it if anyone has suggestions. Otherwise I might end up
doing that when I get back to working on snapshotting tree-level
conflicts soon.

This fixes #1785.
2023-08-01 06:31:52 +00:00
Martin von Zweigbergk
02f2325fae working_copy: add test for .gitignores in ignored directory
This tests the scenario that was repored in #1785.
2023-08-01 06:31:52 +00:00
Martin von Zweigbergk
aff483c431 working_copy: test changes in tracked-but-ignored directory
It's currently the same code path for handling changes to tracked
paths in ignored directories as outside ignored directories, but I'm
about to change that.

I also updated the assertion in the test to compare all entries
instead of just the tree id, so it's easier to spot errors if it
fails.
2023-08-01 06:31:52 +00:00
Yuya Nishihara
b3ae7e7657 revset_graph: preserve original parents order
It seemed awkward if merged PR is sometimes rendered as a first branch.
Instead of sorting edges in index order, let's build a HashSet only when
deduplication is needed.
2023-07-29 05:36:09 +09:00
Martin von Zweigbergk
3ef552c4c1 tests: add TestWorkspace::snapshot() to simplify snapshotting
It's currently a bit complicated to snapshot the working copy and
there's a lot of duplication in tests. This commit introduces a
function to simplify it. I made the function snapshot the working copy
and save the updated state. Some of the tests I changed previously
discarded the changes instead of saving them, but I think they all did
so because it was simpler. I left a few call sites unchanged because
they make concurrent changes.
2023-07-28 09:32:18 -07:00
Martin von Zweigbergk
2ccd3247f9 tests: remove a duplicate code block
It looks like this has been duplicated since I added the test in
cd4fbd3565.
2023-07-28 09:32:18 -07:00
Yuya Nishihara
4834d12c37 simple_op_store: serialize RefTarget in new format (breaks downgrades)
This is breaking change. Old jj binary will panic if it sees a view saved by
new jj. Alternatively, we can store both new and legacy data for backward
compatibility.
2023-07-27 15:32:48 +09:00
Martin von Zweigbergk
4348d0b487 working_copy: leverage create_tree() in test
This also lets us compare the resulting tree because the working copy
now exactly matches the tree (it used to be that the `.gitignore` file
wasn't initially snapshotted).
2023-07-26 23:30:10 -07:00
Martin von Zweigbergk
84a60d15bc op_store: make ViewId and OperationId implement ObjectId 2023-07-26 14:17:21 -07:00
Waleed Khan
70d3c64b1e operation: propagate OpStoreError
This is a rote propagation of the error value to hopefully improve the panic in https://github.com/martinvonz/jj/issues/1907.
2023-07-25 12:46:59 -05:00
Martin von Zweigbergk
8d1cb1e1d7 working_copy: add test of racy checkout followed by file write
We don't seem to have any tests that our protection from undetected
changes caused by writes happening right after checkout, so let's add
one. The test case loops 100 times and each iteration fails slightly
more than 80% of the time on my machine (if I remove the protection in
`TreeState::update_file_state()`), so it seems quite good at
triggering the race.
2023-07-24 16:41:44 -07:00
Martin von Zweigbergk
e364c49854 test_working_copy_concurrent: run only with git backend
I don't see any reason for these tests to differ depending on backend,
so let's avoid running them twice (there are probably lots of other
tests that we're also running with different backends for little
reason).
2023-07-24 16:41:44 -07:00
Yuya Nishihara
e2f9ed439e revset: extract graph-related types to separate module
I'm going to add a topo-grouping iterator adapter, and the revset module is
already big enough to split.
2023-07-25 01:45:37 +09:00
Yuya Nishihara
2bbaa4352a refs: rename RefTarget::is_conflict() to has_conflict()
Copied from MergedTree::has_conflict(). I feel it's slightly better since
RefTarget "is" always a Conflict-based type. It could be inverted to
RefTarget::is_resolved(), but refs are usually resolved, and all callers
have special case for !is_resolved() state.
2023-07-23 22:25:57 +09:00
Martin von Zweigbergk
deb4ae476d merged_tree: add an iterator over conflicts
With `MergedTree`, we can iterate over conflicts by descending into
only the subdirectories that cannot be trivially resolved. We assume
that the trees have previously been resolved as much as possible, so
we don't attempt to resolve conflicts again.
2023-07-19 22:04:16 -07:00
Martin von Zweigbergk
828d528361 merged_tree: add a function for resolving conflicts
This adds a function for resolving conflicts that can be automatically
resolved, i.e. like our current `merge_trees()` function. However, the
new function is written to merge an arbitrary number of trees and, in
case of unresolvable conflicts, to produce a `Conflict<TreeId>` as
result instead of writing path-level conflicts to the backend. Like
`merge_trees()`, it still leaves conflicts unresolved at the file
level if any hunks conflict, and it resolves paths that can be
trivially resolved even if there are other paths that do conflict.
2023-07-19 22:04:16 -07:00
Martin von Zweigbergk
4f30417ffd merged_tree: introduce a type for a set of trees to merge on the fly
In order to store conflicts in the commit, as conflicts between a set
of trees, we want to be able merge those trees on the fly. This
introduces a type for that. It has a `Merge(Conflict(Tree))` variant,
where the individual trees cannot have path-level conflicts. It also
has a `Legacy(Tree)` variant, which does allow path-level conflicts. I
think that should help us with the migration.
2023-07-19 22:04:16 -07:00
Yuya Nishihara
84f0c96c8f view: replace .tags().get(name) with .get_tag(name) 2023-07-19 08:27:42 +09:00
Yuya Nishihara
bf46eb5ad2 view: replace .branches().get(name) with .get_branch(name) 2023-07-19 08:27:42 +09:00
Yuya Nishihara
ecb0850f1a view: return RefTarget by reference, clone() by caller 2023-07-19 08:27:42 +09:00
Yuya Nishihara
4da8483228 refs: reimplement RefTarget as Conflict<Option<CommitId>> wrapper
ContentHash is preserved at this point. I'll update it together with the
serialization format changes.
2023-07-18 18:12:09 +09:00
Yuya Nishihara
b6967a1dc9 refs: pass &Option<RefTarget> into merge_ref_targets() 2023-07-18 18:12:09 +09:00
Yuya Nishihara
443391bf8f view: store Option<RefTarget> in maps, add extension trait to flatten Option
Alternatively, we can wrap BTreeMap<String, Option<RefTarget>> to flatten
Option<&Option<..>> internally, but doing that would be tedious. It would
also be unclear if map.remove(name) should construct an absent RefTarget if
the ref doesn't exist.
2023-07-18 18:12:09 +09:00
Yuya Nishihara
7461370f6e view: add wrapper that will exclude absent RefTarget entries from ContentHash
The next commit will change these maps to store Option<RefTarget> entries, but
None entries will still be omitted from the serialized data. Since ContentHash
should describe the serialized data, relying on the generic ContentHash would
cause future hash conflict where absent RefTarget entries will be preserved.

For example, ([remove], [None, add]) will be serialized as ([remove], [add]),
and deserialized to ([remove], [add, None]). If we add support for lossless
serialization, hash(([remove], [None, add])) should differ from the lossy one.
2023-07-18 18:12:09 +09:00
Austin Seipp
9d05e9902a refactor(jj-lib): remove allow(unknown_lints)
Summary: Unneeded with the MSRV bump. The other one
will have to wait until Rust 1.72.0

Signed-off-by: Austin Seipp <aseipp@pobox.com>
Change-Id: Ifb3d862aedd3f2aeb05a86ce76978d4f
2023-07-17 18:38:26 -05:00
Yuya Nishihara
0461a8575a refs: add stub constructors for absent RefTarget, replace None with it
Now we're mostly ready to reimplement RefTarget on top of
Conflict<Option<CommitId>>.
2023-07-17 08:24:24 +09:00
Yuya Nishihara
9c69a7cb15 refs: leverage Option<RefTarget> extension methods
is_fast_forward() has any() loop that defaults to true, so comment added.
2023-07-17 08:24:24 +09:00
Yuya Nishihara
4cfe2591b6 refs: turn RefTarget::adds()/removes() into iterator, rename accordingly
Since RefTarget will be reimplemented on top of Conflict<Option<CommitId>>,
we won't be able to simply return a slice of type &[CommitId]. These functions
are also renamed in order to disambiguate from Conflict::adds()/removes().
2023-07-17 08:24:24 +09:00
Martin von Zweigbergk
8e7e32710d tests: attempt to fix a flaky test
We saw a failure in `test_merge_views_git_heads()` in the GitHub CI,
but I wasn't able to reproduce it locally. Using the
`commit_transactions()` helper in the test should fix it.
2023-07-16 23:06:13 +01:00
Yuya Nishihara
a1e92563ae refs: add constructor of Some(RefTarget::Conflict { .. })
It's named after Conflict::from_legacy_form(). If RefTarget is migrated to
new Conflict type, from_legacy_form([], [add]) will create a normal target,
and from_legacy_form([], []) will be equivalent to the current None target.
That's why this function isn't named as RefTarget::conflict().
2023-07-17 01:13:21 +09:00
Yuya Nishihara
e55c533add refs: add constructor of Some(RefTarget::Normal(id))
If RefTarget is migrated to new Conflict type, this function will create
RefTarget(Conflict::resolved(Some(id))).

We still need some .unwrap() to insert Option<RefTarget> into map, but maps
will be changed to store new RefTarget type, and their mutation API will
guarantee that Conflict::resolved(None) is eliminated.
2023-07-17 01:13:21 +09:00
Yuya Nishihara
5ed070a78a view: use get_git_ref(name) where extra .clone() wouldn't matter
Perhaps, all view.get_<kind>(name) functions can return reference, but I'm
not willing to change the interface at this point. I'll revisit this after
migrating Option<RefTarget> to new Conflict-based type.
2023-07-15 05:57:49 +09:00
Yuya Nishihara
d80a719042 view: unify set/remove_local_branch() functions to take Option<RefTarget> 2023-07-15 05:57:49 +09:00
Yuya Nishihara
78eea06c36 view: unify set/remove_remote_branch() functions to take Option<RefTarget> 2023-07-15 05:57:49 +09:00
Yuya Nishihara
fc609a6d8d view: unify set/remove_tag() functions to take Option<RefTarget> 2023-07-15 05:57:49 +09:00
Yuya Nishihara
3be462266c view: unify set/remove_git_ref() functions to take Option<RefTarget>
New function takes name by reference since it doesn't always need an owned
String.
2023-07-15 05:57:49 +09:00
Yuya Nishihara
5218591a82 view: unify set/clear_git_head() functions to take Option<RefTarget>
If we migrate RefTarget to new Conflict-based type, it won't store
Conflict<CommitId>, but Conflict<Option<CommitId>>. As the Option will
be internalized, new RefTarget type will also represent an absent target.
The 'target: Option<RefTarget>' argument will be replaced with new RefTarget
type.

I've also renamed the function for consistency with the following changes.
It would be surprising if set_local_branch(name, target) could remove the
branch. I feel the name set_local_branch_target() is less confusing.
2023-07-15 05:57:49 +09:00
Waleed Khan
6d7998f8c5 working_copy: return Result from WorkingCopy::tree_state/WorkingCopy::tree_state_mut 2023-07-14 13:45:40 -07:00
Waleed Khan
60f1d7e307 working_copy: create and propagate TreeStateError 2023-07-14 13:03:57 -07:00
Yuya Nishihara
424786def1 refs: leverage Conflict::flatten() and simplify() to premerge ref targets
The order of conflicted ids slightly changed since Conflict::simplify()
tries to preserve diff pairs. It shouldn't matter so long as the result is
stable.
2023-07-12 21:29:41 +09:00
Yuya Nishihara
ddaf226108 tests: make RefTarget::Conflict { .. } assertion not rely on id order
Here we don't need to care about the order at all. We have test_refs.rs
to ensure that the order is stable.
2023-07-12 21:29:41 +09:00
Yuya Nishihara
9f06bf8be5 repo: do not discard old working-copy commit if it's pointed by local branch
It would be confusing if a branch moved backwards by checking out unrelated
commit.

#1854
2023-07-12 21:29:11 +09:00
Yuya Nishihara
f5f61f6bfe revset: resolve "HEAD@git" just like other pseudo @git branches
I don't think this would be practically useful, but consistent UX is
important.

Fixes #1843
2023-07-11 16:29:27 +09:00
Yuya Nishihara
79955d9cb3 git: do not import/export local branch named "HEAD"
Git CLI rejects it (though the data model would probably work fine with
"HEAD" branch.) This ensures that HEAD@git in JJ world is not an exported
branch named "HEAD".
2023-07-11 16:29:27 +09:00
Yuya Nishihara
9aa308fb4a revset: include pseudo @git remote in suggestion
Since collect_branch_symbols() doesn't have to be fast, I made it simply
build a new branches map.
2023-07-10 06:17:44 +09:00
Martin von Zweigbergk
aac5b7aa25 cargo: rename crates from jujutsu/jujutsu-lib to jj-cli/jj-lib
Almost everyone calls the project "jj", and there seeems to be
consensus that we should rename the crates. I originally wanted the
crates to be called `jj` and `jj-lib`, but `jj` was already
taken. `jj-cli` is probably at least as good for it anyway.

Once we've published a 0.8.0 under the new names, we'll release 0.7.1
versions under the old names with pointers to the new crates names.
2023-07-09 06:40:43 +02:00
Yuya Nishihara
3e294ca2d6 revset: do not suggest deleted local branches, suggest @remote branches
It seemed too verbose to always include @remote branches, so synced remotes
are omitted by default. If the given symbol contained '@', all remote symbols
are populated so that the distance of remote fragment is taken into account.
2023-07-09 10:42:14 +09:00
Yuya Nishihara
4a5060a618 revset: don't resolve deleted branch symbol to empty set
A deleted branch disappears immediately if there's no remote counterpart,
so I don't think a local name should be resolvable like zombie.
2023-07-09 10:42:14 +09:00
Yuya Nishihara
5c1352d31c revset: add tests for branch symbol resolution
I'm going to fix resolution of remote-only branches. This also includes some
typo tests because I need to fix suggestion as well.
2023-07-09 10:42:14 +09:00
Yuya Nishihara
564506a7c7 tests: fix test_fetch_prune_deleted_ref() to set up refs on remote
This is broken since aa78f97d55 "git: refactor tests by extracting some
common setup."
2023-07-09 10:08:46 +09:00
Waleed Khan
ef83f2beeb feat(fsmonitor): Watchman filesystem monitor implementation 2023-07-08 18:48:14 +03:00
Waleed Khan
092dce0625 refactor(working_copy): create SnapshotOptions struct
Required in a later commit.
2023-07-08 18:48:14 +03:00
Ilya Grigoriev
b6a9423f38 git export: (almost) no-op refactor to export_refs to use RefName
This follows 3779b45, but in this case the refactor makes the logic more
complicated. The main goal here is to prepare for the next commit.
2023-07-03 11:01:22 -07:00
Yuya Nishihara
a07574a233 git: pass RefName enum to git_ref_filter callback
I think it's slightly better to compare each ref fragment than building
"refs/remotes/{remote}/{branch}" pattern to be matched.
2023-07-02 09:49:07 +09:00
Martin von Zweigbergk
c0ffce781e store: cache tree on write and return it
This matches what we do when writing commits.
2023-06-30 14:12:36 +02:00
Martin von Zweigbergk
134efabcef test_merge_trees: make merge_trees() wrapper's signature match original 2023-06-30 14:12:36 +02:00
Martin von Zweigbergk
779b8ba318 files: replace uses of MergeHunk by Conflict<ContentHunk>
Since `Conflict`s can represent the resolved state, so
`Conflict<ContentHunk>` can represent the states that we use
`MergeHunk` for. `MergeHunk` does force the user to handle the
resolved case, which may be useful. I suppose one could use the same
argument for making `Conflict` an enum, i.e. if we think that
`MergeHunk`'s two variants are beneficial, then we should consider
making `Conflict` an enum with those two variants.
2023-06-28 06:51:37 +02:00
Martin von Zweigbergk
c625e9352d files: make MergeHunk::Conflict be a Conflict<ContentHunk>
The `ConflictHunk` type doesn't add anything over
`Conflict<ContentHunk>`.
2023-06-27 21:06:32 +02:00
Martin von Zweigbergk
b1f2e80349 files: add a newtype around Vec<u8> for content hunks
It's useful to have a more readable `Debug` format for `Vec<u8>`
(`"foo"` is better than `[102, 111, 111]`). It might also make types
in function signatures and elsewhere more readable.
2023-06-27 21:06:32 +02:00
Glen Choo
7afaa2487b git: add .gitmodules parser
This only parses the fields relevant to us, i.e.:

- name: the stable identifier of the submodule
- path: the path to the submodule in the current commit
- url: the remote we can clone the submodule from

The full list of .gitmodules fields can be found at
https://git-scm.com/docs/gitmodules.
2023-06-27 10:07:00 -07:00
Kevin Liao
86b6a11e63 Fix jj init --git-repo fails and leaves broken .jj folder
This commit fixes #1305

Before this commit, running `jj init --git-repo=./` in a folder that
does not have a .git would cause jj to panick and leave an unfinished corrupted jj repo.

This commit fixes that by changing the call chain to return an error
instead of calling .unwrap() and panicking. This commit also adds logic to delete the unfinished jj
repository when the git backend initialization failed.

Before this commit, running the above command would result in the following
```
Running `jj/target/debug/jj init --git-repo=./`
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error { code: -3, klass: 2, message: "failed to resolve path '/Users/kevincliao/github/jj/test-repo/.jj/repo/store/../../../.git': No such file or directory" }', lib/src/git_backend.rs:83:75
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
```

After this commit, the result is the following and the jj repo is deleted:
```
Running `jj/target/debug/jj init --git-repo=./`
Error: Failed to access the repository: Error: Failed to open git repository: failed to resolve path '/Users/kevincliao/github/jj/test-repo/.jj/repo/store/../../../.git': No such file or directory; class=Os (2); code=NotFound (-3)
```
2023-06-20 11:02:06 -07:00
Glen Choo
6621f261cc repo: add submodule_store, default impl
..and other assorted boilerplate. These are just stubs for now, but now
that we've reserved the `submodule_store` subdirectory, we can start
adding more functionality.
2023-06-19 09:48:58 -07:00
Martin von Zweigbergk
82883e648d conflicts: move describe_conflict() etc. onto Conflict
Before we had `conflicts::Conflict`, most of these functions took a
`backend::Conflict`. I think I didn't want to pollute the `backend`
module with this kind of logic, trying to keep it focused on
storage. Now that we have the type in `conflicts`, however, I think it
makes sense to move these functions onto it.
2023-06-19 07:05:02 +02:00
Yuya Nishihara
a67d8b5a65 index: turn CompositeIndex::walk_revs() into position-based API
This gets rid of round-trip conversion from queries like "(main..)-". I have
such expression in my default log/disambiguation revset, and the query could
take ~150ms to convert head positions back and forth if the repository had
tons of unmerged commits.
2023-06-19 13:41:43 +09:00
Martin von Zweigbergk
19fd8a917a conflicts: remove ConflictId from update_conflict_from_content()
For tree-level conflicts (#1624), I plan to remove `ConflictId`
completely. This commit removes `ConflictId` from
`update_conflict_from_content()` by instead making it take a
`Conflict<Option<TreeValue>>` and return a possibly different such
value.

I made the call site in `working_copy` avoid writing the conflict to
the store if it's unchanged, but I didn't make the same optimization
in `merge_tools` becuase it's much more likely to have changed there.
2023-06-13 08:49:46 +02:00
Ilya Grigoriev
096538ba18 revsets: stop jj parsing br as a git_ref refs/heads/br
Use `br@git` instead.

Before, if there is not a local branch `br`, jj tried to resolve
it as a git ref `refs/heads/br`. Unchanged from before, `br` can
still be resolved as a tag `refs/tag/br`.
2023-06-12 14:31:44 -07:00
Ilya Grigoriev
738f99ddf1 test_revset.rs: disable nightly clippy false-positive warning
I opened a bug for the clippy error:
https://github.com/frondeus/test-case/issues/122
2023-06-11 13:11:01 -07:00
Waleed Khan
23a4124d20 feat(revset): suggest similar branch names 2023-06-05 11:11:17 -05:00
Martin von Zweigbergk
29b676f24f store: do conversion to/from backend::Conflict
We now convert to/from `backend::Conflict` right before/after calling
the `Store` methods, so we can simplify by having the `Store` do the
conversion.
2023-06-04 06:48:34 -07:00
Martin von Zweigbergk
c378503991 conflicts: replace remaining uses of backend::Conflict 2023-06-04 06:48:34 -07:00
Martin von Zweigbergk
f4499aa65e conflicts: fix bug when modifying modify/delete conflicts
Currently, if the user modifies a modify/delete conflict, we always
consider the result resolved. That happens because we materialize the
missing side of the conflict as an empty string but when we parse the
conflict, we expect only the number of sides in the input
conflict. For example, if the input is a regular modify/delete
conflict with one remove and one add, the materialized markers will
have one remove and two adds (one of them empty), but when we try to
parse it, we expect one remove and only one add. When we fail to parse
it, we consider it resolved.

This commit fixes the bug by using
`conflicts::Conflict<Option<TreeValue>>` and keeping track of which
sides were supposed to be empty. We could have fixed the bug without
switching to `conflicts::Conflict`, but we want to switch anyway, and
the fix happens naturally when switching.
2023-06-04 06:48:34 -07:00
Yuya Nishihara
78a64edb22 dag_walk: rename bfs() to dfs() because it's depth-first
No callers appear to rely on traversal order, so let's just fix the function
name. Maybe it was a typo.
2023-06-04 11:47:49 +09:00
Yuya Nishihara
3ba544414c dag_walk: unbox bfs() callback, use iter::from_fn() to implement iterator
I just wanted to remove syntactic noise from callers. iter::from_fn() helps
to avoid declaring struct with lots of type parameters.
2023-06-04 11:47:49 +09:00
Yuya Nishihara
3d449c55b7 tree_builder: do not omit file entry which was previously a directory 2023-06-01 09:38:06 +09:00
Yuya Nishihara
ae9f7aba52 tests: add test for file-directory-file transition
TreeBuilder fails to handle directory-file transition right now, and leaves
the dirty file as "clean".
2023-06-01 09:38:06 +09:00
Martin von Zweigbergk
fb17e6a50e revset: use different errors for ambiguous commit/change IDs
I made a typo and got something like this:

```
Error: Commit or change id prefix "wl" is ambiguous
```

Since we can tell commit ids from change ids these days, let's make
the error message say which kind of id it is. Changing that also kind
of forced me to make a special error for empty strings. Otherwise we
would have to arbitrarily say that an empty string is a commit id or
change id. A specific error message for empty strings seems helpful,
so that's probably for the better anyway.
2023-05-31 06:28:32 -07:00
Yuya Nishihara
b7b9b8c88e index: pass only CompositeIndex to default_revset_engine::evaluate() 2023-05-29 08:15:40 +09:00
Yuya Nishihara
cf5cb380bb index: implement Index for CompositeIndex
We can't get rid of the other "impl Index"es because .as_composite() must
return a real reference type. Maybe we could turn CompositeIndex into an
owned wrapper, but I don't know if that would be worth the effort.
2023-05-29 08:15:40 +09:00
Yuya Nishihara
5989bdf781 index: move Index::as_any() to MutableIndex, obtain CompositeIndex from there
It might sound scary to add public .mutable_index() accessor, but I think
it's okay because immutable MutableIndex reference has no more power than
Index.

This allows us to implement Index for lifetime-bound type such as
CompositeIndex<'_>.
2023-05-29 08:15:40 +09:00
Yuya Nishihara
93284a153f index: obtain CompositeIndex from ReadonlyIndexWrapper
I'll remove Index::as_any() so that Index can be implemented for reference
wrapper.
2023-05-29 08:15:40 +09:00
Yuya Nishihara
fb77c55268 index: use as_composite() to access to index stats
The idea is that .as_composite() is equivalent to .as_index(), but for the
implementation type. I'm going to add "impl Index for CompositeIndex" to
clean up index references passed to revset engine.
2023-05-29 08:15:40 +09:00
Martin von Zweigbergk
f838d083d3 tree: add test of merge of executable bit
We didn't seem to have any tests of this, so let's add one before I
change the implementation.
2023-05-24 22:00:38 -07:00
Yuya Nishihara
e24fe817c9 tests: invoke .walk_revs() through CompositeIndex
Prepares for removal of the index trait method.
2023-05-24 01:02:37 +09:00
Yuya Nishihara
b01614bbdd cleanup: leverage scoped thread in tests 2023-05-21 21:02:58 +09:00
Yuya Nishihara
38a7e7fd62 git_backend: on read_commit(), bulk-update extra metadata table of ancestors
Otherwise, "jj init --git-repo ." would create extra table files per commit,
and merge them.

I considered adding an explicit GitBackend method to be called from
git::import_refs(), but the call order matters. The method should be invoked
before calling store.get_commit(..) or mut_repo.add_head(..). Since commits
are likely to be loaded from the head, we can instead make read_commit()
import ancestor metadata at all.

Alternatively, we could make a Git commit hidden until it's inserted into
the extra table. It's rather big change, and I wouldn't like to do that
without thinking more thoroughly.
2023-05-21 08:29:00 +09:00
Yuya Nishihara
a9422460cb git_backend: ensure change id generated from git commit id never reassigned
Fixes #924
2023-05-20 15:53:23 +09:00
Yuya Nishihara
9aa72f6f1d git_backend: add lock to prevent racy change id assignments
My first attempt was to fix up corrupted index when merging, but it turned
out to be not easy because the self side may contain corrupted data. It's
also possible that two concurrent commit operations have exactly the same
view state (because change id isn't hashed into commit id), and only the
table heads diverge.

#924
2023-05-20 15:53:23 +09:00
Yuya Nishihara
3655da4f01 tests: add tests for concurrent git commit/change id assignment
Since non-Git metadata isn't hashed, we can't rely on the consistency
provided by content-addressed storage. The problem is also described in
https://github.com/martinvonz/jj/issues/3#issuecomment-947998487

#924
2023-05-20 15:53:23 +09:00
Ilya Grigoriev
714aff63e6 git.rs: properly abandon commits from moved/deleted branches on remote (#864)
This bug concerns the way `import_refs` that gets called by `fetch` computes
the heads that should be visible after the import.

Previously, the list of such heads was computed *before* local branches were
updated based on changes to the remote branches. So, commits that should have
been abandoned based on this update of the local branches weren't properly
abandoned.

Now, `import_refs` tracks the heads that need to be visible because of some ref
in a mapping keyed by the ref. If the ref moves or is deleted, the
corresponding heads are updated.

Fixes #864
2023-05-17 17:57:58 -07:00
Ilya Grigoriev
cf4a603eb4 Tests demonstrating a similar bug with moved rather than deleted branch 2023-05-17 17:57:58 -07:00
Ilya Grigoriev
a0ee2b0dbd lib/tests/test_git.rs: New test to demonstrate #864's root cause 2023-05-17 17:57:58 -07:00
Ilya Grigoriev
bda3d3e50b test_import_refs_reimport: very minor improvement to a test 2023-05-17 17:57:58 -07:00
Martin von Zweigbergk
87a925d736 git_backend: return timestamps for what was actually written
Now that we return the written commit from `write_commit()`, let's
make the timestamps match what was actually written, accounting for
the whole-second precision and the adjustment we do to avoid
collisions.
2023-05-12 15:20:44 -07:00
Martin von Zweigbergk
e7419e76a1 backend: replace git_repo() by as_any()
This has several advantages:

 * Makes it possible to downcast to non-Git custom backends (might be
   useful at Google, but we haven't needed it yet)

 * Lets us access more specific functionality on the `GitBackend`,
   making it possible to access the `git2::Repository` without
   creating a copy of it.

 * Removes the dependency on Git from the backend
2023-05-12 08:05:09 -07:00
Yuya Nishihara
f58beca760 revset: move resolve_symbol() to tests
It's no longer used in library code.
2023-05-12 21:31:29 +09:00
Martin von Zweigbergk
916b00c33e id_prefix: remove repo field from IdPrefixContext
By passing the repo as argument to the methods instead, we can remove
the `repo` field and the associated lifetime. Thanks to Yuya for the
suggestion.
2023-05-11 23:41:24 -07:00
Martin von Zweigbergk
6a4502cb5d prefixes: allow resolving shorter ids within a revset
In large repos, the unique prefixes can get somewhat long (~6 hex
digits seems typical in the Linux repo), which makes them less useful
for manually entering on the CLI. The user typically cares most about
a small set of commits, so it would be nice to give shorter unique ids
to those. That's what Mercurial enables with its
`experimental.revisions.disambiguatewithin` config. This commit
provides an implementation of that feature in `IdPrefixContext`.

In very large repos, it can also be slow to calculate the unique
prefixes, especially if it involves a request to a server. This
feature becomes much more important in such repos.
2023-05-11 23:41:24 -07:00
Martin von Zweigbergk
efd743339c revset: don't allow symbols in RevsetExpression::resolve()
When creating `RevsetExpression` programmatically, I think we should
use commit ids instead of symbols in the expression. This commit adds
a check for that by using a `SymbolResolver` that always errors
out.
2023-05-11 23:41:24 -07:00
Martin von Zweigbergk
99e9cd70d1 cli: make WorkspaceCommandHelper create SymbolResolver
I would eventually want the `SymbolResolver` to be customizable (in
custom `jj` binaries), so we want to make sure we always use the
customized version of it.

I left `RevsetExpression::resolve()` unchanged. I consider that to be
for programmatically created expressions.
2023-05-11 23:41:24 -07:00
Yuya Nishihara
92cfffd843 git: on external HEAD move, do not abandon old branch
The current behavior was introduced by 20eb9ecec1 "git: don't abandon
HEAD commit when it loses a branch." While the change made HEAD mutation
behavior more consistent with a plain ref operation, HEAD can also move on
checkout, and checkout shouldn't be considered a history rewriting operation.

I'm not saying the new behavior is always correct, but I think it's safer
than losing old HEAD branch. I also think this change will help if we want
to extract HEAD management function from git::import_refs().

Fixes #1042.
2023-05-11 10:15:31 +09:00
Yuya Nishihara
66d405fa5f git: add tests that simulate external checkout/amend in colocated repo
I'm going to change the behavior of _without_ref() case to mitigate #1042.
2023-05-11 10:15:31 +09:00
Benjamin Saunders
ccbb34fddb working_copy: introduce snapshot progress callback 2023-05-06 11:07:46 -07:00
Yuya Nishihara
837e8aa81a revset: add substitution rule for nested descendants/children
The substitution rule and tests are copied from ancestors/parents. The backend
logic will be reimplemented later. For now, it naively repeats children().
2023-04-24 20:45:13 +09:00
Martin von Zweigbergk
c60f14899a index: remove entry_by_id() from trait
It no longer needs to be on the `Index` trait, thereby removing the
last direct use of `IndexEntry` in the trait (it's still used
indirectly in `walk_revs()`).
2023-04-18 18:32:23 -07:00
Yuya Nishihara
5351371d51 revset: resolve visible heads prior to evaluation 2023-04-10 00:39:58 +09:00
Yuya Nishihara
0fcc13a6f4 revset: make resolve() return different type describing evaluation plan
New ResolvedExpression enum ensures that the evaluation engine doesn't have
to know the symbol resolution details. In this commit, I've moved Filter
and NotIn resolution to resolve_visibility(). Implicit All/VisibleHeads
resolution will be migrated later.

It's tempting to combine resolve_symbols() and resolve_visibility() to get
rid of panic!()s, but the resolution might have to be two passes to first
resolve&collect explicit commit ids, and then substitute "all()" with
"(:visible_heads())|commit_id|..". It's also possible to apply some tree
transformation after symbol resolution.
2023-04-10 00:39:58 +09:00
Yuya Nishihara
6c2525cb93 revset: add "resolve" method to RevsetExpression, always call it
I'll make the resolution stage mandatory, and have it return a "resolved"
type. RevsetExpression::evaluate() will be moved to the "resolved" type.
2023-04-10 00:39:58 +09:00
Yuya Nishihara
adfd52445b revset: reimplement children to not scan visible ancestors twice
It's slightly faster, and removes the use of RevsetExpression::descendants()
API.
2023-04-08 12:13:30 +09:00
Yuya Nishihara
85fb1f74c3 revset: for roots:heads, terminate ancestor lookup at min(roots) 2023-04-08 12:13:30 +09:00
Martin von Zweigbergk
24a512683b revset: add a revset function for finding commits with conflicts
This adds `conflict()` revset that selects commits with conflicts. We
may want to extend it later to consider only conflicts at certain
paths.
2023-04-06 16:46:21 -07:00
Martin von Zweigbergk
e1c57338a1 revset: split out no-args head() to visible_heads()
The `heads()` revset function with one argument is the counterpart to
`roots()`. Without arguments, it returns the visible heads in the
repo, i.e. `heads(all())`. The two use cases are quite different, and
I think it would be good to clarify that the no-arg form returns the
visible heads, so let's split that out to a new `visible_heads()`
function.
2023-04-03 23:46:34 -07:00
Yuya Nishihara
982062bd75 revset: do not always evaluate filter node to InternalRevset
This basically removes hidden 'all() &' from union/negation of filters. To
achieve that, I have two options: 1. add separate evaluation path (like the
one this commit introduced), or 2. wrap "all()" revset to override predicate
as Box::new(|_| true) function. I took the former since it's less ad-hoc.

We can add an explicit RevsetExpression node to branch between evaluate()
and evaluate_predicate(), but I don't think it would simplify the
implementation at this point. We might need such node if we want to resolve
"all()" at resolve_symbols(). It might be even better to extract a subset of
RevsetExpression enum, which only contains evaluatable nodes.

The cost of 'all() &' isn't significant for most filters. '~merges()' is
the exception. For jj repo,

    revsets/:v0.3.0 & (author(martinvonz) | committer(martinvonz))
    --------------------------------------------------------------
    base     1.06      11.2±0.04m
    new      1.00      10.5±0.05m

    revsets/~merges()
    -----------------
    base     1.69     750.0±8.47µ
    new      1.00     444.1±3.50µ
2023-04-04 15:21:21 +09:00
Yuya Nishihara
c28d2d7784 revset: split RevsetError into RevsetResolution/EvaluationError
This makes it clear that RevsetExpression::Present node is noop at the
evaluation stage.

RevsetEvaluationError::StoreError is unused right now, but I'm not sure if
it should be removed. It makes some sense that evaluate() can propagate
StoreError as it has access to the store.
2023-04-03 10:55:03 +09:00
Martin von Zweigbergk
3546cc1bf6 revset: pass in store, index, and heads instead of whole Repo
The `Repo` is a higher-level type that the index shouldn't have to
know about. With this change, a custom revset implementation should be
able evaluate the revset on a server without knowing which repo it
refers to.
2023-03-30 20:15:45 -07:00
Martin von Zweigbergk
3ff1ab520b revset: remove public_heads()
The `public_heads()` revset only contains the root commit in
practice. I'm not sure what we want to do about phases, but since we
don't have any real support for them yet, let's just remove this
revset. I didn't update the changelog because we don't seem to have
documented the revset function (and it seems unlikely that users who
found out about it found it useful enough to use it when they could
just use `root`).
2023-03-30 20:15:45 -07:00
Yuya Nishihara
0532301e03 revset: add latest(candidates, count) predicate
This serves the role of limit() in Mercurial. Since revsets in JJ is
(conceptually) an unordered set, a "limit" predicate should define its
ordering criteria. That's why the added predicate is named as "latest".

Closes #1110
2023-03-25 23:48:50 +09:00
Martin von Zweigbergk
baea314fc0 index: get generation number from specific impl in test 2023-03-24 10:09:40 -07:00
Martin von Zweigbergk
75605e36af revset: iterate over commit ids instead of index entries
There are no remaining places where we iterate over a revset and need
the `IndexEntry`s, so we can now make `Revset::iter()` yield
`CommitId`s instead.
2023-03-23 21:58:15 -07:00
Martin von Zweigbergk
b5ea79f32e revset: add new graph iterator function for tests
I'm about to make `Revset::iter()` yield just `CommitId`s, but the
tests in `test_default_revset_graph_iterator.rs` need an `IndexEntry`
iterator so they can pass it into `RevsetGraphIterator::new()`. This
commits prepares for the change by adding a
`RevsetImpl::iter_graph_impl()` that returns `RevsetGraphIterator`,
keeping `InternalRevset` still hidden within the revset engine. We
could instead have made that (and `ToPredicateFn`) visible to tests. I
can't say which is better.
2023-03-23 21:58:15 -07:00
Martin von Zweigbergk
c8f387d5b3 revset: pass IndexEntry iterator to graph iterator
The graph iterator is specific to the index implementation, and it
needs access to `IndexEntry`, which `Revset::iter()` will soon not
yield.
2023-03-23 21:58:15 -07:00
Martin von Zweigbergk
27a7fccefa revset: add a method returning a change id index
One of the remaining places we depend on index positions is when
creating a `ChangeIdIndex`. This moves that into the revset engine
(which is coupled to the commit index implementation) by adding a
`Revset::change_id_index()` method. We will also use this function
later when add support for resolving change id prefixes within a small
revset.

The current implementation simply creates an in-memory index using the
existing `IdIndex` we have in `repo.rs`.

The custom implementation at Google might do the same for small
revsets that are available on the client, but for revsets involving
many commits on the server, it might use a suboptimmal implementation
that uses longer-than-necessary prefixes for performance reasons. That
can be done by querying a server-side index including changes not in
the revset, and then verifying that the resulting commits are actually
in the revset.
2023-03-23 20:49:15 -07:00
Martin von Zweigbergk
d3cf543abc revset: move revset_for_commits() to test
The function is only used in tests, so it doesn't belong in
`default_revset_engine`. Also, it's not specific to that
implementation, so I rewrote as a revset evaluation.
2023-03-23 04:50:33 -07:00
Martin von Zweigbergk
5f74dd5db3 repo: implement Repo on ReadonlyRepo instead of its Arc
I'd like to be able to pass a `self` of `type `&ReadonlyRepo` to
functions that take a `&dyn Repo`. For that, we need `ReadonlyRepo`
itself to implement `Repo` instead of having `Arc<ReadonlyRepo>`
implement it. I could have solved it in a different way, but the `Arc`
requirement seems like an unnecessary constraint.
2023-03-21 21:43:44 -07:00
Martin von Zweigbergk
01d7239732 revset: make graph iterator yield commit ids (not index entries)
We only need `CommitId`s, and `IndexEntry` is specific to the default
index implementation.
2023-03-20 01:45:54 -07:00
Martin von Zweigbergk
2f876861ae graphlog: key by commit id (not index position)
The index position is specific to the default index implementation and
we don't want to use it in outside of there. This commit removes the
use of it as a key for nodes in the graphlog.

I timed it on the git.git repo using `jj log -r 'all()' -T commit_id`
(the worst case I can think of) and it slowed down from ~2.02 s to
~2.20 s (~9%).
2023-03-20 01:45:54 -07:00
Martin von Zweigbergk
91df7ec4c5 revset: rename graph iterator test to match implementation 2023-03-20 01:45:54 -07:00
Martin von Zweigbergk
f758b646a9 commit_builder: add accessors for most fields
I'd like to be able to access the current committer on a
`CommitBuilder`.
2023-03-19 00:48:05 -07:00
Martin von Zweigbergk
70d4a0f42e revset: remove context parameter from evaluate()
The `RevsetWorkspaceContext` argument is now instead used by the new
`resolve_symbol()` function.
2023-03-17 22:42:41 -07:00
Martin von Zweigbergk
d971148e4e revset: move resolve_symbol() back to revset module
The only caller is now in `revset.rs`.
2023-03-17 22:42:41 -07:00
Martin von Zweigbergk
94aec90bee revset: resolve symbols earlier, before passing to revset engine
For large repos, it's useful to be able to use shorter change id and
commit id prefixes by resolving the prefix in a limited subset of the
repo (typically the same subset that you'd want to see in your default
log output). For very large repos, like Google's internal one, the
shortest unique prefix evaluated within the whole repo is practically
useless because it's long enough that the user would want to copy and
paste it anyway.

Mercurial supports this with its `revisions.disambiguatewithin` config
(added in https://www.mercurial-scm.org/repo/hg/rev/503f936489dd). I'd
like to add the same feature to jj. Mercurial's implementation works
by attempting to resolve the prefix in the whole repo and then, if the
prefix was ambiguous, it resolves it in the configured subset
instead. The advantage of doing it that way is that there's no extra
cost of resolving the revset defining the subset if the prefix was not
ambiguous within the whole repo. However, there are two important
reasons to do it differently in jj:

* We support very large repos using custom backends, and it's probably
  cheaper to resolve a prefix within the subset because it can all be
  cached on the client. Resolving the prefix within the whole repo
  requires a roundtrip to the server.

* We want to be able to resolve change id prefixes, which is always
  done in *some* revset. That revset is currently `all()`, i.e. all
  visible commits. Even on local disk, it's probably cheaper to
  resolve a small revset first and then resolve the prefix within that
  than it is to build up the index of all visible change ids.

We could achieve the goal by letting each revset engine respect the
configured subset, but since the solution proposed above makes sense
also for local-disk repos, I think it's better to do it outside of the
revset engine, so all revset engines can share the code.

This commit prepares for the new functionality by moving the symbol
resolution out of `Index::evaluate_revset()`.
2023-03-17 22:42:41 -07:00