This will help deduplicate template parsing functions. We don't care about the
cost of config.get_string(), but I don't want to copy the config key to every
caller.
This could be a provided method of the TemplateLanguage trait, but it's
unlikely that this method would have to be customized by implementors. And
I'm going to add thin wrapper method to CommandHelper, so no users would
write language.parse(..) anyway.
I'm going to split commit/operation_templater::parse() into two parts, and
the first half will be Commit/OperationTemplateLanguage::new(..). This patch
also makes CommitTemplateLanguage::wrap_() functions public because extension
methods should be able to return property of these types.
`cargo doc` complains that two URLs aren't actually links:
```
warning: this URL is not a hyperlink
--> lib/src/fsmonitor.rs:66:6
|
66 | /// (https://facebook.github.io/watchman/). Requires `watchman` to already be
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: use an automatic link instead: `<https://facebook.github.io/watchman/>`
|
= note: bare URLs are not automatically turned into clickable links
= note: `#[warn(rustdoc::bare_urls)]` on by default
warning: `jj-lib` (lib doc) generated 1 warning (run `cargo fix --lib -p jj-lib` to apply 1 suggestion)
Documenting jj-cli v0.14.0 (/Users/emesterhazy/oss/github.com/martinvonz/jj/cli)
Documenting testutils v0.14.0 (/Users/emesterhazy/oss/github.com/martinvonz/jj/lib/testutils)
warning: this URL is not a hyperlink
--> cli/src/cli_util.rs:2077:41
|
2077 | /// To get started, see the tutorial at https://github.com/martinvonz/jj/blob/main/docs/tutorial.md.
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: use an automatic link instead: `<https://github.com/martinvonz/jj/blob/main/docs/tutorial.md.>`
|
= note: bare URLs are not automatically turned into clickable links
= note: `#[warn(rustdoc::bare_urls)]` on by default
warning: `jj-cli` (lib doc) generated 1 warning (run `cargo fix --lib -p jj-cli` to apply 1 suggestion)
```
This commit fixes the warnings by making the watchman URL a hyperlink and by
disabling the lint for the jj-cli error. Disabling the link is the right thing
to do because the comment is captured by clap and printed when `jj --help`
runs and any markdown formatting like `<>` is passed through.
The ExternalMergeTool struct has four 24-byte fields plus one bool. It could
be shrunk by dropping Vec/String capacity, but the resulting type would still
be bigger compared to the default Builtin variant.
I considered inlining tx.select_diff(), but that looked a bit cryptic because
the arguments orders are reasonably different. This thin wrapper will help
enforce the common interactive editing behavior.
This constructor has too many arguments enough to introduce a parameter struct,
which would be identical to the CommandHelper type. Let's simply inline it as
there are no external callers.
write!(ui.hint(), ..) error is suppressed because it seemed weird if the
configuration error had io::Error variant. The write error isn't important
anyway.
This moves the config loading closer to CLI args where --tool=<name> option
will be processed. The factory function are proxied through the command helper
so that the base_ignores can be attached there later.
I'm going to make them be loaded by caller, and these newtypes will provide
extra compile-time safety (plus nicer API to be added later.) The error types
will be cleaned up later patches.
This gets rid of the last UserSettings dependency from edit_diff_external().
I'm going to remove it from edit_diff() too, and let callers pass a
preconfigured MergeTool struct instead.
These changes will make it easier to add --tool=<name> argument #2575.
Because the snapshot directory is removed at the end of the function, it doesn't
make sense to enable watchman in it. The max_new_file_size parameter might be
somewhat useful, but it's unlikely that the temporary directory contains
gigantic node_modules tree for example. OTOH, the base_ignores matters since it
may contain common ignore patterns like *~.
This eliminates most of the UserSettings dependencies from this function.
I'm going to add string.len() method which will return a length in bytes. The
number of the UTF-8 code points is useless metrics, and strings here are often
ASCII bytes, so let's simply use byte indices in substr().
If the given index is not at a char boundary, it will be rounded. I considered
making it an error, but that would be annoying. I would want to see something
printed by author.name().substr() even if it contained latin characters.
I've extracted index normalization function which might be used by other string
methods. The remaining part of substr() is trivial, so inlined it.
Before, --tool=:builtin argument was ignored and the tool was loaded from
"ui.diff.tool" option. Since there is no single builtin diff format, :builtin
doesn't make sense here. Maybe we can translate ":<format>" to the internal
diff format instead, but that will also mean "ui.diff.tool" and ".format" can
be merged.
This partially reverts 409356fa5b "merge_tools: enable `:builtin` as default
diff/merge editor."
I'm going to split get_tool_config() to fix "diff --tool=:builtin", and it
doesn't make sense to duplicate get_tool_config_from_args() per backing
get_tool_config() functions.
This make :builtin render conflicts as conflict markers instead of
panicking. To support conflicts properly, we also need to parse the
conflict markers (calling `update_from_content()`) after the user
closes the editor.
A runtime error will be printed inline. This simplifies error handling, and I
think it's better behavior overall. "jj log" won't be terminated just because
"gpg" crashed during signature verification for example. When property
evaluation failed, the error propagates to the closest template expression, and
the error message is printed there. Then, template output continues as long as
the output stream is open.
If we add revset() function for example, dynamic revset evaluation error will be
displayed inline. Static revset expression will still be processed at parsing
phase (to cache the evaluation result), and the error will be reported early.
One caveat: a string argument passed to e.g. .contains(needle) can be a
template, so the evaluation error would be swallowed there.
It was moved to CLI at 42252a2f00 "cli: on `jj init --git-repo=.`, use
relative path to `.git/`." As far as I can tell, .canonicalize() is needed
to calculate relative path, which is now processed differently in
Workspace::init_external_git() and GitBackend::init_external().
This reverts dc074363d1 "no-op: Move external git repo canonicalization into
Workspace::init_git_external." As I said in the PR comment, appending ".git"
is normalization of the user input, which is IMHO more appropriate to be done
in the CLI layer.
The translation from method error to keyword error can go wrong if the context
object had n-ary methods (n > 0), which isn't the case as of now. For
simplicity, arguments error is mapped to "self.<name>(..)" suggestion.
Local variables and "self" could be merged without using extra method, but
we'll need extend_*_candidates() to merge in symbol/function aliases anyway.
This seems more useful if aliases are nested. The innermost error usually
contains the problem, and the outer errors are contexts where aliases are
expanded.
Except for the generic list and template methods. We'll need a bit more
refactoring to migrate List<T> method builders to be compatible with
non-capturing fn() type.
The recovery commit we create when we run into a stale working copy
with a missing operation currently has an empty tree. Our commit
backend at Google creates an index of which files changed in each
commit. That gets really expensive when a commit deletes all files in
the repo, as these recovery commits do. So for our backend, it is much
better to make the recovery commit empty instead. That's what this
patch does.
It almost doesn't matter functionally what tree we use for it since we
don't care much about the current tree when snapshotting the working
copy. It does matter in a few cases, however. One case is for
conflicts. In that case, it's likely better to use the recovery
commit's parent as base tree (as we do by making the recovery commit
empty) than to use an empty tree, as that would guarantee that all
conflicts would be considered resolved. (Side note: perhaps we should
start looking at the current commit's parent instead of looking at the
current commit when snapshotting, but that's a topic for another day.)
When we abandon a working-copy commit, we create a new working-copy
commit on top. This behave is very useful, but it's not obvious. Let's
document it.
Thankfully, 2bbefcc338 (rewrite: default to not simplifying ancestor
merges) means that there are much fewer commands where we need to
document this behavior.
There are two major goals:
* provide typo hints in a similar way to revset
* make methods extensible
The created method table is bound to the 'repo lifetime because of the problem
described in the inline comment. It would be nice if we can build cachable
core method table for<'repo> CommitTemplateLanguage<'repo, '_>, but I couldn't
figure out how.
Add an option to list tracked branches only
This option keeps most of the current `--all` printing logic, but:
- Omits local Git-tracking branches by default (can be extended to
support filtering by remote).
- Skip over the branch altogether if it doesn't contain tracked remotes
- Don't print the untracked_remote_refs at the end
Usage:
`jj branch list -t`
`jj branch list --tracked`
`jj branch list --tracked <branch name>`
I think the user usually wants to abandon only newly empty commits. I
think they should use `jj abandon` if they want to get rid of already
empty commits. By keeping already empty commits, we don't need to
special-case the working copy and merge commits.
This allows us to call alias function with the top-level object.
For convenience, all self.<method>()s are available as keywords. I don't think
we'll want to deprecate them. It would be tedious if we had to specify
-T'self.commit_id()' instead of -Tcommit_id.
If I remember correctly, wrap_fn() was added to help type inference. It no
longer makes sense because the type is coerced by TemplateFunction::new()
and language.wrap_*().
The description of `jj diff` was lost in commit b5e4e670. We later got
a short description for it in b5e4e670. This patch restores the
original description.
This eliminates the separate keywords table. All keywords are resolved through
the pseudo "self" property. Maybe we'll add "self" keyword/variable later.
This is copied from the commit templater. I'm going to extract the "self"
property handling to the core template builder, and build_keyword() methods
will be replaced with that.
The default immutable_heads() includes tags(), which makes sense, but computing
heads(tags()) can be expensive because the tags() set is usually sparse. For
example, "jj bench revset 'heads(tags())'" took 157ms in my linux stable
mirror. We can of course optimize the heads evaluation by using bit set or
segmented index, but the query includes many historical heads if the repository
has per-release branches, which are uninteresting anyway. So, this patch
replaces heads(immutable_heads()) with trunk().
The reason we include heads(immutable_heads()) is to mitigate the following
problem. Suppose trunk() is the branch to be based off, I think using trunk()
here is pretty good.
```
A B
*---*----* trunk() ⊆ immutable_heads()
\
* C
```
https://github.com/martinvonz/jj/pull/2247#discussion_r1335078879
In my linux stable mirror, this makes the default log revset evaluation super
fast. immutable_heads(), if configured properly, includes many historical
branch heads which are also the visible heads.
revsets/immutable_heads()..
---------------------------
0 12.27 117.1±0.77m
3 1.00 9.5±0.08m
It should be useful at least in the presentation layer to know which
operations correspond to working-copy snapshots. They might be
rendered differently in the graph, for example. Or maybe an undo
command wants to warn if you just undid a snapshot operation. This
patch just introduces a field in the metadata to store the
information.
I think I prefer this behavior because it's less lossy. The user can
manually simplify the history with `jj rebase -s <merge commit> -d
<one of the parents>` afterwards. We can roll this change back later
if we find it annoying.
We now have lots of tests of ancestor merges in `test_bug_2600()`, so
we don't need the ones in `test_basics()`. Since it doesn't have the
"nottherootcommit" commit, it would break when we change the default
to preserve ancestor merges.
I think the conclusion from #2600 is that at least auto-rebasing
should not simplify merge commits that merge a commit with its
ancestor. Let's start by adding an option for that in the library.
The shortest change id prefix will become a few digits longer, but I think
that's acceptable. Entries included in the "revsets.short-prefixes" set are
unaffected.
The reachable set is calculated eagerly, but this is still faster as we no
longer need to sort the reachable entries by change id. The lazy version will
save another ~100ms in mid-size repos.
"jj log" without working copy snapshot:
```
% hyperfine --sort command --warmup 3 --runs 20 -L bin jj-0,jj-1,jj-2 \
-s "target/release-with-debug/{bin} -R ~/mirrors/linux debug reindex" \
"target/release-with-debug/{bin} -R ~/mirrors/linux \
--ignore-working-copy log -r.. -l100 --config-toml='revsets.short-prefixes=\"\"'"
Benchmark 1: target/release-with-debug/jj-0 -R ~/mirrors/linux --ignore-working-copy log -r.. -l100 --config-toml='revsets.short-prefixes=""'
Time (mean ± σ): 353.6 ms ± 11.9 ms [User: 266.7 ms, System: 87.0 ms]
Range (min … max): 329.0 ms … 365.6 ms 20 runs
Benchmark 2: target/release-with-debug/jj-1 -R ~/mirrors/linux --ignore-working-copy log -r.. -l100 --config-toml='revsets.short-prefixes=""'
Time (mean ± σ): 271.3 ms ± 9.9 ms [User: 183.8 ms, System: 87.7 ms]
Range (min … max): 250.5 ms … 282.7 ms 20 runs
Relative speed comparison
1.99 ± 0.16 target/release-with-debug/jj-0 -R ~/mirrors/linux --ignore-working-copy log -r.. -l100 --config-toml='revsets.short-prefixes=""'
1.53 ± 0.12 target/release-with-debug/jj-1 -R ~/mirrors/linux --ignore-working-copy log -r.. -l100 --config-toml='revsets.short-prefixes=""'
```
"jj status" with working copy snapshot (watchman enabled):
```
% hyperfine --sort command --warmup 3 --runs 20 -L bin jj-0,jj-1,jj-2 \
-s "target/release-with-debug/{bin} -R ~/mirrors/linux debug reindex" \
"target/release-with-debug/{bin} -R ~/mirrors/linux \
status --config-toml='revsets.short-prefixes=\"\"'"
Benchmark 1: target/release-with-debug/jj-0 -R ~/mirrors/linux status --config-toml='revsets.short-prefixes=""'
Time (mean ± σ): 396.6 ms ± 10.1 ms [User: 300.7 ms, System: 94.0 ms]
Range (min … max): 373.6 ms … 408.0 ms 20 runs
Benchmark 2: target/release-with-debug/jj-1 -R ~/mirrors/linux status --config-toml='revsets.short-prefixes=""'
Time (mean ± σ): 318.6 ms ± 12.6 ms [User: 219.1 ms, System: 94.1 ms]
Range (min … max): 294.2 ms … 333.0 ms 20 runs
Relative speed comparison
1.85 ± 0.14 target/release-with-debug/jj-0 -R ~/mirrors/linux status --config-toml='revsets.short-prefixes=""'
1.48 ± 0.12 target/release-with-debug/jj-1 -R ~/mirrors/linux status --config-toml='revsets.short-prefixes=""'
```
When adding a new workspace, I would expect that it inherits the
patterns from the workspace I ran the command in. We currently don't
do that. That's quite annoying when your repo has very many files
(like at Google).
`graphlog::Edge` is used somewhat inconsistently. I've replaced `Edge::Present`
with two distinct `Edge::Direct` and `Edge::Indirect` which simplifies the
construction of the enum.
The `ContentHash` documentation specifies that implementations for enums should
hash the ordinal number of the variant contained in the enum as a 32-bit
little-endian number and then hash the contents of the variant, if any.
The current implementations for `std::Option`, `MergedTreeId`, and
`RemoteRefState` are non-conformant since they hash the ordinal number as a u8
with platform specific endianness.
Fixes#3051
The legacy parsing rules are turned into compatibility errors. The x:y rule
is temporarily enabled when parsing string patterns. It's weird, but we can't
isolate the parsing function because a string pattern may be defined in an
alias.
This adds a config to render a synthetic node with a "(elided
revisions)" description for elided segments of the graph.
I didn't add any templating support for the elided nodes because I'm
not sure how we would want that to work. In particular, I don't know
what `commit_id` and most other keywords should return for elided
revisions.
Users who edit non-head commits usually expect `jj next/prev` to
continue to edit the next/previous commit, so let's make that the
default behavior. This should not confuse users who don't edit
non-head commits since they will simply not be in this state. My main
concern is that doing `jj next; jj prev` will now usually take you
back to the previous commit, but not if you started on the parent of a
head commit.
The main goal is to avoid having a symlink in our source tree. Currently, there
is no good way to work with the `jj` repo with `jj` on Windows. Currently `jj`
just crashes with symlinks. This is being worked on, see e.g. #2939, but it will
always depend on whether Developer Mode is enabled in Windows or whether
symlinks are materialized as text files with symlinks. Finally, MkDocs has
trouble following symlinks on Windows, so building docs wouldn't work there.
Another advantage is that, previously, we were lucky that MkDocs treats `insta`
header in `cli-reference@.md.snap` as a Markdown header and follows symlinks at
all. Now, we no longer depend on that.
Follows up 9702a425e5 "Allow negative numbers in the template grammar."
Since we've added parsing rules for operator expressions, it makes sense to
parse unary '-' as operator.
If the operation corresponding to a workspace is missing for some reason
(the specific situation in the test in this commit is that an operation
was abandoned and garbage-collected from another workspace), currently,
jj fails with a 255 error code. Teach jj a way to recover from this
situation.
When jj detects such a situation, it prints a message and stops
operation, similar to when a workspace is stale. The message tells the
user what command to run.
When that command is run, jj loads the repo at the @ operation (instead
of the operation of the workspace), creates a new commit on the @
commit with an empty tree, and then proceeds as usual - in particular,
including the auto-snapshotting of the working tree, which creates
another commit that obsoletes the newly created commit.
There are several design points I considered.
1) Whether the recovery should be automatic, or (as in this commit)
manual in that the user should be prompted to run a command. The user
might prefer to recover in another way (e.g. by simply deleting the
workspace) and this situation is (hopefully) rare enough that I think
it's better to prompt the user.
2) Which command the user should be prompted to run (and thus, which
command should be taught to perform the recovery). I chose "workspace
update-stale" because the circumstances are very similar to it: it's
symptom is that the regular jj operation is blocked somewhere at the
beginning, and "workspace update-stale" already does some special work
before the blockage (this commit adds more of such special work). But it
might be better for something more explicitly named, or even a sequence
of commands (e.g. "create a new operation that becomes @ that no
workspace points to", "low-level command that makes a workspace point to
the operation @") but I can see how this can be unnecessarily confusing
for the user.
3) How we recover. I can think of several ways:
a) Always create a commit, and allow the automatic snapshotting to
create another commit that obsoletes this commit.
b) Create a commit but somehow teach the automatic snapshotting to
replace the created commit in-place (so it has no predecessor, as viewed
in "obslog").
c) Do either a) or b), with the added improvement that if there is no
diff between the newly created commit and the former @, to behave as if
no new commit was created (@ remains as the former @).
I chose a) since it was the simplest and most easily reasoned about,
which I think is the best way to go when recovering from a rare
situation.
A subsequent commit will need to handle the return value of
check_stale_working_copy() in 3 different ways, so a boolean will soon
not be sufficient. In preparation for that, inline is_stale() into its
caller, converting it into a "match".
These operator symbols are different from the ones in the revset language. I
have no idea if we need bitwise operators, but we'll probably add comparison
operators. It would look weird if 'x == y & z' were parsed as '(x == y) & z'.
this greatly speeds up the time to run all tests, at the cost of slightly larger recompile times for individual tests.
this unfortunately adds the requirement that all tests are listed in `runner.rs` for the crate.
to avoid forgetting, i've added a new test that ensures the directory is in sync with the file.
## benchmarks
before this change, recompiling all tests took 32-50 seconds and running a single test took 3.5 seconds:
```
; hyperfine 'touch lib/src/lib.rs && cargo t --test test_working_copy'
Time (mean ± σ): 3.543 s ± 0.168 s [User: 2.597 s, System: 1.262 s]
Range (min … max): 3.400 s … 3.847 s 10 runs
```
after this change, recompiling all tests take 4 seconds:
```
; hyperfine 'touch lib/src/lib.rs ; cargo t --test runner --no-run'
Time (mean ± σ): 4.055 s ± 0.123 s [User: 3.591 s, System: 1.593 s]
Range (min … max): 3.804 s … 4.159 s 10 runs
```
and running a single test takes about the same:
```
; hyperfine 'touch lib/src/lib.rs && cargo t --test runner -- test_working_copy'
Time (mean ± σ): 4.129 s ± 0.120 s [User: 3.636 s, System: 1.593 s]
Range (min … max): 3.933 s … 4.346 s 10 runs
```
about 1.4 seconds of that is the time for the runner, of which .4 is the time for the linker. so
there may be room for further improving the times.
It tries to push all deleted branches, but can only succeed for tracked
branches.
Closes#2946, as there's probably no need to have `--deleted --tracked`
that would only be different in that the warnings wouldn't be printed.
Thanks to @yuja for pointing this out.
Our virtual file system at Google (CitC) would like to know the commit
so it can scan backwards and find the closest mainline tree based on
it. Since we always record an operation id (which resolves to a
working-copy commit) when we write the working-copy state, it doesn't
seem like a restriction to require a commit.
When the user doesn't have a configured default command, we show a
hint saying to set `ui.default-command`. I think the user is very
likely to want to set that in the user-wide config, so let's include
the command in the hint.
I'll probably add infix logical operators later, but the surround() function
is still useful because we don't have to repeat the condition:
if(x || y, "<" ++ separate(" ", x, y) ++ ">")
surround("<", ">", separate(" ", x, y))
It can't be used if we want to add placeholder text, though:
if(x || y, "<" ++ separate(" ", x, y) ++ ">", "(none)")
Closes#2924
This reimplements the change 9faa4670d5 "cli: on init, import git refs prior
to importing HEAD." Initialization is special because the HEAD ref isn't
available to jj yet, and there is an empty working-copy commit.
The initialization function could be refactored to go through the common code
path, but I think doing that would make future improvement harder. We might
want to initialize tracking branches based on .git/config for example.
Fixes#2942
This initializes a git backed repo.
* It does the same thing as `jj init --git` except that it
has a --colocated flag to explicitly specify that we want
the .git repo to be side-by-side the .jj repo in the working
directory.
* `jj init --git` will keep the current behaviour and will not
be able to create colocated git backed repos.
* Update test snapshots.
A few minor updates after 0f27152 AKA #2945
Specifically, I tried to change the help text so that it looks better as
Markdown. I also reworded the deprecation warning and added a hint.
this has two main advantages:
- it makes it clear that the shells are mutually exclusive
- it allows us to extend the command with shell-specific options in the future if necessary
as a happy accident, it also adds support for `elvish` and `powershell`.
for backwards compatibility, this also keeps the existing options as hidden flags.
i am not super happy with how the new help looks; the instructions for setting up the shell are
squished together and IMO a little harder to read. i'm open to suggestions.
it's somewhat confusing to me that the `--bash` flag exists at all, since it does nothing - maybe it makes sense to give a hard error? or just remove the flag?
but in any case, it seems good to document the existing behavior.
previously, `jj diff` would show the full contents of binary files such as images.
after this change, it instead shows "(binary)". it still shows the filename and metadata so that
users can open the file in the viewer of their choce.
future work could involve showing binary files as Sixel or similar; finding a way to compare large
non-binary files without filling up the screen; or extending the data backends to avoid having to
read the whole file contents into memory.
Previously, `jj git push; jj git push` would tell the user that "No
branches point to the specified revisions.". I found this confusing,
even though strictly speaking it is correct (as the default revset only
considers revisions that haven't been pushed to the remote).
Closes#2241
The error output gets more verbose because all gix error sources are printed.
Maybe we'll need a better formatting, but changing to multi-line output doesn't
look nice either.
These error types are special because the message is embedded in ASCII art. I
think it would be a source of bugs if some error types had ": {source}" but
others don't. So I'm going to remove all ": {source}"s, and let the callers
concatenate them when needed.
Summary: As discussed in Discord, on GitHub, and elsewhere, this change
deprecates the use of `jj merge` and suggests users use `jj new` exclusively
instead. `merge` isn't completely unfit as a name; but we think it obscures
the generality of `new` and we want people to use it instead.
To further drive the bit home, by default, `jj merge` is now hidden. This will
hopefully stop new users from running into it.
Signed-off-by: Austin Seipp <aseipp@pobox.com>
Change-Id: I94938aca9d3e2aa12d1394a5fbc58acce3185b56
Summary: As discussed in Discord, on GitHub, and elsewhere, this change
deprecates the use of `jj checkout` and suggests users use `jj new` exclusively
instead. The verb `checkout` is a relic of a bygone era the — days of RCS
file locking, before 3-way merge — and is not a good, fitting name for the
functionality it provides.
To further drive the bit home, by default, `jj checkout` (and `jj co`) is now
hidden. This will hopefully stop new users from running into it.
Signed-off-by: Austin Seipp <aseipp@pobox.com>
Change-Id: I7a1adfc9168fce1f25cf5d4c4c313304769e41a1
These didn't really need to use `jj checkout`, and it will be deprecated in a
future change anyway. Move them out of there and into `new`.
Ideally this would go into `test_conflicts.rs`, but that exists in `jj-lib`, so
it doesn't have `TestEnvironment` available to it.
Signed-off-by: Austin Seipp <aseipp@pobox.com>
Change-Id: If0173b27ab4d1f6036a4ec632ec77b6824f310c3
* Move git_init() to cli/src/commands/git.rs and call it from there.
* Move print_trackable_remote_branches into cli_util since it's not git specific,
but would apply to any backend that supports remote branches.
* A no-op change. A follow up PR will make use of this.
* Create a git_init() function in cli/src/commands/init.rs where all git related work is done.
This function will be moved to cli/src/commands/git.rs in a subsequent PR.
I'm going to rewrite the error output to print source chain one by one.
The "{message}: {err}" pattern is wrapped in an error struct. InternalError
variant could be a { message, source } pair, but not all errors need an
additional message. This will also apply to UserError.
This prevents rust doctests from trying to compile the code blocks.
I don't think we use doctests much or at all, but I'd like
`cargo insta test --test-runner nextest --accept --workspace`
to pass.
Unfortunately, this affects the output of `jj help` as well, but I
can't think of a workaround for that.
I am using a very hacky approach, using `insta` to generate the markdown help.
This is based on a lucky coincidence that `insta` chose to use a header
format for snapshot files that MkDocs interprets as a Markdown header (and
ignores).
I considered several other approaches, but I couldn't resist the facts that:
- This doesn't require new developers to install any extra tools or run any
extra commands.
- There is no need for a new CI check.
- There is no need to compile `jj` in the "Make HTML docs" GitHub action,
which is currently very fast and cheap.
Downside: I'm not sure how well MkDocs will work on Windows, unless the
developer explicitly enables symbolic links (which IIUC is not trivial).
### Possible alternatives
My next favorite approaches (which we could switch to later) would be:
#### `xtask`
Set up a CI check and a [Cargo `xtask`] so that `cargo xtask cli-help-to-md`
essentially runs `cargo run -- util markdown-help > docs/cli-reference.md` from
the project root.
Every developer would have to know to run `cargo xtask cli-help-to-md` if
they change the help text.
Eventually, we could have `cargo xtask preflight` that runs `cargo +nightly
fmt; cargo xtask cli-help-to-md; cargo nextest run`, or `cargo insta`.
#### Only generate markdown for CLI help when building the website, don't track it in Git.
I think that having the file in the repo will be nice to preview changes to
docs, and it'll allow people to consult the file on GitHub or in their repo.
The (currently) very fast job of building the website would now require
installing Rust and building `jj`.
#### Same as the `xtask`, but use a shell script instead of an `xtask`
An `xtask` might seem like overkill, since it's Rust instead of a shell script.
However, I don't want this to be a shell script so that new contributors on
Windows can still easily run it ( since this will be necessary for the CI to
pass) without us having to support a batch file.
#### Cargo Alias
My first attempt was to set up a [cargo alias] to run this, but that doesn't
support redirection (so I had to change the `jj util` command to output to a
file) and, worse, is incapable of executing the command *in the project root*
regardless of where in the project the current directory is. Again, this seems
to be too inconvenient for a command that every new PR author would have to run
to pass CI.
Overall, this just seems a bit ugly. I did file
https://github.com/rust-lang/cargo/issues/13348, I'm not really sure that was
worthwhile, though.
**Aside:** For reference, the alias was:
```toml
# .cargo/config.toml
alias.gen-cli-reference = "run -p jj-cli -- util markdown-help docs/cli-reference.md"
```
### Non-alternatives
#### Clap's new feature
`clap` recently obtained a similarly-sounding feature in
https://github.com/clap-rs/clap/pull/5206. However, it only prints short help
for subcommands and can't be triggered by an option AFAICT, so it won't help us
too much.
[Cargo `xtask`]: https://github.com/matklad/cargo-xtask
[cargo alias]: https://doc.rust-lang.org/cargo/reference/config.html#alias
This uses the [`clap-markdown`] library. It's not very flexible, but seems to
work. Its implementation is not difficult. If needed, we could later
reimplement the part that iterates over all subcommands and have a different
output (e.g., the boring version of each help text inside its own code block,
or a different file per subcommand, or some fancy template in handlebars or
another rust-supported templating language). I don't want to do it right now,
though.
The library does turn out to have some annoying bugs, e.g.
https://github.com/ConnorGray/clap-markdown/pull/18, but I think that's not a
deal-braker. The fix seems to be 3 lines; if the fix doesn't get merged soon, we
could vendor the library maybe?
[`clap-markdown`]: https://docs.rs/clap-markdown/latest/clap_markdown/
GitBackend::gc() will need to check if a commit is reachable from any
historical operations. This could be calculated from the view and commit
objects, but the Index will do a better job.
The "::HEAD" set is usually smaller than "::git_refs". If these sets were
imported in that order, "HEAD..git_refs" commits would be indexed on top of
the "::HEAD" commits. It's not a problem, but undesirable.
I'm going to make WorkspaceCommandHelper::maybe_snapshot() snapshot the working
copy before importing refs. git::import_some_refs() can rebase the working copy
branch and therefore @ can be moved. git::import_head() doesn't, and it should
be invoked before snapshotting.
git::import_head() is inserted to some of the git:import_refs() callers where
HEAD seems to matter. I feel it's a bit odd that the HEAD ref is imported to
non-colocated repo, but "jj init --git-repo" relies on that, and I think the
existence of HEAD@git is harmless. It's merely a ref to the revision checked
out somewhere else.
This is a convenient command, for scripting things like `cd $(jj root)
&& do something`, and it seems better to allow people to find it
before they learn about workspaces.
This goes with `c`, `s`, `f`. I feel like it's a relatively
common command now that `auto-local-branch` defaults to
`false`.
I didn't add `u` for `untrack` because it seems a little
ambiguous (un-what?); there may be other `branch` commands that undo
something in the future.
The count() function in this trait is used by "jj branch" to determine
(and then report) how many commits a certain branch is ahead/behind
another branch. This is currently implemented by walking all commits
in the revset, counting how many were encountered. But this could be
improved: if the number is large, it is probably sufficient to report
"at least N" (instead of walking all the way), and this does not scale
well to jj backends that may not have all commits present locally (which
may prefer to return an estimate, rather than access the network).
Therefore, add a function that is explicitly documented to be O(1)
and that can return a range of values if the backend so chooses.
Also remove count(), as it is not immediately obvious that it is an
expensive call, and callers that are willing to pay the cost can obtain
the exact same functionality through iter().count() anyway. (In this
commit, all users of count() are migrated to iter().count() to preserve
all existing functionality; they will be migrated to count_estimate() in
a subsequent commit.)
"branch" needed to be updated due to this change. Although jj
is currently only available in English, I have attempted to keep
user-visible text from being assembled piece by piece, so that if we
later decide to translate jj into other languages, things will be easier
for translators.
A subsequent commit will teach more variants of ahead/behind messages,
so refactor the existing code to avoid code duplication and a future
combinatorial explosion.
Gerrit also needs to be able to push low-level refs into upstream remotes, just
like `jj git push` does. Doing so requires providing callbacks e.g. for various
password entry mechanisms, which was private to the `git` command module.
Pull these out to a new module `git_utils` so we can reuse it across the two
call sites.
This also moves a few other strictly Git-related functions into `git_utils`
as well, just for the sake of consistency.
Signed-off-by: Austin Seipp <aseipp@pobox.com>
* Move canonicalization of the external git repo path into the Workspace::init_git_external().
This keeps necessary code together.
* Add a new variant of WorkspaceInitError for reporting path not found errors. The user error
string is written to pass existing tests.
It seems obvious in hindsight to have a virtual root operation just
like we have a virtual root commit. It removes the same kind of
problems by making sure there's always a common ancestor (or multiple)
between any two commits.
I think the reason I didn't add a root operation from the beginning
was that there used to be a mandatory working-copy commit in the view
(this was before support for multiple workspaces).
Perhaps we should remove the "initialize repo" operation now. The only
difference between their view objects is that the "initialize repo"
operation adds the root commit as a head. We could add that to the
root operation, but then the root operation's value depends on the
commit backend.
We've had the public_heads for as long as we've had the View object,
IIRC (I didn't check), but we still don't use it for anything. I don't
have any concrete plans for using it either. Maybe our config for
immutable commits is good enough, or maybe we'll want something more
generic (like Mercurial's phases). For now, I think we should simplify
by removing it the storage for public heads.
This function respects --ignore-working-copy and --at-op arguments, but the
name snapshot() sounds like it could forcibly trigger the snapshotting. Let's
clarify the actual behavior.
import_git_refs_and_head() and snapshot_working_copy() are unchecked functions,
and there are no external callers.
When debugging behavior of badly-GCed repos, I find it's annoying that "op log"
fails because the index can't be loaded. Since "op log" doesn't need a repo, I
think it's better to display the exact op-heads state without merging.
I'm going to make "op log" not load a repo nor resolve concurrent operations,
so there will be the case where no unique "current" operation exists. Since the
"current" operation represents the repo to be loaded, I think it's better to
not consider multi-head operations as the "current" ones. That's why the
templater field isn't Vec<_> but Option<_>.
If indexing failed due to missing commit objects, the repo won't be loadable
without --ignore-working-copy (at least in colocated environment.) In that
case, we can use "op abandon" to recover, but we had to work around the failed
index loading by --ignore-working-copy. Since "op abandon" isn't a repo-level
command, it's better to bypass working-copy snapshot and import of git refs at
all.
--at-op is rejected because it's useless and we'll need extra care for "@"
expression resolution and working-copy updates.
The formatting is closer to hg than git just because it's easier to
conditionalize the whole line per keyword. Our template language doesn't
have infix logical operators.
Unlike the other default templates, all remote branches are displayed because
it's "detailed" output.
Closes#2509
If the existing index was corrupt, it would have to be rebuilt while loading a
repo (at least in colocated environment.) Then, the index will be rebuilt again.
Since new operations and views may be added concurrently by another process,
there's a risk of data corruption. The keep_newer parameter is a mitigation
for this problem. It's set to preserve files modified within the last 2 weeks,
which is the default of "git gc". Still, a concurrent process may replace an
existing view which is about to be deleted by the gc process, and the view
file would be lost.
#12
It doesn't make sense to do gc from a non-head operation because that means
either the head operation would be corrupted or the --at-op argument is
ignored.
We would like to use a non-static version string at Google. We have a
workaround using Lazy, but it seems unfortunate to have to do
that. Using dynamic strings requires Clap's `string` feature, which
increases the binary size by 140 kiB (0.6%).
I'm going to add try_from_hex(), which requires Self: Sized. Such trait bound
could be added, but I don't think we'll need abstracted ObjectId constructors
at all.
I'm going to add a prefix resolution method to OpStore, but OpStore is
unrelated to the index. I think ObjectId, HexPrefix, and PrefixResolution can
be extracted to this module.
In order to implement GC (#12), we'll need to somehow prune old operations.
Perhaps the easiest implementation is to just remove unwanted operation files
and put tombstone file instead (like git shallow.) However, the removed
operations might be referenced by another jj process running in parallel. Since
the parallel operation thinks all the historical head commits are reachable, the
removed operations would have to be resurrected (or fix up index data, etc.)
when the op heads get merged.
The idea behind this patch is to split the "op log" GC into two steps:
1. recreate operations to be retained and make the old history unreachable,
2. delete unreachable operations if the head was created e.g. 3 days ago.
The latter will be run by "jj util gc". I don't think GC can be implemented
100% safe against lock-less append-only storage, and we'll probably need some
timestamp-based mechanism to not remove objects that might be referenced by
uncommitted operation.
FWIW, another nice thing about this implementation is that the index is
automatically invalidated as the op id changes. The bad thing is that the
"undo" description would contain an old op id. It seems the performance is
pretty okay.
This removes CommandError dependency from these resolution functions. We might
want to refactor the error types again if we introduce a real "opset" evaluator.
The error message for unresolved op heads now includes "@" instead of the whole
expression.
The resolver callback usually returns wider error type, which I don't think
is a variant of OpHeadResolutionError.
To help type inference, resolver's error type is E, not E1 where E: From<E1>.
I'll probably add change id lookup methods to CompositeIndex. The Index trait
won't gain resolve_change_id_prefix(), but I also renamed its resolve_prefix()
for consistency.
