It seems to me that we have never created a Git index in order to
create a commit, not even in the earliest versions of the code (before
it was moved to Git).
Now that I'm using GitHub PRs instead of pushing directly to the main
branch, it's quite annoying to have to abandon the old commits after
GitHub rebases them. This patch makes it so we compare the remote's
previous heads to the new heads and abandons any commits that were
removed on the remote. As usual, that means that descendants get
rebased onto the closest remaining commit.
This is half of #241. The other half is to detect rewritten branches
and rebase on top.
Let's say we have a simple history like this:
```
B C D
\|/
A
```
Branch `main` initially points to commit B. Two concurrent operations
then move the branch to commits C and D. When the two concurrent
operations get merged, the branch will be recorded as pointing to
"C+D-B". If a subsequent operation now abandons commit B, we would
update the "removed" side of the branch conflict. That seems a little
dishonest. I think the reason I did it that way was in order to not
keep B visible back when having it present in the "removed" side would
keep it visible (before 33bf6ce1d5).
I noticed this issue while working on #241 because
`test_import_refs_reimport()` started failing. That test case is
pretty much exactly the case above.
When committing the working copy, we try to not visit ignored
directories (as e.g. `target/` often is), but we need to visit it if
there are already tracked files in it. I initially missed that in
c1060610bd and then fixed it in a028f33e3b. The fix works by
checking if the next path after the ignored path is inside the ignore
path (viewed as a directory). However, I forgot to handle the case
where there are no paths at all after the ignored path. So, for
example, if the `target/` directory should be ignored and it there
were no tracked paths after `target/` in alphabetical order, we would
still visit the directory. That's why the bug reproduced in the
`git-branchless` repo but not in the `jj` repo (because there are
files under `testing/` and `tests/` here).
Closes#247.
This patch makes room for sparse patterns in the `TreeState` proto
message. We also start setting that value to a list of just the
pattern `.` when we create new working copies. Old working copies
without the sparse patterns are also interpreted as having that single
pattern. Note that this absence of sparse patterns is different from a
present list of no patterns. The latter is a valid state and means
that no paths are included in the sparse checkout.
This adds a matcher that takes two input matchers and creates a new
matcher from them. The composite matcher matches paths matched by the
first matcher but not matched by the second matcher. I plan to use
this for sparse checkouts. They'll also be useful if we add support
for negative patterns to filter e.g. `jj files` by.
Knowing that a matchers matches everything recursively from a certain
directory is useful for various optimizations. For example, it lets
you avoid visiting a directory if you're using a matcher with a
negative condition (so you return what does *not* match).
The `DescendantRebaser` keeps a map of branches from the source
commit, so it gets efficient lookup of branches to update when a
commit has been rebased. This map was not kept up to date as we
rebased. That could lead to branches getting left on hidden
intermediate commits. Specifically, if a commit with a branch was
rewritten by some command, and an ancestor of it was also rewritten,
then we'd only update the branch only the first step and not update it
again when rebasing onto the rewritten ancestor.
When a directory is missing in one merge input (base or one side), we
would consider that a merge conflict. This patch changes that so we
instead merge trees by treating the missing tree as empty.
This introduces a `connected(x)` function, which is simply the same as
`x:x`. It's occasionally useful if `x` is a long expression. It's also
useful as a building block for `root(x)` (coming soon).
This release is mostly about the fix for #177, which looks pretty bad
even though I think it is actually harmless. It also has `jj log -p`
contributed by @yuja!
We do it for all the other kinds of objects already. It's useful to
have the path for backends that store objects by path (we don't have
any such backends yet). I think the reason I didn't do it from the
beginning was because we had separate `RepoPath` types for files and
directories back then.
We depend on comparing the workspace root with the Git repo's path to
know if we're sharing the working copy with it. For that to work
reliably, we need the paths to be canonicalized, so that's what this
patch tries to do.
There was a TODO about adding a test case for a delete/modify conflict
in a branch target that got resolved by abandoning a commit. The
resolution is to delete the branch. That case couldn't happend with
our old evolution-based mechanism for tracking rewrites (because we
couldn't un-prune a commit then).
This involved copying `UnresolvedHeadRepo::resolve()` into the CLI
crate (and modifying it a bit to print number of rebased commit),
which is unfortunate.
The function is now pretty simple, and there's only one caller, so
let's inline it. It probably makes sense to move the code out of
`repo.rs` at some point.
It's the transaction's job to create a new operation, and that's where
the knowledge of parent operations is. By moving the logic for merging
in another operation there, we can make it contiuously update its set
of parent operations. That removes the risk of forgetting to add the
merged-in operation as a parent. It also makes it easier to reuse the
function from the CLI so we can inform the user about the process
(which is what I was investigating when I noticed that this cleanup
was possible).
If we have recorded in `MutableRepo` that commits have been abandoned
or rewritten, we should always rebase descendants before committing
the transaction (otherwise there's no reason to record the
rewrites). That's not much of a risk in the CLI because we already
have that logic in a central place there (`finish_transaction()`), but
other users of the library crate could easily miss it. Perhaps we
should automatically do any necessary rebasing we commit the
transaction in the library crate instead, but for now let's just have
a check for that to catch such bugs.
Certain commands should never rewrite commits, or they take care of
rebasing descendants themselves. We have an optimization in
`commands.rs` for those commands, so they skip the usual automatic
rebasing before committing the transaction. That's risky to have to
remember and `MutableRepo` already knows if any commits have been
rewritten (that wasn't the case before, in the Evolution-based
code). So let's just have `MutableRepo` do the check instead.
It's useful for the UI layer to know that there's been concurrent
operations, so it can inform the user that that happened. It'll be
even more useful when we soon start making the resolution involve
rebasing commits, since that's even more important for the UI layer to
present to the user. This patch gets us a bit closer to that by moving
the resolution to the repo level.
We had a few lines of similar code where we added a new of the
operation log and then removed the old heads. By moving that code into
a new type, we prepare for further refactorings.
I want to make it so when we apply a repo-level change that removes a
head, we rebase descendants of that head onto the closes visible
ancestor, or onto its successor if the head has been rewritten (see
#111 for details). The view itself doesn't have enough information for
that; we need repo-level information (to figure out relationships
between commits).
The view doesn't have enough information to do the.
It's unusual for the current commit to have descendants, but it can
happen. In particular, it can easily happen when you run `jj new`. You
probably don't want to abandon it in those cases.
I forgot to bump the version to 0.3.2 before tagging and releasing it,
so the released 0.3.2 has version number 0.3.1 in the source code and
(therefore) reported from `jj --version`. I'm therefore bumping it
from 0.3.1 to 0.3.3 now, so there can be a matching 0.3.3 release.
I was able to build a working musl binary with this change, by running
this command:
```
cargo build --release --target x86_64-unknown-linux-musl
```
Thanks to @arxanas for the tip.
There's been *a lot* of changes since 0.2.0 almost a year ago. With
the attention the project has gotten recently, I feel like I should
cut a new release and start keeping a changelog. So let's start by
bumping the version to 0.3.0.
The library crate shouldn't look up the user's `$HOME` directory
(maybe the library is used by a server process), so let's have the
caller pass it into the library crate instead.
I'm not sure it'll be useful, but it seems nice to be able to set the
same values via config or environment variables. Perhap we should
simply use `config::Environment` to make everything configurable via
environment variables, but I'll leave that for later.
It's useful to have `signature()` live on `UserSettings` because that
will let us cache information (such as the timestamp) in the
instance. It will also make it easier to have the timestamp settable
via regular config files. I don't know that that will be useful, but
it seems like a clean way of implementing it if we can have
environment variables simply as an overlay of configs.
We don't really need a BTreeMap for keeping the unchanged ranges. The
only place it helps a bit is when refining a diff because we may then
insert some more unchanged ranges in the list. I think there has to be
very many unchanged ranges for that to matter, however. This patch
therefore replace the BTreeMap by a sorted Vec. `cargo bench` says
that a few tests got ~20% faster.
I'm looking into this code now because I'm thinking of copying some of
it for the "partial conflict resolution" tool I'm working on for
Mercurial.
I wanted to replace the BTreeMap by a Vec and noticed that we actually
sometimes end up having a `0..n` range followed by a `0..0` after
refinement. We currently compare those two as equal because I had not
thought that we could end up attempting to add two ranges with the
same start point. When trying to insert the second range (`0..0`), the
BTreeMap will keep the existing key (`0..n`) and replace the
value. That's probably works, but it's clearly not what I
intended. Let's fix by sorting by the end point if the start point is
equal. This actually improves some benchmarks by a few percent (maybe
because the subsequent compaction can then remove the `0..0` range).
When the backing Git repo is inside the workspace (typically directly
in `.git/`), let's point to it by a relative path so the whole
workspace can be moved without breaking the link.
Closes#72.
This patch introduces a `JJ_TIMESTAMP` environment variable that lets
us specify the timestamp to use in tests. It also updates the tests to
use it, which means we get to simplify the tests a lot now that that
the hashes are predictable.
Originally, I had thought that these warnings would only potentially show up in nightly because there was a feature which exposed these functions, and we would be able to enable that feature and conditionally not define the conflicting methods. But it looks like these warnings also show up in stable. I've just suppressed each of them individually. Other options would be to rename them and just make them wrapper methods, or to disable `unstable_name_collisions` warnings at a higher scope (possibly including at the crate level).
1b6efdc3f8 moved `.jj/git/` into `.jj/store/` for consistency with
the layout of native stores. It provided automatic format upgrades for
repos with the old format. It's been about four months now, so let's
remove the migration code.
We no longer need the commit ID, so we shouldn't make the callers pass
it. This lets us simplify several tests, because they no longer to
create commits just to check out a tree in the working copy.
We used to use the value to detect races, but we use the tree ID and
the operation ID these days, so we don't need the commit ID.
By changing this, we can avoid creating some commit IDs in tests,
which is why I tackled this issue now.
There are only two callers of `LockedWorkingCopy::check_out()`. One is
in `commands.rs`. That caller already checks after taking the lock
that the old commit ID is as expected. The other caller is
`WorkingCopy::check_out()`. We can simply move the check to that level
since it's the only caller that cares now.
We resolve checkouts in favor of the first-committed operation (which
is more likely to have managed to update the working copy). The test
case has been flaky on GitHub lately. I've run it 1000 times on my
machine without failure. I don't know if GitHub's machines are just
faster in some way (SSD, maybe) that makes them finish the two
operations in the test in the same millisecond. Let's add a
1-millisecond sleep to see if that helps. If it doesn't, then maybe
the issue is that the clock has lower precision (or their clocks can
go backwards?).
`LockedWorkingCopy::discard()` shouldn't result in changes to the
on-disk state, but `LockedWorkingCopy::check_out()` may have already
written a state file, which is surprising. The changes also remain in
memory, which is also surprising. Let's fix both of those issues.
One of the .gitignore tests writes a tree from the working copy
twice. However, it discards the `LockedWorkingCopy` instance after the
first write, so the second write shouldn't really see the changes from
the first write. It does see them because we don't clear them in
memory (and we also surprisingly write them to disk). I'm about to fix
that, so the test needs to be fixed first.
It's useful to be able to match path prefixes for many commands,
e.g. to allow `jj restore src` to restore all files in under `src/`
(or a file called `src`). I also plan to use it for sparse checkouts.
We'll need to be able to match path prefixes
This is just to avoid the lifetime parameter. It was a premature
optimization to return a reference (we don't even use the matchers
yet, so it cloning these sets clearly doesn't show up in profiling).
Most tests need a repo but don't need a working copy. Let's have a
function for setting up a test repo. But first, let's free up the name
`init_repo()` by renaming it to `init_workspace()` (which is also more
accurate).
In workspaces added after the initial one, the idea is to have
`.jj/repo` be a file whose contents is a path to the location of the
repo directory in some other workspace.
As part of creating a new repository, we create an open commit on top
of the root and set that as the current checkout. Now that we have
support for multiple checkouts in the model, we also have support for
zero checkouts, which means we don't need to create that commit on top
of the root when creating the repo. We can therefore move out of
`ReadonlyRepo`'s initialization code and let `Workspace` instead take
care of it. A user-visible effect of this change is that we now create
one operation for initilizing the repo and another one for checking
out the root commit. That seems fine, and will be consistent with the
additional operation we will create when adding further workspaces.
Because we record each workspace's checkout in the repo view, we can
-- unlike other VCSs -- let the user refer to any workspace's checkout
in revsets. This patch adds syntax for that, so you can show the
contents of the checkout in workspace "foo" with `jj show foo@`. That
won't automatically commit that workspace's working copy, however.
When checking out a new commit, we look at the old checkout to see if
it's empty so we should abandon it. We current use the default
workspace's checkout. We need to respect the workspace ID we're given
in `MutableRepo::check_out()`, and we need to be able to deal with
that workspace not existing yet (i.e. this being the first checkout in
that workspace).
This patch teaches the `View` object to keep track of the checkout in
each workspace. It serializes that information into the `OpStore`. For
compatibility with existing repos, the existing field for a single
workspace's checkout is interpreted as being for the workspace called
"default".
This is just an early step towards support for multiple
workspaces. Remaining things to do:
* Record the workspace ID somewhere in `.jj/` (maybe in
`.jj/working_copy/`)
* Update existing code to use the workspace ID instead of assuming
it's always "default" as we do after this patch
* Add a way of indicating in `.jj/` that the repo lives elsewhere and
make it possible to load a repo from such workspaces
* Add a command for creating additional workspaces
* Show each workspace's checkout in log output
The `.jj/` directory contains information about two distinct parts:
the repo and the working copy. Most subdirectories are related to the
repo; only `.jj/working_copy/` is about the working copy. Let's move
the repo-related bits into a new `.jj/repo/` subdirectory. That makes
it clearer that they're related to the repo. It will probably also be
easier to manage when we have support for multiple workspaces backed
by a single repo.
The `DescendantRebaser` was designed to help with rebasing in two
different use cases: 1) after regular rewriting of commits where the
change ID is preserved, and 2) after importing moved branches from
other repo (e.g. backing Git repo or remote). Many of the tests are
for the second use case, such as where a branch was moved
forward. However, I just noticed that there's a pretty common scenario
from the first use case that is not supported.
Let's say you have this history:
```
D
|
C C'
|/
B B'
|/
A
```
Here we want C' to be rebased onto B' and then D to be rebased onto
C''. However, because of the support for moving branches forward, we
would not rebase commits that were already rewritten, such as C' here
(see affected tests for details), which resulted in D getting rebased
onto C', and both B and B' remaining visible.
I think I was thinking when I designed it that it would be nice if you
could just tell `DescendantRebaser` that any descendants of a commit
should be moved forward. That may be useful, but I don't think we'll
want that for the general case of a branch moving forward. Perhaps
we'll want to make it configurable which branches it should happen
for. Either way, the way it was coded by not rebasing already
rewritten commits did not work for the case above. We may be able to
handle both cases better by considering each rewrite separately
instead of all destinations at once. For now, however, I've decided to
keep it simple, so I'm fixing the case above by sacrificing some of
the potentially useful functionality for moving branches forward.
Another fix necessary for the scenario shown above was to make sure we
always rebase C' before D. Before this patch, that depended on the
order in the index. This patch fixes that by modifying the topological
order to take rewrites into account, making D depend not only on C but
also on C'. (I suppose you could instead say that C depends on both B
and C'; I don't know if that'd make a difference.)
Despite what the documentation said, we don't clear the record of
rewritten and abandoned commits at the end. This change fixes that,
and adds a test showing that it's possible to call
`MutableRepo::rebase_descendants()` multiple times.
When there are concurrent operations that want to update the working
copy, it's useful to know which operation was the last to successfully
update the working copy. That can help use decide how to resolve a
mismatch between the repo view's record and the working copy's
record. If we detect such a difference, we can look at the working
copy's operation ID to see if it was updated by an operation before or
after we loaded the repo.
If the working copy's record says that it was updated at operation A
and we have loaded the repo at operation B (after A), we know that the
working copy is stale, so we can automatically update it (or tell the
user to run some command to update it if we think that's more
user-friendly).
Conversely, if we have loaded the repo at operation A and the working
copy's record says that it was updated at operation B, we know that
there was some concurrent operation that updated it. We can then
decide to print a warning telling the user that we skipped updating
because of the conflict. We already have logic for not updating the
working copy if the repo is loaded at an earlier operation, but maybe
we can drop that if we record the operation in the working copy (as
this patch does).
Having the checkout functionality in `LockedWorkingCopy` makes it a
little more flexible (one could imagine using it for udating working
copy files and then discarding the state changes, for example). It
also lets us reuse a few lines of code for locking. I left
`WorkingCopy::check_out()` for convenience because that's what all
current users want.
`WorkingCopy::check_out()` currently fails if the commit recorded on
disk has changed since it was last read. It fails with a "concurrent
checkout" error. That usually works well in practice, but one can
imagine cases where it's not correct. For an example where the current
behavior is wrong, consider this sequence of events:
1. Process A loads the repo and working copy.
2. Process B loads the repo at operation A. It has not loaded the
working copy yet.
3. Process A writes an operation and updates the working copy.
4. Process B loads the working copy and sees that it is checked out
to the commit process B set it to. We don't currently have any
checks that the working copy commit matches the view's checkout
(though I plan to add that).
5. Process B finishes its operation (which is now divergent with the
operation written by process A). It updates the working copy to
the checkout set in the repo view by process B. There's no data
loss here, but the behavior is surprising because we would usually
tell the user that we detected a concurrent update to the working
copy.
We should instead check that the working copy's commit on disk matches
what the previous repo view said, i.e. the view at the start of the
operation we just committed. This patch does that by having the caller
pass in the expected old commit ID.
We already have two usecases that can be modeled as updating the
`TreeState` without touching the working copy:
1. `jj untrack` can be implemented as removing paths from the tree
object and then doing a reset of the working copy state.
2. Importing Git HEAD when sharing the working copy with a Git repo.
This patch adds that functionality to `TreeState`.
I was surprised that we save the `TreeState` before
`LockedWorkingCopy::finish()`. That means that even if the caller
instead decides to discard the changes, some changes will already have
been written.
This patch changes the interface for making changes to the working
copy by replacing `write_tree()` and `untrack()` by a single
`start_mutation()` method. The two functions now live on the returned
`LockedWorkingCopy` object instead. That is more flexible because the
caller can make multiple changes while the working copy is locked. It
also helps us reduce the risk of buggy callers that read the commit ID
before taking the lock, because we can now make it accessible only on
`LockedWorkingCopy`.
