Commit graph

137 commits

Author SHA1 Message Date
Yuya Nishihara
4834d12c37 simple_op_store: serialize RefTarget in new format (breaks downgrades)
This is breaking change. Old jj binary will panic if it sees a view saved by
new jj. Alternatively, we can store both new and legacy data for backward
compatibility.
2023-07-27 15:32:48 +09:00
Yuya Nishihara
84f0c96c8f view: replace .tags().get(name) with .get_tag(name) 2023-07-19 08:27:42 +09:00
Yuya Nishihara
bf46eb5ad2 view: replace .branches().get(name) with .get_branch(name) 2023-07-19 08:27:42 +09:00
Yuya Nishihara
ecb0850f1a view: return RefTarget by reference, clone() by caller 2023-07-19 08:27:42 +09:00
Yuya Nishihara
443391bf8f view: store Option<RefTarget> in maps, add extension trait to flatten Option
Alternatively, we can wrap BTreeMap<String, Option<RefTarget>> to flatten
Option<&Option<..>> internally, but doing that would be tedious. It would
also be unclear if map.remove(name) should construct an absent RefTarget if
the ref doesn't exist.
2023-07-18 18:12:09 +09:00
Yuya Nishihara
7461370f6e view: add wrapper that will exclude absent RefTarget entries from ContentHash
The next commit will change these maps to store Option<RefTarget> entries, but
None entries will still be omitted from the serialized data. Since ContentHash
should describe the serialized data, relying on the generic ContentHash would
cause future hash conflict where absent RefTarget entries will be preserved.

For example, ([remove], [None, add]) will be serialized as ([remove], [add]),
and deserialized to ([remove], [add, None]). If we add support for lossless
serialization, hash(([remove], [None, add])) should differ from the lossy one.
2023-07-18 18:12:09 +09:00
Yuya Nishihara
0461a8575a refs: add stub constructors for absent RefTarget, replace None with it
Now we're mostly ready to reimplement RefTarget on top of
Conflict<Option<CommitId>>.
2023-07-17 08:24:24 +09:00
Yuya Nishihara
a1e92563ae refs: add constructor of Some(RefTarget::Conflict { .. })
It's named after Conflict::from_legacy_form(). If RefTarget is migrated to
new Conflict type, from_legacy_form([], [add]) will create a normal target,
and from_legacy_form([], []) will be equivalent to the current None target.
That's why this function isn't named as RefTarget::conflict().
2023-07-17 01:13:21 +09:00
Yuya Nishihara
e55c533add refs: add constructor of Some(RefTarget::Normal(id))
If RefTarget is migrated to new Conflict type, this function will create
RefTarget(Conflict::resolved(Some(id))).

We still need some .unwrap() to insert Option<RefTarget> into map, but maps
will be changed to store new RefTarget type, and their mutation API will
guarantee that Conflict::resolved(None) is eliminated.
2023-07-17 01:13:21 +09:00
Yuya Nishihara
5ed070a78a view: use get_git_ref(name) where extra .clone() wouldn't matter
Perhaps, all view.get_<kind>(name) functions can return reference, but I'm
not willing to change the interface at this point. I'll revisit this after
migrating Option<RefTarget> to new Conflict-based type.
2023-07-15 05:57:49 +09:00
Yuya Nishihara
d80a719042 view: unify set/remove_local_branch() functions to take Option<RefTarget> 2023-07-15 05:57:49 +09:00
Yuya Nishihara
79955d9cb3 git: do not import/export local branch named "HEAD"
Git CLI rejects it (though the data model would probably work fine with
"HEAD" branch.) This ensures that HEAD@git in JJ world is not an exported
branch named "HEAD".
2023-07-11 16:29:27 +09:00
Martin von Zweigbergk
aac5b7aa25 cargo: rename crates from jujutsu/jujutsu-lib to jj-cli/jj-lib
Almost everyone calls the project "jj", and there seeems to be
consensus that we should rename the crates. I originally wanted the
crates to be called `jj` and `jj-lib`, but `jj` was already
taken. `jj-cli` is probably at least as good for it anyway.

Once we've published a 0.8.0 under the new names, we'll release 0.7.1
versions under the old names with pointers to the new crates names.
2023-07-09 06:40:43 +02:00
Yuya Nishihara
564506a7c7 tests: fix test_fetch_prune_deleted_ref() to set up refs on remote
This is broken since aa78f97d55 "git: refactor tests by extracting some
common setup."
2023-07-09 10:08:46 +09:00
Ilya Grigoriev
b6a9423f38 git export: (almost) no-op refactor to export_refs to use RefName
This follows 3779b45, but in this case the refactor makes the logic more
complicated. The main goal here is to prepare for the next commit.
2023-07-03 11:01:22 -07:00
Yuya Nishihara
a07574a233 git: pass RefName enum to git_ref_filter callback
I think it's slightly better to compare each ref fragment than building
"refs/remotes/{remote}/{branch}" pattern to be matched.
2023-07-02 09:49:07 +09:00
Glen Choo
7afaa2487b git: add .gitmodules parser
This only parses the fields relevant to us, i.e.:

- name: the stable identifier of the submodule
- path: the path to the submodule in the current commit
- url: the remote we can clone the submodule from

The full list of .gitmodules fields can be found at
https://git-scm.com/docs/gitmodules.
2023-06-27 10:07:00 -07:00
Kevin Liao
86b6a11e63 Fix jj init --git-repo fails and leaves broken .jj folder
This commit fixes #1305

Before this commit, running `jj init --git-repo=./` in a folder that
does not have a .git would cause jj to panick and leave an unfinished corrupted jj repo.

This commit fixes that by changing the call chain to return an error
instead of calling .unwrap() and panicking. This commit also adds logic to delete the unfinished jj
repository when the git backend initialization failed.

Before this commit, running the above command would result in the following
```
Running `jj/target/debug/jj init --git-repo=./`
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error { code: -3, klass: 2, message: "failed to resolve path '/Users/kevincliao/github/jj/test-repo/.jj/repo/store/../../../.git': No such file or directory" }', lib/src/git_backend.rs:83:75
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
```

After this commit, the result is the following and the jj repo is deleted:
```
Running `jj/target/debug/jj init --git-repo=./`
Error: Failed to access the repository: Error: Failed to open git repository: failed to resolve path '/Users/kevincliao/github/jj/test-repo/.jj/repo/store/../../../.git': No such file or directory; class=Os (2); code=NotFound (-3)
```
2023-06-20 11:02:06 -07:00
Glen Choo
6621f261cc repo: add submodule_store, default impl
..and other assorted boilerplate. These are just stubs for now, but now
that we've reserved the `submodule_store` subdirectory, we can start
adding more functionality.
2023-06-19 09:48:58 -07:00
Yuya Nishihara
38a7e7fd62 git_backend: on read_commit(), bulk-update extra metadata table of ancestors
Otherwise, "jj init --git-repo ." would create extra table files per commit,
and merge them.

I considered adding an explicit GitBackend method to be called from
git::import_refs(), but the call order matters. The method should be invoked
before calling store.get_commit(..) or mut_repo.add_head(..). Since commits
are likely to be loaded from the head, we can instead make read_commit()
import ancestor metadata at all.

Alternatively, we could make a Git commit hidden until it's inserted into
the extra table. It's rather big change, and I wouldn't like to do that
without thinking more thoroughly.
2023-05-21 08:29:00 +09:00
Yuya Nishihara
a9422460cb git_backend: ensure change id generated from git commit id never reassigned
Fixes #924
2023-05-20 15:53:23 +09:00
Yuya Nishihara
9aa72f6f1d git_backend: add lock to prevent racy change id assignments
My first attempt was to fix up corrupted index when merging, but it turned
out to be not easy because the self side may contain corrupted data. It's
also possible that two concurrent commit operations have exactly the same
view state (because change id isn't hashed into commit id), and only the
table heads diverge.

#924
2023-05-20 15:53:23 +09:00
Yuya Nishihara
3655da4f01 tests: add tests for concurrent git commit/change id assignment
Since non-Git metadata isn't hashed, we can't rely on the consistency
provided by content-addressed storage. The problem is also described in
https://github.com/martinvonz/jj/issues/3#issuecomment-947998487

#924
2023-05-20 15:53:23 +09:00
Ilya Grigoriev
714aff63e6 git.rs: properly abandon commits from moved/deleted branches on remote (#864)
This bug concerns the way `import_refs` that gets called by `fetch` computes
the heads that should be visible after the import.

Previously, the list of such heads was computed *before* local branches were
updated based on changes to the remote branches. So, commits that should have
been abandoned based on this update of the local branches weren't properly
abandoned.

Now, `import_refs` tracks the heads that need to be visible because of some ref
in a mapping keyed by the ref. If the ref moves or is deleted, the
corresponding heads are updated.

Fixes #864
2023-05-17 17:57:58 -07:00
Ilya Grigoriev
cf4a603eb4 Tests demonstrating a similar bug with moved rather than deleted branch 2023-05-17 17:57:58 -07:00
Ilya Grigoriev
a0ee2b0dbd lib/tests/test_git.rs: New test to demonstrate #864's root cause 2023-05-17 17:57:58 -07:00
Ilya Grigoriev
bda3d3e50b test_import_refs_reimport: very minor improvement to a test 2023-05-17 17:57:58 -07:00
Martin von Zweigbergk
e7419e76a1 backend: replace git_repo() by as_any()
This has several advantages:

 * Makes it possible to downcast to non-Git custom backends (might be
   useful at Google, but we haven't needed it yet)

 * Lets us access more specific functionality on the `GitBackend`,
   making it possible to access the `git2::Repository` without
   creating a copy of it.

 * Removes the dependency on Git from the backend
2023-05-12 08:05:09 -07:00
Yuya Nishihara
92cfffd843 git: on external HEAD move, do not abandon old branch
The current behavior was introduced by 20eb9ecec1 "git: don't abandon
HEAD commit when it loses a branch." While the change made HEAD mutation
behavior more consistent with a plain ref operation, HEAD can also move on
checkout, and checkout shouldn't be considered a history rewriting operation.

I'm not saying the new behavior is always correct, but I think it's safer
than losing old HEAD branch. I also think this change will help if we want
to extract HEAD management function from git::import_refs().

Fixes #1042.
2023-05-11 10:15:31 +09:00
Yuya Nishihara
66d405fa5f git: add tests that simulate external checkout/amend in colocated repo
I'm going to change the behavior of _without_ref() case to mitigate #1042.
2023-05-11 10:15:31 +09:00
Martin von Zweigbergk
37151e0ff9 index: load store based on type recorded in .jj/repo/index/type
This is another step towards allowing a custom `jj` binary to have its
own index type. We're going to have a server-backed index
implementation at Google, for example.
2023-03-11 22:22:46 -08:00
Samuel Tardieu
182919ff6f git: add function to import a selection of the git refs 2023-03-02 10:09:08 +01:00
Samuel Tardieu
0ca4e2dad2 git: absence of globs is None rather than &[]
In `git_fetch()`, any glob present in `globs` is an "allow" mark. Using
`&[]` to represent an "allow-all" may be misleading, as it could
indicate that no branch (only the git HEAD) should be fetched.

By using an `Option<&[&str]>`, it is clearer that `None` means that
all branches are fetched.
2023-03-02 10:09:08 +01:00
Ilya Grigoriev
30d03a66e6 cmd: --branch option for git fetch.
Thanks to @samueltardieu for noticing a subtle bug in the refspecs, providing
the fix, as well as the two `conflicting_branches` tests.
2023-02-21 18:33:40 -08:00
Martin von Zweigbergk
f6a4cb57da repo: extract a Repo trait for Arc<ReadonlyRepo> and MutableRepo
This will soon replace the `RepoRef` enum, just like how the `Index`
trait replaced the `IndexRef` enum.
2023-02-15 19:15:17 -08:00
Martin von Zweigbergk
4e8fbaa210 git: allow conflicts in "HEAD@git"
Git's HEAD ref is similar to other refs and can logically have
conflicts just like the other refs in `git_refs`. As with the other
refs, it can happen if you run concurrent commands importing two
different updates from Git. So let's treat `git_head` the same as
`git_refs` by making it an `Option<RefTarget>`.
2023-01-30 09:05:03 -08:00
Glen Choo
3418c8ff73 git: add git.auto-local-branch
Add a new git.auto-local-branch config option. When set to false, a
remote-tracking branch imported from Git will not automatically create a
local branch target. This is implemented by a new GitSettings struct
that passes Git-related settings from UserSettings.

This behavior is particularly useful in a co-located jj and Git repo,
because a Git remote might have branches that are not of everyday
interest to the user, so it does not make sense to export them as local
branches in Git. E.g. https://github.com/gitster/git, the maintainer's
fork of Git, has 379 branches, most of which are topic branches kept
around for historical reasons, and Git developers wouldn't be expected
to have local branches for each remote-tracking branch.
2023-01-29 20:17:49 -08:00
Glen Choo
4716c1e9e2 git: test import of remote-only branch
This test coverage will become more important when we make changes to
remote branch importing.
2023-01-29 20:17:49 -08:00
Waleed Khan
7f8a196ab2 backend: create ObjectId trait
This lets us operate over various kinds of objects polymorphically (e.g. call `.hex()` on any kind of object hash).
2023-01-02 12:28:51 -06:00
Martin von Zweigbergk
f3208f59c4 store: propagate error from Backend::write_commit() 2022-12-26 23:30:52 -08:00
Martin von Zweigbergk
f1d7bbe508 testutils: create a function for writing a random commit to MutableRepo
We already have `create_random_commit()`, which returns a
`CommitBuilder`. Most callers directly write that to a
`MutableRepo`. That currently returns a `Commit`, but I'm about to
make it propagate errors from the backend. That would add an
`unwrap()` to this sequence, making it longer. Let's create a simple
helper for these callers to simplify this common pattern.
2022-12-26 23:30:52 -08:00
Martin von Zweigbergk
49b2f3b6ca commit_builder: keep MutableRepo reference
When you're done with the `CommitBuilder`, you're going to have to
call `write_to_repo()`, passing it a mutable `MutableRepo`
reference. It's a bit simpler to pass that reference when we create
the `CommitBuilder` instead, so that's what this patch does.

A drawback of passing in the mutable reference when we create the
builder is that we can't have multiple unfinished `CommitBuilder`
instance live at the same time. We don't have any such use cases yet,
and it's not hard to work around them, so I think this change is worth
it.
2022-12-26 23:30:52 -08:00
Daniel Ploch
e9bd6fbeae op_heads_store: give the OpHeadsStore factory semantics 2022-12-16 10:47:48 -08:00
Daniel Ploch
bd31bfd2d7 repo: give OpStore factory load semantics 2022-12-14 14:10:30 -08:00
Martin von Zweigbergk
8a440d8042 git: on export, use repo view's git_refs as record of old export state
@yuja asked on #701 about the difference between the state in the
`git_export_view` and what we have in `mut_repo.view()`. It's true
that the branches in `mut_repo.view().git_refs()` should match what we
wrote to disk. We can therefore remove the on-disk storage and
simplify quite a bit. For now, I create the `last_export_view` from
the `mut_repo.view().git_refs()` before calling
`export_changes()`. I'll clean up a bit more next.

I think this is correct even considering e.g. undo. Let's consider
what would happen in a non-colocated Git repo (not because tricky
cases cannot happen there but because the explicit exports and imports
make it easier to discuss, and more cases can occur). If the user
moved a branch and then did `jj git export`, `jj undo`, and then `jj
git export` again, we would think on the second export that we should
perform the same changes to the Git repo, which should have no effect.

This patch also fixes the bug we were forced to work around in the
test case in the previous patch.

This removes one of our uses of Thrift.
2022-12-03 09:32:49 -08:00
Martin von Zweigbergk
39792368ba git: when exporting, don't overwrite changes made by git
This fixes the bugs shown by the tests added in the previous patch by
checking that the git branches we're about to update have not been
updated by git since our last export. If they have, we fail those
branches. The user can then re-import from the git repo and resolve
any conflicts before exporting again.

I had to update the `test_export_import_sequence` to make it
pass. That shows a new bug, which I'll fix next. The problem is that
the exported view doesn't get updated on import, so we would try to
export changes compared to an earlier export, even though we actually
knew (because of the `jj git import`) that the state in git had
changed.
2022-12-03 09:32:49 -08:00
Martin von Zweigbergk
9b59461242 git: add test for concurrent change in git repo between exports
If you update a branch using regular `git` (or some Git-based tool)
between two `jj git export`, we will overwrite that change if you had
also changed the branch in jj land. There's a similar problem if you
delete the branch in jj land. Let's have a test for that. I'm going to
make us not overwrite it soon. This patch adds a test for those cases,
plus many other cases in consistent way. Since the new test covers
some cases tested by existing tests, I removed those tests.
2022-12-03 09:32:49 -08:00
Martin von Zweigbergk
d8feed9be4 copyright: change from "Google LLC" to "The Jujutsu Authors"
Let's acknowledge everyone's contributions by replacing "Google LLC"
in the copyright header by "The Jujutsu Authors". If I understand
correctly, it won't have any legal effect, but maybe it still helps
reduce concerns from contributors (though I haven't heard any
concerns).

Google employees can read about Google's policy at
go/releasing/contributions#copyright.
2022-11-28 06:05:45 -10:00
Martin von Zweigbergk
a90ef20976 git: on export, delete deleted refs before adding added refs
To reduce conflicts between branches like `main` and `main/sub`, it's
better to first delete refs in git that have been deleted in jj, and
then add/update refs that have been added/updated in jj.
2022-11-26 06:05:29 -10:00
Martin von Zweigbergk
4a03b94d65 git: on export, skip failed refs instead of failing whole export
Since we now write a (partial) view object of the exported branches to
disk (since 7904474320), we can safely skip exporting some
branches. We already skip conflicted branches. This commit makes us
also skip branches that we fail to write to the backing Git repo,
instead of failing the whole operation (after possibly updating some
Git refs).

I made the `export_refs()` function return the branches that
failed. We should probably make that a struct later and have a
separate field for branches that we skipped due to conflicts.

Closes #493.
2022-11-26 06:05:29 -10:00