Commit graph

61 commits

Author SHA1 Message Date
Martin von Zweigbergk
96e75e6ad1 simple_op_store: return NotFound for missing views
I guess we don't depend on `read_view()` ever returning `NotFound` the
way `read_operation()` does, but it seems like it still should return
`NotFound` when the view doesn't exist.
2023-07-26 14:17:21 -07:00
Martin von Zweigbergk
769426f99a op_store: make OpStoreError::Other preserve source error object
This is the OpStore version of e1e75daa8e.
2023-07-26 14:17:21 -07:00
Yuya Nishihara
2bbaa4352a refs: rename RefTarget::is_conflict() to has_conflict()
Copied from MergedTree::has_conflict(). I feel it's slightly better since
RefTarget "is" always a Conflict-based type. It could be inverted to
RefTarget::is_resolved(), but refs are usually resolved, and all callers
have special case for !is_resolved() state.
2023-07-23 22:25:57 +09:00
Yuya Nishihara
4da8483228 refs: reimplement RefTarget as Conflict<Option<CommitId>> wrapper
ContentHash is preserved at this point. I'll update it together with the
serialization format changes.
2023-07-18 18:12:09 +09:00
Yuya Nishihara
443391bf8f view: store Option<RefTarget> in maps, add extension trait to flatten Option
Alternatively, we can wrap BTreeMap<String, Option<RefTarget>> to flatten
Option<&Option<..>> internally, but doing that would be tedious. It would
also be unclear if map.remove(name) should construct an absent RefTarget if
the ref doesn't exist.
2023-07-18 18:12:09 +09:00
Yuya Nishihara
7461370f6e view: add wrapper that will exclude absent RefTarget entries from ContentHash
The next commit will change these maps to store Option<RefTarget> entries, but
None entries will still be omitted from the serialized data. Since ContentHash
should describe the serialized data, relying on the generic ContentHash would
cause future hash conflict where absent RefTarget entries will be preserved.

For example, ([remove], [None, add]) will be serialized as ([remove], [add]),
and deserialized to ([remove], [add, None]). If we add support for lossless
serialization, hash(([remove], [None, add])) should differ from the lossy one.
2023-07-18 18:12:09 +09:00
Yuya Nishihara
0461a8575a refs: add stub constructors for absent RefTarget, replace None with it
Now we're mostly ready to reimplement RefTarget on top of
Conflict<Option<CommitId>>.
2023-07-17 08:24:24 +09:00
Yuya Nishihara
4cfe2591b6 refs: turn RefTarget::adds()/removes() into iterator, rename accordingly
Since RefTarget will be reimplemented on top of Conflict<Option<CommitId>>,
we won't be able to simply return a slice of type &[CommitId]. These functions
are also renamed in order to disambiguate from Conflict::adds()/removes().
2023-07-17 08:24:24 +09:00
Yuya Nishihara
d29b10a8dd simple_op_store: rewrite RefTarget deserialization to work with Option<_>
.unwrap() calls will be removed if we migrate RefTarget to new Conflict-based
type. Some of them were previously .unwrap_or_default(), but they would panic
anyway inside ref_target_from_proto().
2023-07-17 01:13:21 +09:00
Yuya Nishihara
75429293f2 simple_op_store: rewrite RefTarget serialization to work with Option<_>
is_none()/is_some() are wrapped as is_absent()/is_present() respectively.
If we see new RefTarget as a Conflict, an "absent" target isn't an emptyish
value like None, but an add of single [None]. It seemed a bit weird to call
such target is_none().
2023-07-17 01:13:21 +09:00
Yuya Nishihara
a1e92563ae refs: add constructor of Some(RefTarget::Conflict { .. })
It's named after Conflict::from_legacy_form(). If RefTarget is migrated to
new Conflict type, from_legacy_form([], [add]) will create a normal target,
and from_legacy_form([], []) will be equivalent to the current None target.
That's why this function isn't named as RefTarget::conflict().
2023-07-17 01:13:21 +09:00
Yuya Nishihara
e55c533add refs: add constructor of Some(RefTarget::Normal(id))
If RefTarget is migrated to new Conflict type, this function will create
RefTarget(Conflict::resolved(Some(id))).

We still need some .unwrap() to insert Option<RefTarget> into map, but maps
will be changed to store new RefTarget type, and their mutation API will
guarantee that Conflict::resolved(None) is eliminated.
2023-07-17 01:13:21 +09:00
Waleed Khan
54dba51a08 docs: warn about missing docs for jj-lib crate 2023-07-10 18:28:59 +03:00
Ilya Grigoriev
c298339f98 proto_op_store: Add two minor comments 2023-04-16 22:04:27 -07:00
Martin von Zweigbergk
6643fb2bff op_store: inline ProtoOpStore into SimpleOpStore
The `ProtoOpStore` was separated out to simplify the migration from
Thrift. Now that the `ThriftOpStore` is gone, we can inline
`ProtoOpStore` as the TODO says.
2023-03-30 20:00:33 -07:00
Martin von Zweigbergk
68fb46b2af op_store: drop support for upgrading from Thrift implementation 2023-03-30 20:00:33 -07:00
Martin von Zweigbergk
4e8fbaa210 git: allow conflicts in "HEAD@git"
Git's HEAD ref is similar to other refs and can logically have
conflicts just like the other refs in `git_refs`. As with the other
refs, it can happen if you run concurrent commands importing two
different updates from Git. So let's treat `git_head` the same as
`git_refs` by making it an `Option<RefTarget>`.
2023-01-30 09:05:03 -08:00
Waleed Khan
7f8a196ab2 backend: create ObjectId trait
This lets us operate over various kinds of objects polymorphically (e.g. call `.hex()` on any kind of object hash).
2023-01-02 12:28:51 -06:00
Martin von Zweigbergk
fb396e6b45 cleanup: address unnecessary_borrow lints reported by Clippy 1.66
Interestingly, the nightly release doesn't complain about this. One
instance in `test_working_copy.rs` is a bug in Clippy.
2022-12-15 12:38:01 -08:00
Daniel Ploch
bd31bfd2d7 repo: give OpStore factory load semantics 2022-12-14 14:10:30 -08:00
Daniel Ploch
25c379429c op_store: init/load by &Path, for consistency with other stores 2022-12-14 14:10:30 -08:00
Martin von Zweigbergk
84a4dee673 thrift: add legacy-thrift Cargo feature
This patch adds a `legacy-thrift` Cargo feature that's enabled by
default. If it's disabled, the upgrade from Thrift-based operation log
does not happen, and the `thrift` depdendency is not included.
2022-12-03 22:31:02 -08:00
Martin von Zweigbergk
c269b72fb3 op_store: auto-upgrade existing repos from Thrift to Protobuf
With this patch, we auto-upgrade existing repos that use Thrift format
for the operation log to use Protobuf format. That would only be repos
used with an unreleased version of jj after 0.5.1 (which may be the
majority of repos?).

The upgrade from Thrift is simpler because we now use the same hashing
scheme for the Protobuf-based storage, so the operation and view IDs
remain the same as they were in the Thrift-based storage. We could
simplify the code a bit more as a result, but since this code is
supposed to be short-lived, I didn't bother.

Since the change from the Protobuf format with the old hashing scheme
to a the (same) Protobuf format with the new hashing scheme shouldn't
impact users, I removed the entry we had in the changelog about the
format change.
2022-12-03 22:31:02 -08:00
Martin von Zweigbergk
5a9d1e5fdd op_store: rename variables in upgrade code to be direction-agnostic
The code for migrating from ProtoBuf to Thrift is almost completely
independent of which direction the upgrade goes, so we can very easily
reuse it for migrating from Thrift to Protobuf. This patch renames
some variables to "old/new" instead of "proto/thrift", making the next
patch even simpler.
2022-12-03 22:31:02 -08:00
Martin von Zweigbergk
472dee6403 op_store: move ThriftOpStore to new file
Since we're now allowed to use the `protobuf` crate, I'm going to make
`SimpleOpStore` use it again. This moves the `ThriftOpStore` into a
new `legacy_thrift_op_store.rs` file.
2022-12-03 22:31:02 -08:00
Martin von Zweigbergk
fdf43b845a content_hash: absorb duplicate hash() functions
We use the same blake2b hash for `ContentHash` impls in several
places, and I'm about to add more places, so let's centralize the
helper function.
2022-12-03 22:31:02 -08:00
Martin von Zweigbergk
c0a819e94a op_store: remove legacy_protobuf Cargo feature
Since we now have approval to use the `protobuf` crate at Google, it's
no longer the "legacy" format, so we should remove it. I'll almost
definitely soon add `legacy_thrift` feature instead.
2022-12-03 22:31:02 -08:00
Martin von Zweigbergk
d8feed9be4 copyright: change from "Google LLC" to "The Jujutsu Authors"
Let's acknowledge everyone's contributions by replacing "Google LLC"
in the copyright header by "The Jujutsu Authors". If I understand
correctly, it won't have any legal effect, but maybe it still helps
reduce concerns from contributors (though I haven't heard any
concerns).

Google employees can read about Google's policy at
go/releasing/contributions#copyright.
2022-11-28 06:05:45 -10:00
Ilya Grigoriev
0b528eb587 Fix some cargo +nightly clippy warnings 2022-11-24 20:29:17 -08:00
Yuya Nishihara
e0fe3ffd7d simple_op_store: create temporary Thrift store in .jj/repo directory
Otherwise rename() would fail if /tmp is on different device.
2022-11-14 11:13:41 +09:00
Martin von Zweigbergk
f3f41cc2cc git: write view object - not operation ID - to disk on export
We could write the view object to the operation store instead, but
then we would need to make sure we don't GC it (once we add support
for GC of the operation store).
2022-11-13 15:06:10 -08:00
Martin von Zweigbergk
4ee261717e simple_op_store: replace Protobuf by Thrift
As mentioned in the previous commit, we need to remove the Protobuf
dependency in order to be allowed to import jj into Google's
repo. This commit makes `SimpleOpStore` store its data using Thrift
instead of Protobufs. It also adds automatic upgrade of existing
repos. The upgrade process took 18 s in my repo, which has 22k
operations. The upgraded storage uses practically the same amount of
space. `jj op log` (the full outut) in my repo slowed down from 1.2 s
to 3.4 s. Luckily that's an uncommon operation. I couldn't measure any
difference in `jj status` (loading a single operation).
2022-11-13 11:39:33 -08:00
Martin von Zweigbergk
8acb54b26e op_store: extract current Protobuf-based implementation to separate file
In order to allow building jj inside of Google, our Protobuf team
doesn't want to us to use a Google-unsupported implementation. Since
there is no supported implementation in Rust, we have to migrate off
of Protobufs. I'm starting with the operation store. This commit moves
the current implementation to a separate file so it can easily be
disabled by a Caargo feature.
2022-11-13 11:39:33 -08:00
Benjamin Saunders
2447dfeed8 simple_op_store: hash view/operation data directly
Decouples view/operation IDs from serialized forms, which are not
necessarily stable. Not breaking as these IDs are persistent, never
recomputed or used for integrity checking.
2022-11-12 21:40:36 -08:00
Martin von Zweigbergk
3c7c4e9f5c tests: move testutils module into separate crate
The `testutils` module should ideally not be part of the library
dependencies. Since they're used by the integration tests (and the CLI
tests), we need to move them to a separate crate to achieve that.
2022-11-08 07:29:35 -08:00
Martin von Zweigbergk
3fc7b549ec cleanup: rename some more variables from checkout_* to ~wc_ 2022-09-29 23:51:34 -07:00
Martin von Zweigbergk
b654a1fe84 cleanup: remove extern crate declarations
`extern crate` is no longer needed since edition 2018.
2022-09-21 22:24:09 -07:00
Martin von Zweigbergk
6812bd9584 cleanup: rename checkout to wc_commit
`wc_commit` seems clearer than `checkout` and not too much longer. I
considered `working_copy` but it was less clear (could be the path to
the working copy, or an instance of `WorkingCopy`). I also considered
`working_copy_commit`, but that seems a bit too long.
2022-09-18 16:19:58 -07:00
Yuya Nishihara
872081c867 tests: use testutils::new_temp_dir() thoroughly 2022-09-07 23:49:46 +09:00
Martin von Zweigbergk
ffc57310f6 cargo: upgrade protobuf crates to 3.0.1
The biggest difference in the API is that fields are now public. The
exception from that is `oneof` fields, which still require setters and
getters.

I couldn't measure any difference in performance. I didn't expect any
difference either, but it's good that it didn't seem to regress. I
timed `jj debug operation <some hash prefix>`, which will read the
whole operation log (to check that the prefix is unambiguous).
2022-05-04 17:02:11 -07:00
Martin von Zweigbergk
cba89886e5 cleanup: replace some for x in xs.iter() by for x in &xs 2022-05-04 17:02:11 -07:00
Martin von Zweigbergk
33cc6b1253 cargo: upgrade blake2 2022-02-09 22:19:04 -08:00
Martin von Zweigbergk
0098edd3d2 op_store: add support for tracking multiple workspaces (#13)
This patch teaches the `View` object to keep track of the checkout in
each workspace. It serializes that information into the `OpStore`. For
compatibility with existing repos, the existing field for a single
workspace's checkout is interpreted as being for the workspace called
"default".

This is just an early step towards support for multiple
workspaces. Remaining things to do:

 * Record the workspace ID somewhere in `.jj/` (maybe in
   `.jj/working_copy/`)

 * Update existing code to use the workspace ID instead of assuming
   it's always "default" as we do after this patch

 * Add a way of indicating in `.jj/` that the repo lives elsewhere and
   make it possible to load a repo from such workspaces

 * Add a command for creating additional workspaces

 * Show each workspace's checkout in log output
2022-02-02 08:15:22 -08:00
Martin von Zweigbergk
8a2f630ac0 git: start tracking HEAD of underlying Git repo
This patch adds a place for tracking the current `HEAD` commit in the
underlying Git repo. It updates `git::import_refs()` to record it. We
don't use it anywhere yet.

This is part of #44.
2021-12-01 11:08:53 -08:00
Martin von Zweigbergk
046b3c0541 op_store: make Vec inside ViewId and OperationId non-public 2021-11-19 23:19:13 -08:00
Martin von Zweigbergk
8cf5dd286a backend: make Vec inside CommitId non-public
The recent e5dd93cbf7, whose description says "cleanup: make Vec
inside CommitId etc. non-public", made all ID types in the `backend`
module *except* for `CommitId` non-public :P This patch makes
2021-11-19 23:19:00 -08:00
Martin von Zweigbergk
8bbfe9731e cleanup: let newer Clippy fix a few things it found 2021-10-13 08:27:44 -07:00
Martin von Zweigbergk
ce5e95fa80 store: rename Store to Backend and StoreWrapper to Store
For what's currently called `Store` in the code, I have been using
"backend" in plain text. That probably means that `Backend` is a good
name for it.
2021-09-12 12:02:10 -07:00
Martin von Zweigbergk
044f23bc33 view: add support for ref-based branches and tags to model
I've finally decided to copy Git's branching model (issue #21), except
that I'm letting the name identify the branch across
remotes. Actually, now that I think about, that makes them more like
Mercurial's "bookmarks". Each branch will record the commit it points
to locally, as well as the commits it points to on each remote (as far
as the repo knows, of course). Those records are effectively the same
thing as Git's "remote-tracking branches"; the difference is that we
consider them the same branch. Consequently, when you pull a new
branch from a remote, we'll create that branch locally.

For example, if you pull branch "main" from a remote called "origin",
that will result in a local branch called "main", and also a record of
the position on the remote, which we'll show as "main@origin" in the
CLI (not part of this commit). If you then update the branch locally
and also pull a new target for it from "origin", the local "main"
branch will be divergent. I plan to make it so that pushing "main"
will update the remote's "main" iff it was currently at "main@origin"
(i.e. like using Git's `git push --force-with-lease`).

This commit adds a place to store information about branches in the
view model. The existing git_refs field will be used as input for the
branch information. For example, we can use it to tell if
"refs/heads/main" has changed and how it has changed. We will then use
that ref diff to update our own record of the "main" branch. That will
come later. In order to let git_refs take a back seat, I've also added
tags (like Git's lightweight tags) to the model in this commit.

I haven't ruled out *also* having some more persistent type of
branches (like Mercurials branches or topics).
2021-08-04 11:33:57 -07:00
Martin von Zweigbergk
7511b02da8 simple_op_store: add some tests
I'm about to start branches and tags in the view and we didn't have
any tests of the store itself (only via higher-level tests).
2021-07-31 19:49:30 -07:00