Commit graph

1488 commits

Author SHA1 Message Date
dependabot[bot]
3575c1f499 cargo: bump digest from 0.10.6 to 0.10.7
Bumps [digest](https://github.com/RustCrypto/traits) from 0.10.6 to 0.10.7.
- [Commits](https://github.com/RustCrypto/traits/compare/digest-v0.10.6...digest-v0.10.7)

---
updated-dependencies:
- dependency-name: digest
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-05-22 16:16:04 +00:00
Yuya Nishihara
1675aec388 git: handle lock error that could occur while adding GC-preventing refs
If I spawned ~20 "jj status &" processes, some of them panicked there.
Spotted when debugging #924.
2023-05-22 08:36:29 +09:00
Yuya Nishihara
b01614bbdd cleanup: leverage scoped thread in tests 2023-05-21 21:02:58 +09:00
Yuya Nishihara
38a7e7fd62 git_backend: on read_commit(), bulk-update extra metadata table of ancestors
Otherwise, "jj init --git-repo ." would create extra table files per commit,
and merge them.

I considered adding an explicit GitBackend method to be called from
git::import_refs(), but the call order matters. The method should be invoked
before calling store.get_commit(..) or mut_repo.add_head(..). Since commits
are likely to be loaded from the head, we can instead make read_commit()
import ancestor metadata at all.

Alternatively, we could make a Git commit hidden until it's inserted into
the extra table. It's rather big change, and I wouldn't like to do that
without thinking more thoroughly.
2023-05-21 08:29:00 +09:00
Yuya Nishihara
fe97dccd02 git_backend: move add_entry() of extra metadata table to caller
I'm going to add a caller which will insert multiple entries at once.
2023-05-21 08:29:00 +09:00
Yuya Nishihara
e6addf7905 git_backend: extract helper that converts git2::Commit to backend::Commit
The root parent id is filled by caller because empty parents list is more
convenient while walking ancestors.
2023-05-21 08:29:00 +09:00
Yuya Nishihara
0149e7b311 git_backend: generate change id from git2::Commit object
I'm going to extract a helper function that converts git2::Commit to
backend::Commit struct, and the commit id can also be obtained from the
git2::Commit object.
2023-05-21 08:29:00 +09:00
Yuya Nishihara
5dba0502cb git_backend: cache head of saved extra metadata table
Just because we know the latest table head.
2023-05-21 08:29:00 +09:00
Yuya Nishihara
a9422460cb git_backend: ensure change id generated from git commit id never reassigned
Fixes #924
2023-05-20 15:53:23 +09:00
Yuya Nishihara
9aa72f6f1d git_backend: add lock to prevent racy change id assignments
My first attempt was to fix up corrupted index when merging, but it turned
out to be not easy because the self side may contain corrupted data. It's
also possible that two concurrent commit operations have exactly the same
view state (because change id isn't hashed into commit id), and only the
table heads diverge.

#924
2023-05-20 15:53:23 +09:00
Yuya Nishihara
e224044dea git_backend: consistently use CommitId type to look up extra metadata table 2023-05-20 15:53:23 +09:00
Yuya Nishihara
78c8dbc8fe git_backend: extract helper to add extra metadata entry and save table 2023-05-20 15:53:23 +09:00
Yuya Nishihara
8a0fcfb032 git_backend: leverage read_extra_metadata_table() in write_commit()
And use the readonly table for lookup, which allows us to extract a helper
method to add/save entry.
2023-05-20 15:53:23 +09:00
Yuya Nishihara
14243a85a0 git_backend: extract helper to read extra metadata table and maintain cache 2023-05-20 15:53:23 +09:00
Yuya Nishihara
83753e59f6 stacked_table: extract method that resolves head without releasing lock
GitBackend will reuse this lock to not assign multiple change ids to a
single commit. We could add a separate lock file that covers the section
from get_head() to save_table(), but I think reusing the table lock is good
enough.
2023-05-20 15:53:23 +09:00
Yuya Nishihara
3655da4f01 tests: add tests for concurrent git commit/change id assignment
Since non-Git metadata isn't hashed, we can't rely on the consistency
provided by content-addressed storage. The problem is also described in
https://github.com/martinvonz/jj/issues/3#issuecomment-947998487

#924
2023-05-20 15:53:23 +09:00
Yuya Nishihara
b8e0ba9872 stacked_table: do not remove head on empty save_table()
If the head is unchanged, it shouldn't be removed. Otherwise the next reader
wouldn't find the existing table.
2023-05-18 22:55:26 +09:00
Ilya Grigoriev
db8fcf933a export_refs: add or edit some comments (no-op)
This is supposed to make `export_refs` a little more readable.
2023-05-17 17:57:58 -07:00
Ilya Grigoriev
344b3bfa59 import_refs/export_refs: refactor conversion from branch names to refs (no-op)
This is supposed to make `import_refs` and `export_refs` a little less prone to typos
2023-05-17 17:57:58 -07:00
Ilya Grigoriev
714aff63e6 git.rs: properly abandon commits from moved/deleted branches on remote (#864)
This bug concerns the way `import_refs` that gets called by `fetch` computes
the heads that should be visible after the import.

Previously, the list of such heads was computed *before* local branches were
updated based on changes to the remote branches. So, commits that should have
been abandoned based on this update of the local branches weren't properly
abandoned.

Now, `import_refs` tracks the heads that need to be visible because of some ref
in a mapping keyed by the ref. If the ref moves or is deleted, the
corresponding heads are updated.

Fixes #864
2023-05-17 17:57:58 -07:00
Ilya Grigoriev
cf4a603eb4 Tests demonstrating a similar bug with moved rather than deleted branch 2023-05-17 17:57:58 -07:00
Ilya Grigoriev
a0ee2b0dbd lib/tests/test_git.rs: New test to demonstrate #864's root cause 2023-05-17 17:57:58 -07:00
Ilya Grigoriev
bda3d3e50b test_import_refs_reimport: very minor improvement to a test 2023-05-17 17:57:58 -07:00
Ilya Grigoriev
07e7b82a0d import_refs/export_refs: rename some local variables (no-op)
This is supposed to make `import_refs` and `export_refs` a little more
readable.
2023-05-17 17:57:58 -07:00
Martin von Zweigbergk
87a925d736 git_backend: return timestamps for what was actually written
Now that we return the written commit from `write_commit()`, let's
make the timestamps match what was actually written, accounting for
the whole-second precision and the adjustment we do to avoid
collisions.
2023-05-12 15:20:44 -07:00
Martin von Zweigbergk
a95188ddbc backend: take commit to write by value and return new value
The internal backend at Google doesn't let you write any value you
want for in the committer field. The `Store` type still caches the
value it attempted to write, which gets a little weird when the
written value is not what we tried to write. We should use the value
the backend actually wrote. However, we don't know if the backend
changed anything without reading the value back, which is often
wasteful. This commit changes the API to return the written value.

I only changed the signature of `write_commit()` for now. Maybe we
should make a similar change to `write_tree()`.
2023-05-12 15:20:44 -07:00
Martin von Zweigbergk
e7419e76a1 backend: replace git_repo() by as_any()
This has several advantages:

 * Makes it possible to downcast to non-Git custom backends (might be
   useful at Google, but we haven't needed it yet)

 * Lets us access more specific functionality on the `GitBackend`,
   making it possible to access the `git2::Repository` without
   creating a copy of it.

 * Removes the dependency on Git from the backend
2023-05-12 08:05:09 -07:00
Yuya Nishihara
8d56b199bc revset: initialize with default prefix resolver
Since repo is passed as argument, we can define the default resolver as
a plain function.
2023-05-12 21:31:29 +09:00
Yuya Nishihara
f58beca760 revset: move resolve_symbol() to tests
It's no longer used in library code.
2023-05-12 21:31:29 +09:00
Martin von Zweigbergk
916b00c33e id_prefix: remove repo field from IdPrefixContext
By passing the repo as argument to the methods instead, we can remove
the `repo` field and the associated lifetime. Thanks to Yuya for the
suggestion.
2023-05-11 23:41:24 -07:00
Martin von Zweigbergk
6a4502cb5d prefixes: allow resolving shorter ids within a revset
In large repos, the unique prefixes can get somewhat long (~6 hex
digits seems typical in the Linux repo), which makes them less useful
for manually entering on the CLI. The user typically cares most about
a small set of commits, so it would be nice to give shorter unique ids
to those. That's what Mercurial enables with its
`experimental.revisions.disambiguatewithin` config. This commit
provides an implementation of that feature in `IdPrefixContext`.

In very large repos, it can also be slow to calculate the unique
prefixes, especially if it involves a request to a server. This
feature becomes much more important in such repos.
2023-05-11 23:41:24 -07:00
Martin von Zweigbergk
2e12aad1f7 id_prefix: add IdIndex::has_key()
For the support for shorter prefixes within a revset, we'll want to be
able to check if an id is in the index.
2023-05-11 23:41:24 -07:00
Martin von Zweigbergk
481b8c5d0e id_prefix: add IdIndex::resolve_prefix()
I'll use this in `IdPrefixContext` soon.
2023-05-11 23:41:24 -07:00
Martin von Zweigbergk
f657bcb6ae prefixes: move IdIndex to id_prefix module
I'll reuse it there next.
2023-05-11 23:41:24 -07:00
Martin von Zweigbergk
5612a3106c revset: use IdPrefixContext for resolving commit/change ids
This is another step towards resolving abbreviated commit ids within a
configured revset.
2023-05-11 23:41:24 -07:00
Martin von Zweigbergk
f66efcf6f9 revset: inline resolution of change/commit ids
This prepares for adding callbacks to resolve these ids.
2023-05-11 23:41:24 -07:00
Martin von Zweigbergk
c8648cb300 templater: move id prefix shortening onto a new type
I would like to copy Mercurial's way of abbreviating ids within a
user-configurable revset. We would do it for both commit ids and
change ids. For that feature, we need a place to keep the set of
commits the revset evaluates to. This commit adds a new
`IdPrefixContext` type which will eventually be that place. The new
type has functions for going back and forth between full and
abbreviated ids. I've updated the templater to use it.
2023-05-11 23:41:24 -07:00
Martin von Zweigbergk
efd743339c revset: don't allow symbols in RevsetExpression::resolve()
When creating `RevsetExpression` programmatically, I think we should
use commit ids instead of symbols in the expression. This commit adds
a check for that by using a `SymbolResolver` that always errors
out.
2023-05-11 23:41:24 -07:00
Martin von Zweigbergk
99e9cd70d1 cli: make WorkspaceCommandHelper create SymbolResolver
I would eventually want the `SymbolResolver` to be customizable (in
custom `jj` binaries), so we want to make sure we always use the
customized version of it.

I left `RevsetExpression::resolve()` unchanged. I consider that to be
for programmatically created expressions.
2023-05-11 23:41:24 -07:00
Martin von Zweigbergk
5e7c57c527 revset: introduce a trait for resolving symbols
I'd like to make the symbol resolution more flexible, both so we can
support customizing it (in custom `jj` binaries) and so we can use it
for resolving short prefixes within a small revset.
2023-05-11 23:41:24 -07:00
Martin von Zweigbergk
ac31c83e13 cli: rename ui.default-revset to revsets.log
I plan to add `revsets.short-prefixes` and `revsets.immutable` soon,
and I think `[revsets]` seems like reasonable place to put them. It
seems consistent with our `[templates]` section. However, it also
suffers from the same problem as that section, which is that the
difference between `[templates]` and `[template-aliases]` is not
clear. We can decide about about templates and revsets later.
2023-05-11 23:41:24 -07:00
Yuya Nishihara
92cfffd843 git: on external HEAD move, do not abandon old branch
The current behavior was introduced by 20eb9ecec1 "git: don't abandon
HEAD commit when it loses a branch." While the change made HEAD mutation
behavior more consistent with a plain ref operation, HEAD can also move on
checkout, and checkout shouldn't be considered a history rewriting operation.

I'm not saying the new behavior is always correct, but I think it's safer
than losing old HEAD branch. I also think this change will help if we want
to extract HEAD management function from git::import_refs().

Fixes #1042.
2023-05-11 10:15:31 +09:00
Yuya Nishihara
66d405fa5f git: add tests that simulate external checkout/amend in colocated repo
I'm going to change the behavior of _without_ref() case to mitigate #1042.
2023-05-11 10:15:31 +09:00
Benjamin Saunders
d747b2f362 lib: use advisory locks on Unix targets
Allows automatic recovery when encountering stale lockfiles, and more
efficient blocking rather than polling for fresh ones. The previous
implementation is preserved for other platforms.
2023-05-07 09:52:09 -07:00
Benjamin Saunders
ccbb34fddb working_copy: introduce snapshot progress callback 2023-05-06 11:07:46 -07:00
Yuya Nishihara
770ea0c705 tree: rewrite recursive diff iterator to not use machine stack
While mutable borrow in the next() method is a bit tricky, I think it's
easier to follow than invoking iterators recursively.
2023-05-06 14:50:37 +09:00
Yuya Nishihara
a8f77e46a9 tree: extract helper method that sets up subdir iterator
This will be reimplemented as a constructor of subdir item.
2023-05-06 14:50:37 +09:00
Yuya Nishihara
3f7b0929d7 tree: make internal TreeEntryDiffIterator yield all values by reference
This isn't important, but I feel it's a bit inconsistent that name is cloned
by callee, but tree values aren't.
2023-05-06 14:50:37 +09:00
Grégoire Geis
104f8e154c Fix Git/SSH on Windows
There were two issues on my end:
1. `known_hosts` doesn't seem to be recognized
2. SSH Agent is ignored despite running

A workaround for 1. is to set the HOME environment variable on Windows, so I added a hint to suggest this. Ideally we would add a `certificate_check` callback to the remote callbacks, but the git2 crate doesn't expose whether the certificate check already succeeded, which makes it useless for this purpose (as we'd be prompting users to accept a certificate even though that certificate is already known to be valid).

As for 2., I changed the behavior from "check SSH Agent if some env variables exist" to "check SSH Agent and only fail if some env variables exist". On Windows SSH Agent doesn't use these env variables (but trying to communicate with it will still work), so now Windows properly works with SSH Agent.
2023-05-04 23:57:06 +09:00
Martin von Zweigbergk
cd0023e796 tree: if matcher says that tree should be skipped, skip it
When using a sparse working copy (e.g. with no files at all) and
updating the working copy from the root commit to a commit with
millions of files, we shouldn't have to walk the parts of the diff
that doesn't match the sparse patterns. However, we still do the full
walk because our `Tree::diff()` currently doesn't care about what the
matcher tells us to visit, it only filters out unwanted files after
visiting them. This commit fixes that for the special (but common)
case of matching nothing in a directory.

I tried also adding special handling for when the matcher says that we
should only visit a few entries, but it wasn't clearly better in the
cases I tested it on. I'll keep that patch around and might send it if
I find some cases where it helps.
2023-05-03 18:21:51 -07:00