Commit graph

21 commits

Author SHA1 Message Date
Anton Bulakh
eb1c0ab4a2 sign: Implement a test signing backend and add a few basic tests 2023-11-30 23:36:56 +02:00
Martin von Zweigbergk
5174489959 backend: make read functions async
The commit backend at Google is cloud-based (and so are the other
backends); it reads and writes commits from/to a server, which stores
them in a database. That makes latency much higher than for disk-based
backends. To reduce the latency, we have a local daemon process that
caches and prefetches objects. There are still many cases where
latency is high, such as when diffing two uncached commits. We can
improve that by changing some of our (jj's) algorithms to read many
objects concurrently from the backend. In the case of tree-diffing, we
can fetch one level (depth) of the tree at a time. There are several
ways of doing that:

 * Make the backend methods `async`
 * Use many threads for reading from the backend
 * Add backend methods for batch reading

I don't think we typically need CPU parallelism, so it's wasteful to
have hundreds of threads running in order to fetch hundreds of objects
in parallel (especially when using a synchronous backend like the Git
backend). Batching would work well for the tree-diffing case, but it's
not as composable as `async`. For example, if we wanted to fetch some
commits at the same time as we were doing a diff, it's hard to see how
to do that with batching. Using async seems like our best bet.

I didn't make the backend interface's write functions async because
writes are already async with the daemon we have at Google. That
daemon will hash the object and immediately return, and then send the
object to the server in the background. I think any cloud-based
solution will need a similar daemon process. However, we may need to
reconsider this if/when jj gets used on a server with a custom backend
that writes directly to a database (i.e. no async daemon in between).

I've tried to measure the performance impact. That's the largest
difference I've been able to measure was on `jj diff
--ignore-working-copy -s --from v5.0 --to v6.0` in the Linux repo,
which increases from 749 ms to 773 ms (3.3%). In most cases I've
tested, there's no measurable difference. I've tried diffing from the
root commit, as well as `jj --ignore-working-copy log --no-graph -r
'::v3.0 & author(torvalds)' -T 'commit_id ++ "\n"'` (to test a
commit-heavy load).
2023-10-08 23:36:49 -07:00
Waleed Khan
1633eccdca Use { workspace = true } to appease VS Code's Cargo.toml parser
The VS Code "Better TOML" plugin (which I think most of our VS Code developers use?) doesn't support the `x.y = z` syntax at the top level, even though it's valid TOML. 

This is also useful if we ever want to add additional properties in different sub-crates (although unlikely for the near future).
2023-08-22 21:38:53 -07:00
Austin Seipp
d858db7e85 cargo: unify a lot of crate metadata in the workspace
Summary: There's no need to go around specifying `rust-version` or `edition` or
`version` several times, now that we have a global workspace. Instead, inherit
workspace metadata from the top-level Cargo.toml file.

Signed-off-by: Austin Seipp <aseipp@pobox.com>
Change-Id: Iaf905445978ed2b3377239dcdb8a6c32
2023-08-06 16:44:33 -05:00
Austin Seipp
13fff3be70 cargo: unify dependency versions through workspace deps
Summary: This moves all dependencies across the jj-lib and jj-cli crates into
the top-level Cargo file; with that, we can change each crate instead to just
inherit the workspace version, with the toggled features enabled, by setting
a dependency such as:

    dep.workspace = true

in the relevant Cargo.toml file.

This doesn't actually change any of the build semantics (from what I can tell)
nor the lockfile, and seems to respond normally. There are more cleanups that
can follow.

Two notes:

- Dependabot seems to work fine, based on what I've seen in other repos.
- `cargo add` doesn't seem to know how to add packages to a top-level
  `workspace.dependencies` field; instead you can `cargo add -p jj-cli`
  and move the entries, at least.

Signed-off-by: Austin Seipp <aseipp@pobox.com>
Change-Id: I307827e5f15c0d8ea8e2a80ec793d3c7
2023-08-06 16:44:33 -05:00
dependabot[bot]
0a6bcb0fbe cargo: bump the cargo-dependencies group with 3 updates
Bumps the cargo-dependencies group with 3 updates: [serde](https://github.com/serde-rs/serde), [tempfile](https://github.com/Stebalien/tempfile) and [thiserror](https://github.com/dtolnay/thiserror).


Updates `serde` from 1.0.173 to 1.0.174
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.173...v1.0.174)

Updates `tempfile` from 3.6.0 to 3.7.0
- [Changelog](https://github.com/Stebalien/tempfile/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Stebalien/tempfile/compare/v3.6.0...v3.7.0)

Updates `thiserror` from 1.0.43 to 1.0.44
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.43...1.0.44)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: cargo-dependencies
- dependency-name: tempfile
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: cargo-dependencies
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: cargo-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-07-21 15:34:03 +00:00
Martin von Zweigbergk
8149ec6df6 release: release version 0.8.0
Thanks to everyone who's contributed!
2023-07-16 21:40:24 +01:00
Martin von Zweigbergk
aac5b7aa25 cargo: rename crates from jujutsu/jujutsu-lib to jj-cli/jj-lib
Almost everyone calls the project "jj", and there seeems to be
consensus that we should rename the crates. I originally wanted the
crates to be called `jj` and `jj-lib`, but `jj` was already
taken. `jj-cli` is probably at least as good for it anyway.

Once we've published a 0.8.0 under the new names, we'll release 0.7.1
versions under the old names with pointers to the new crates names.
2023-07-09 06:40:43 +02:00
dependabot[bot]
e6ab3f132f cargo: bump itertools from 0.10.5 to 0.11.0
Bumps [itertools](https://github.com/rust-itertools/itertools) from 0.10.5 to 0.11.0.
- [Changelog](https://github.com/rust-itertools/itertools/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-itertools/itertools/compare/v0.10.5...v0.11.0)

---
updated-dependencies:
- dependency-name: itertools
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-06-22 18:18:48 +02:00
dependabot[bot]
840c283cac cargo: bump tempfile from 3.5.0 to 3.6.0
Bumps [tempfile](https://github.com/Stebalien/tempfile) from 3.5.0 to 3.6.0.
- [Changelog](https://github.com/Stebalien/tempfile/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Stebalien/tempfile/compare/v3.5.0...v3.6.0)

---
updated-dependencies:
- dependency-name: tempfile
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-06-08 22:53:55 -07:00
dependabot[bot]
b9f2f634be cargo: bump git2 from 0.17.1 to 0.17.2
Bumps [git2](https://github.com/rust-lang/git2-rs) from 0.17.1 to 0.17.2.
- [Changelog](https://github.com/rust-lang/git2-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/git2-rs/compare/0.17.1...0.17.2)

---
updated-dependencies:
- dependency-name: git2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-05-29 20:25:04 +02:00
dependabot[bot]
c7b66606a1 cargo: bump git2 from 0.17.0 to 0.17.1
Bumps [git2](https://github.com/rust-lang/git2-rs) from 0.17.0 to 0.17.1.
- [Release notes](https://github.com/rust-lang/git2-rs/releases)
- [Changelog](https://github.com/rust-lang/git2-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/git2-rs/compare/git2-curl-0.17.0...0.17.1)

---
updated-dependencies:
- dependency-name: git2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-17 09:19:58 -07:00
dependabot[bot]
fb27885c5d cargo: bump git2 from 0.16.1 to 0.17.0
Bumps [git2](https://github.com/rust-lang/git2-rs) from 0.16.1 to 0.17.0.
- [Release notes](https://github.com/rust-lang/git2-rs/releases)
- [Changelog](https://github.com/rust-lang/git2-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/git2-rs/compare/0.16.1...git2-curl-0.17.0)

---
updated-dependencies:
- dependency-name: git2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-03 09:24:05 -07:00
dependabot[bot]
0d581809fc cargo: bump tempfile from 3.4.0 to 3.5.0
Bumps [tempfile](https://github.com/Stebalien/tempfile) from 3.4.0 to 3.5.0.
- [Release notes](https://github.com/Stebalien/tempfile/releases)
- [Changelog](https://github.com/Stebalien/tempfile/blob/master/NEWS)
- [Commits](https://github.com/Stebalien/tempfile/commits)

---
updated-dependencies:
- dependency-name: tempfile
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-30 16:38:19 +00:00
dependabot[bot]
7de17ce4e7 cargo: bump tempfile from 3.3.0 to 3.4.0
Bumps [tempfile](https://github.com/Stebalien/tempfile) from 3.3.0 to 3.4.0.
- [Release notes](https://github.com/Stebalien/tempfile/releases)
- [Changelog](https://github.com/Stebalien/tempfile/blob/master/NEWS)
- [Commits](https://github.com/Stebalien/tempfile/commits)

---
updated-dependencies:
- dependency-name: tempfile
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-02-27 11:06:51 -08:00
Martin von Zweigbergk
eb79a21cc0 release: release version 0.7.0
It's been about 10 weeks and 730 commits since 0.6.0, compared to
about 7 weeks and 350 commits between 0.5.0 and 0.6.0, so it's time
for a new release. There's been significant user-visible changes and
code-quality improvements. Thanks, everyone!
2023-02-16 12:50:52 -08:00
dependabot[bot]
034a23f4ec cargo: bump git2 from 0.16.0 to 0.16.1
Bumps [git2](https://github.com/rust-lang/git2-rs) from 0.16.0 to 0.16.1.
- [Release notes](https://github.com/rust-lang/git2-rs/releases)
- [Commits](https://github.com/rust-lang/git2-rs/compare/git2-curl-0.16.0...0.16.1)

---
updated-dependencies:
- dependency-name: git2
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-01-21 00:05:57 +00:00
Michael Forster
27228ce292 Update MSRV to 1.61
This is needed for compatibility with the sapling dag crate.
2023-01-19 10:29:39 +01:00
dependabot[bot]
eeac6632a2 cargo: bump git2 from 0.15.0 to 0.16.0
Bumps [git2](https://github.com/rust-lang/git2-rs) from 0.15.0 to 0.16.0.
- [Release notes](https://github.com/rust-lang/git2-rs/releases)
- [Commits](https://github.com/rust-lang/git2-rs/compare/git2-curl-0.15.0...git2-curl-0.16.0)

---
updated-dependencies:
- dependency-name: git2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-01-12 09:28:02 -08:00
Martin von Zweigbergk
5caade2705 release: release version 0.6.1
The only changes is that we now depend on a released version of
`thrift`.
2022-12-05 20:50:48 -08:00
Martin von Zweigbergk
3c7c4e9f5c tests: move testutils module into separate crate
The `testutils` module should ideally not be part of the library
dependencies. Since they're used by the integration tests (and the CLI
tests), we need to move them to a separate crate to achieve that.
2022-11-08 07:29:35 -08:00