jj/docs/design/run.md

# Introducing JJ run

Authors: [Philip Metzger](mailto:philipmetzger@bluewin.ch), [Martin von Zweigberk](mailto:martinvonz@google.com), [Danny Hooper](mailto:hooper@google.com), [Waleed Khan](mailto:me@waleedkhan.name)

Initial Version, 10.12.2022 (view full history [here](https://docs.google.com/document/d/14BiAoEEy_e-BRPHYpXRFjvHMfgYVKh-pKWzzTDi-v-g/edit))


**Summary:** This Document documents the design of a new `run` command for 
Jujutsu which will be used to seamlessly integrate with build systems, linters
and formatters. This is achieved by running a user-provided command or script 
across multiple revisions. For more details, read the 
[Use-Cases of jj run](#Use-Cases-of-jj-run).

## Preface

The goal of this Design Document is to specify the correct behavior of `jj run`.
The points we decide on here I (Philip Metzger) will try to implement. There 
exists some prior work in other DVCS:
* `git test`: part of [git-branchless]. Similar to this proposal for `jj run`. 
* `hg run`: Google's internal Mercurial extension. Similar to this proposal for
`jj run`.
Details not available. 
* `hg fix`: Google's open source Mercurial extension: [source code][fix-src]. A
more specialized approach to rewriting file content without full context of the
working directory. 
* `git rebase -x`: runs commands opportunistically as part of rebase. 
* `git bisect run`: run a command to determine which commit introduced a bug.

## Context and Scope

The initial need for some kind of command runner integrated in the VCS, surfaced
in a [github discussion][pre-commit]. In a [discussion on discord][hooks] about
the git-hook model, there was consensus about not repeating their mistakes.

For `jj run` there is prior art in Mercurial, git branchless and Google's 
internal Mercurial. Currently git-branchless `git test` and `hg fix` implement
some kind of command runner. The Google internal `hg run` works in 
conjunction with CitC (Clients in the Cloud) which allows it to lazily apply
the current command to any affected file. Currently no Jujutsu backend
(Git, Native) has a fancy virtual filesystem supporting it, so we 
can't apply this optimization. We could do the same once we have an 
implementation of the working copy based on a virtual file system. Until then, 
we have to run the commands in regular local-disk working copies. 

## Goals and Non-Goals

### Goals

* We should be able to apply the command to any revision, published or unpublished.
* We should be able to parallelize running the actual command, while preserving a
good console output.
* The run command should be able to work in any commit, the working-copy commit
itself or any other commit. 
* There should exist some way to signal hard failure. 
* The command should build enough infrastructure for `jj test`, `jj fix` and 
`jj format`.
* The main goal is to be good enough, as we can always expand the functionality 
in the future.

### Non-Goals

* While we should build a base for `jj test`, `jj format` and `jj fix`, we 
shouldn't mash their use-cases into `jj run`.
* The command shouldn't be too smart, as too many assumptions about workflows 
makes the command confusing for users. 
* The smart caching of outputs, as user input commands can be unpredictable.
makes the command confusing for users. 
* Avoid the smart caching of outputs, as user input commands can be 
unpredictable.
* Fine grained user facing configuration, as it's unwarranted complexity.
* A `fix` subcommand as it cuts too much design space.

## Use-Cases of jj run

**Linting and Formatting:**

- `jj run 'pre-commit run' -r $revset`
- `jj run 'cargo clippy' -r $revset`
- `jj run 'cargo +nightly fmt'`

**Large scale changes across repositories, local and remote:**

- `jj run 'sed /some/test/' -r 'mine() & ~remote_branches(exact:"origin")'`
- `jj run '$rewrite-tool' -r '$revset'`

**Build systems:**

- `jj run 'bazel build //some/target:somewhere'`
- `jj run 'ninja check-lld'`

Some of these use-cases should get a specialized command, as this allows 
further optimization. A command could be `jj format`, which runs a list of 
formatters over a subset of a file in a revision. Another command could be 
`jj fix`, which runs a command like `rustfmt --fix` or `cargo clippy --fix` over
a subset of a file in a revision.

## Design

### Base Design 

All the work will be done in the `.jj/` directory. This allows us to hide all 
complexity from the users, while preserving the user's current workspace.

We will copy the approach from git-branchless's `git test` of creating a 
temporary working copy for each parallel command. The working copies will be 
reused between `jj run` invocations. They will also be reused within `jj run` 
invocation if there are more commits to run on than there are parallel jobs.

We will leave ignored files in the temporary directory between runs. That 
enables incremental builds (e.g by letting cargo reuse its `target/` directory).
However, it also means that runs potentially become less reproducible. We will 
provide a flag for removing ignored files from the temporary working copies to
address that. 

Another problem with leaving ignored files in the temporary directories is that
they take up space. That is especially problematic in the case of cargo (the 
`target/` directory often takes up tens of GBs). The same flag for cleaning up
ignored files can be used to address that. We may want to also have a flag for 
cleaning up temporary working copies *after* running the command. 

An early version of the command will directly use [Treestate] to 
to manage the temporary working copies. That means that running `jj` inside the 
temporary working copies will not work . We can later extend that to use a full
[Workspace]. To prevent operations in the working copies from 
impacting the repo, we can use a separate [OpHeadsStore] for it.

### Modifying the Working Copy

Since the subprocesses will run in temporary working copies, they 
won't interfere with the user's working copy. The user can therefore continue
to work in it while `jj run` is running. 

We want subprocesses to be able to make changes to the repo by updating their
assigned working copy. Let's say the user runs `jj run` on just commits A and 
B, where B's parent is A. Any changes made on top of A would be squashed into 
A, forming A'. Similarly B' would be formed by squasing it into B. We can then
either do a normal rebase of B' onto A', or we can simply update its parent to
A'. The former is useful, e.g when the subprocess only makes a partial update
of the tree based on the parent commit. In addition to these two modes, we may 
want to have an option to ignore any changes made in the subprocess's working 
copy.

### Modifying the Repo

Once we give the subprocess access to a fork of the repo via separate 
[OpHeadsStore], it will be able to create new operations in its fork.
If the user runs `jj run -r foo` and the subprocess checks out another commit,
it's not clear what that should do. We should probably just verify that the 
working-copy commit's parents are unchanged after the subprocess returns. Any
operations created by the subprocess will be ignored. 

### Rewriting the revisions 

Like all commands, `jj run` will refuse to rewrite public/immutable commits.
For private/unpublished revisions, we either amend or reparent the changes, 
which are available as command options.

## Execution order/parallelism

It may be useful to execute commands in topological order. For example, 
commands with costs proportional to incremental changes, like build systems. 
There may also be other revelant heuristics, but topological order is an easy
and effective way to start. 

Parallel execution of commands on different commits may choose to schedule 
commits to still reduce incremental changes in the working copy used by each
execution slot/"thread". However, running the command on all commits 
concurrently should be possible if desired. 

Executing commands in topological order allows for more meaningful use of any 
potential features that stop execution "at the first failure". For example, 
when running tests on a chain of commits, it might be useful to proceed in 
topological/chronological order, and stop on the first failure, because it 
might imply that the remaining executions will be undesirable because they will
also fail.

## Dealing with failure

It will be useful to have multiple strategies to deal with failures on a single
or multiple revisions. The reason for these strategies is to allow customized
conflict handling. These strategies then can be exposed in the ui with a 
matching option.

**Continue:** If any subprocess fails, we will continue the work on child 
revisions. Notify the user on exit about the failed revisions. 

**Stop:** Signal a fatal failure and cancel any scheduled work that has not
yet started running, but let any already started subprocess finish. Notify the
user about the failed command and display the generated error from the 
subprocess. 

**Fatal:** Signal a fatal failure and immediately stop processing and kill any 
running processes. Notify the user that we failed to apply the command to the 
specific revision. 

We will leave any affected commit in its current state, if any subprocess fails.
This allows us provide a better user experience, as leaving revisions in an 
undesirable state, e.g partially formatted, may confuse users.

## Resource constraints

It will be useful to constrain the execution to prevent resource exhaustion. 
Relevant resources could include:
- CPU and memory available on the machine running the commands. `jj run` can
provide some simple mitigations like limiting parallelism to "number of CPUs" 
by default, and limiting parallelism by dividing "available memory" by some 
estimate or measurement of per-invocation memory use of the commands.
- External resources that are not immediately known to jj. For example, 
commands run in parallel may wish to limit the total number of connections
to a server. We might choose to defer any handling of this to the 
implementation of the command being invoked, instead of trying to 
communicate that information to jj.


## Command Options

The base command of any jj command should be usable. By default `jj run` works 
on the `@` the current working copy.
* --command, explicit name of the first argument
* -x, for git compatibility (may alias another command)
* -j, --jobs, the amount of parallelism to use
* -k, --keep-going, continue on failure (may alias another command)
* --show, display the diff for an affected revision
* --dry-run, do the command execution without doing any work, logging all 
intended files and arguments
* --rebase, rebase all parents on the consulitng diff (may alias another 
command)
* --reparent, change the parent of an effected revision to the new change 
(may alias another command)
* --clean, remove existing workspaces and remove the ignored files
* --readonly, ignore changes across multiple run invocations
* --error-strategy=`continue|stop|fatal`, see [Dealing with failure](#Dealing-with-failure)

### Integrating with other commands

`jj log`: No special handling needed
`jj diff`: No special handling needed
`jj st`: For now reprint the final output of `jj run`
`jj op log`: No special handling needed, but awaits further discussion in 
[#963][issue]
`jj undo/jj op undo`: No special handling needed


## Open Points

Should the command be working copy backend specific?  
How do we manage the Processes which the command will spawn?  
Configuration options, User and Repository Wide?

## Future possibilities

- We could rewrite the file in memory, which is a neat optimization  
- Exposing some internal state, to allow preciser resource constraints  
- Integration options for virtual filesystems, which allow them to cache the 
needed working copies.  
- A Jujutsu wide concept for a cached working copy, as they could be expensive
to materialize.  
- Customized failure messages, this maybe useful for bots, it could be similar 
to Bazel's `select(..., message = "arch not supported for $project")`.
- Make `jj run` asynchronous by spawning a `main` process, directly return to the
user and incrementally updating the output of `jj st`. 



[git-branchless]: https://github.com/arxanas/git-branchless
[issue]: https://github.com/martinvonz/jj/issues/963 
[fix-src]: https://repo.mercurial-scm.org/hg/file/tip/hgext/fix.py
[hooks]: https://discord.com/channels/968932220549103686/969829516539228222/1047958933161119795
[OpHeadsStore]: https://github.com/martinvonz/jj/blob/main/lib/src/op_heads_store.rs
[pre-commit]: https://github.com/martinvonz/jj/issues/405
[Treestate]: https://github.com/martinvonz/jj/blob/af85f552b676d66ed0e9ae0d401cd0c4ffbbeb21/lib/src/working_copy.rs#L117
[Workspace]: https://github.com/martinvonz/jj/blob/af85f552b676d66ed0e9ae0d401cd0c4ffbbeb21/lib/src/workspace.rs#L54