<p>Authors: <ahref="mailto:philipmetzger@bluewin.ch">Philip Metzger</a>, <ahref="mailto:martinvonz@google.com">Martin von Zweigberk</a>, <ahref="mailto:hooper@google.com">Danny Hooper</a>, <ahref="mailto:me@waleedkhan.name">Waleed Khan</a></p>
<p>Initial Version, 10.12.2022 (view full history <ahref="https://docs.google.com/document/d/14BiAoEEy_e-BRPHYpXRFjvHMfgYVKh-pKWzzTDi-v-g/edit">here</a>)</p>
<p><strong>Summary:</strong> This Document documents the design of a new <code>run</code> command for
Jujutsu which will be used to seamlessly integrate with build systems, linters
and formatters. This is achieved by running a user-provided command or script
across multiple revisions. For more details, read the
<ahref="#Use-Cases-of-jj-run">Use-Cases of jj run</a>.</p>
<p>The goal of this Design Document is to specify the correct behavior of <code>jj run</code>.
The points we decide on here I (Philip Metzger) will try to implement. There
exists some prior work in other DVCS:</p>
<ul>
<li><code>git test</code>: part of <ahref="https://github.com/arxanas/git-branchless">git-branchless</a>. Similar to this proposal for <code>jj run</code>. </li>
<li><code>hg run</code>: Google's internal Mercurial extension. Similar to this proposal for
<code>jj run</code>.
Details not available. </li>
<li><code>hg fix</code>: Google's open source Mercurial extension: <ahref="https://repo.mercurial-scm.org/hg/file/tip/hgext/fix.py">source code</a>. A
more specialized approach to rewriting file content without full context of the
working directory. </li>
<li><code>git rebase -x</code>: runs commands opportunistically as part of rebase. </li>
<li><code>git bisect run</code>: run a command to determine which commit introduced a bug.</li>
</ul>
<h2id="context-and-scope">Context and Scope<aclass="headerlink"href="#context-and-scope"title="Permanent link">¶</a></h2>
<p>The initial need for some kind of command runner integrated in the VCS, surfaced
in a <ahref="https://github.com/martinvonz/jj/issues/405">github discussion</a>. In a <ahref="https://discord.com/channels/968932220549103686/969829516539228222/1047958933161119795">discussion on discord</a> about
the git-hook model, there was consensus about not repeating their mistakes.</p>
<p>For <code>jj run</code> there is prior art in Mercurial, git branchless and Google's
internal Mercurial. Currently git-branchless <code>git test</code> and <code>hg fix</code> implement
some kind of command runner. The Google internal <code>hg run</code> works in
conjunction with CitC (Clients in the Cloud) which allows it to lazily apply
the current command to any affected file. Currently no Jujutsu backend
(Git, Native) has a fancy virtual filesystem supporting it, so we
can't apply this optimization. We could do the same once we have an
implementation of the working copy based on a virtual file system. Until then,
we have to run the commands in regular local-disk working copies. </p>
<h2id="goals-and-non-goals">Goals and Non-Goals<aclass="headerlink"href="#goals-and-non-goals"title="Permanent link">¶</a></h2>
<p>All the work will be done in the <code>.jj/</code> directory. This allows us to hide all
complexity from the users, while preserving the user's current workspace.</p>
<p>We will copy the approach from git-branchless's <code>git test</code> of creating a
temporary working copy for each parallel command. The working copies will be
reused between <code>jj run</code> invocations. They will also be reused within <code>jj run</code>
invocation if there are more commits to run on than there are parallel jobs.</p>
<p>We will leave ignored files in the temporary directory between runs. That
enables incremental builds (e.g by letting cargo reuse its <code>target/</code> directory).
However, it also means that runs potentially become less reproducible. We will
provide a flag for removing ignored files from the temporary working copies to
address that. </p>
<p>Another problem with leaving ignored files in the temporary directories is that
they take up space. That is especially problematic in the case of cargo (the
<code>target/</code> directory often takes up tens of GBs). The same flag for cleaning up
ignored files can be used to address that. We may want to also have a flag for
cleaning up temporary working copies <em>after</em> running the command. </p>
<p>An early version of the command will directly use <ahref="https://github.com/martinvonz/jj/blob/af85f552b676d66ed0e9ae0d401cd0c4ffbbeb21/lib/src/working_copy.rs#L117">Treestate</a> to
to manage the temporary working copies. That means that running <code>jj</code> inside the
temporary working copies will not work . We can later extend that to use a full
<ahref="https://github.com/martinvonz/jj/blob/af85f552b676d66ed0e9ae0d401cd0c4ffbbeb21/lib/src/workspace.rs#L54">Workspace</a>. To prevent operations in the working copies from
impacting the repo, we can use a separate <ahref="https://github.com/martinvonz/jj/blob/main/lib/src/op_heads_store.rs">OpHeadsStore</a> for it.</p>
<h3id="modifying-the-working-copy">Modifying the Working Copy<aclass="headerlink"href="#modifying-the-working-copy"title="Permanent link">¶</a></h3>
<p>Since the subprocesses will run in temporary working copies, they
won't interfere with the user's working copy. The user can therefore continue
to work in it while <code>jj run</code> is running. </p>
<p>We want subprocesses to be able to make changes to the repo by updating their
assigned working copy. Let's say the user runs <code>jj run</code> on just commits A and
B, where B's parent is A. Any changes made on top of A would be squashed into
A, forming A'. Similarly B' would be formed by squasing it into B. We can then
either do a normal rebase of B' onto A', or we can simply update its parent to
A'. The former is useful, e.g when the subprocess only makes a partial update
of the tree based on the parent commit. In addition to these two modes, we may
want to have an option to ignore any changes made in the subprocess's working
copy.</p>
<h3id="modifying-the-repo">Modifying the Repo<aclass="headerlink"href="#modifying-the-repo"title="Permanent link">¶</a></h3>
<p>Once we give the subprocess access to a fork of the repo via separate
<ahref="https://github.com/martinvonz/jj/blob/main/lib/src/op_heads_store.rs">OpHeadsStore</a>, it will be able to create new operations in its fork.
If the user runs <code>jj run -r foo</code> and the subprocess checks out another commit,
it's not clear what that should do. We should probably just verify that the
working-copy commit's parents are unchanged after the subprocess returns. Any
operations created by the subprocess will be ignored. </p>
<h3id="rewriting-the-revisions">Rewriting the revisions<aclass="headerlink"href="#rewriting-the-revisions"title="Permanent link">¶</a></h3>
<p>Like all commands, <code>jj run</code> will refuse to rewrite public/immutable commits.
For private/unpublished revisions, we either amend or reparent the changes,
<p>The base command of any jj command should be usable. By default <code>jj run</code> works
on the <code>@</code> the current working copy.</p>
<ul>
<li>--command, explicit name of the first argument</li>
<li>-x, for git compatibility (may alias another command)</li>
<li>-j, --jobs, the amount of parallelism to use</li>
<li>-k, --keep-going, continue on failure (may alias another command)</li>
<li>--show, display the diff for an affected revision</li>
<li>--dry-run, do the command execution without doing any work, logging all
intended files and arguments</li>
<li>--rebase, rebase all parents on the consulitng diff (may alias another
command)</li>
<li>--reparent, change the parent of an effected revision to the new change
(may alias another command)</li>
<li>--clean, remove existing workspaces and remove the ignored files</li>
<li>--readonly, ignore changes across multiple run invocations</li>
<li>--error-strategy=<code>continue|stop|fatal</code>, see <ahref="#Dealing-with-failure">Dealing with failure</a></li>
</ul>
<h3id="integrating-with-other-commands">Integrating with other commands<aclass="headerlink"href="#integrating-with-other-commands"title="Permanent link">¶</a></h3>
<p><code>jj log</code>: No special handling needed
<code>jj diff</code>: No special handling needed
<code>jj st</code>: For now reprint the final output of <code>jj run</code>
<code>jj op log</code>: No special handling needed, but awaits further discussion in
<scriptid="__config"type="application/json">{"base":"../..","features":[],"search":"../../assets/javascripts/workers/search.f886a092.min.js","translations":{"clipboard.copied":"Copied to clipboard","clipboard.copy":"Copy to clipboard","search.result.more.one":"1 more on this page","search.result.more.other":"# more on this page","search.result.none":"No matching documents","search.result.one":"1 matching document","search.result.other":"# matching documents","search.result.placeholder":"Type to start searching","search.result.term.missing":"Missing","select.version":"Select version"},"version":{"provider":"mike"}}</script>
