mirrors/salsa
1
0
Fork 0
mirror of https://github.com/salsa-rs/salsa.git synced 2025-01-15 01:39:25 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 2

Merge branch 'salsa-rs:master' into immutable_fields_in_inputs

Browse source
This commit is contained in:
Onigbinde Oluwamuyiwa Elijah 2022-09-16 01:53:42 +01:00 committed by GitHub
commit e3661de899
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
47 changed files with 433 additions and 167 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

3
Cargo.toml
View file

@ -35,6 +35,7 @@ members = [
"components/salsa-macros",
"components/salsa-2022",
"components/salsa-2022-macros",
"calc-example/calc",
"examples-2022/calc",
"examples-2022/lazy-input",
"salsa-2022-tests",
]

58
book/src/common_patterns/on_demand_inputs.md
View file

@ -1,51 +1,37 @@
# On-Demand (Lazy) Inputs
Salsa input queries work best if you can easily provide all of the inputs upfront.
{{#include ../caveat.md}}
Salsa inputs work best if you can easily provide all of the inputs upfront.
However sometimes the set of inputs is not known beforehand.
A typical example is reading files from disk.
While it is possible to eagerly scan a particular directory and create an in-memory file tree in a salsa input query, a more straight-forward approach is to read the files lazily.
That is, when someone requests the text of a file for the first time:
While it is possible to eagerly scan a particular directory and create an in-memory file tree as salsa input structs, a more straight-forward approach is to read the files lazily.
That is, when a query requests the text of a file for the first time:
1. Read the file from disk and cache it.
2. Setup a file-system watcher for this path.
3. Invalidate the cached file once the watcher sends a change notification.
3. Update the cached file when the watcher sends a change notification.
This is possible to achieve in salsa, by caching the inputs in your database structs and adding a method to the database trait to retrieve them out of this cache.
A complete, runnable file-watching example can be found in [the lazy-input example](https://github.com/salsa-rs/salsa/tree/master/examples-2022/lazy-input).
This is possible to achieve in salsa, using a derived query and `report_synthetic_read` and `invalidate` queries.
The setup looks roughly like this:
```rust,ignore
#[salsa::query_group(VfsDatabaseStorage)]
trait VfsDatabase: salsa::Database + FileWatcher {
fn read(&self, path: PathBuf) -> String;
}
trait FileWatcher {
fn watch(&self, path: &Path);
fn did_change_file(&mut self, path: &Path);
}
fn read(db: &dyn VfsDatabase, path: PathBuf) -> String {
db.salsa_runtime()
.report_synthetic_read(salsa::Durability::LOW);
db.watch(&path);
std::fs::read_to_string(&path).unwrap_or_default()
}
#[salsa::database(VfsDatabaseStorage)]
struct MyDatabase { ... }
impl FileWatcher for MyDatabase {
fn watch(&self, path: &Path) { ... }
fn did_change_file(&mut self, path: &Path) {
ReadQuery.in_db_mut(self).invalidate(path);
}
}
{{#include ../../../examples-2022/lazy-input/src/main.rs:db}}
```
- We declare the query as a derived query (which is the default).
- In the query implementation, we don't call any other query and just directly read file from disk.
- Because the query doesn't read any inputs, it will be assigned a `HIGH` durability by default, which we override with `report_synthetic_read`.
- The result of the query is cached, and we must call `invalidate` to clear this cache.
- We declare a method on the `Db` trait that gives us a `File` input on-demand (it only requires a `&dyn Db` not a `&mut dyn Db`).
- There should only be one input struct per file, so we implement that method using a cache (`DashMap` is like a `RwLock<HashMap>`).
A complete, runnable file-watching example can be found in [this git repo](https://github.com/ChristopherBiscardi/salsa-file-watch-example/blob/f968dc8ea13a90373f91d962f173de3fe6ae24cd/main.rs) along with [a write-up](https://www.christopherbiscardi.com/on-demand-lazy-inputs-for-incremental-computation-in-salsa-with-file-watching-powered-by-notify-in-rust) that explains more about the code and what it is doing.
The driving code that's doing the top-level queries is then in charge of updating the file contents when a file-change notification arrives.
It does this by updating the Salsa input in the same way that you would update any other input.
Here we implement a simple driving loop, that recompiles the code whenever a file changes.
You can use the logs to check that only the queries that could have changed are re-evaluated.
```rust,ignore
{{#include ../../../examples-2022/lazy-input/src/main.rs:main}}
```

4
book/src/tutorial/accumulators.md
View file

@ -10,7 +10,7 @@ Salsa defines a mechanism for managing this called an **accumulator**.
In our case, we define an accumulator struct called `Diagnostics` in the `ir` module:
```rust
{{#include ../../../calc-example/calc/src/ir.rs:diagnostic}}
{{#include ../../../examples-2022/calc/src/ir.rs:diagnostic}}
```
Accumulator structs are always newtype structs with a single field, in this case of type `Diagnostic`.
@ -22,7 +22,7 @@ or any functions that they called
The `Parser::report_error` method contains an example of pushing a diagnostic:
```rust
{{#include ../../../calc-example/calc/src/parser.rs:report_error}}
{{#include ../../../examples-2022/calc/src/parser.rs:report_error}}
```
To get the set of diagnostics produced by `parse_errors`, or any other memoized function,

10
book/src/tutorial/db.md
View file

@ -7,10 +7,10 @@ the one which starts up the program, supplies the inputs, and relays the outputs
In `calc`, the database struct is in the [`db`] module, and it looks like this:
[`db`]: https://github.com/salsa-rs/salsa/blob/master/calc-example/calc/src/db.rs
[`db`]: https://github.com/salsa-rs/salsa/blob/master/examples-2022/calc/src/db.rs
```rust
{{#include ../../../calc-example/calc/src/db.rs:db_struct}}
{{#include ../../../examples-2022/calc/src/db.rs:db_struct}}
```
The `#[salsa::db(...)]` attribute takes a list of all the jars to include.
@ -24,7 +24,7 @@ The `salsa::db` attribute autogenerates a bunch of impls for things like the `sa
In addition to the struct itself, we must add an impl of `salsa::Database`:
```rust
{{#include ../../../calc-example/calc/src/db.rs:db_impl}}
{{#include ../../../examples-2022/calc/src/db.rs:db_impl}}
```
## Implementing the `salsa::ParallelDatabase` trait
@ -32,7 +32,7 @@ In addition to the struct itself, we must add an impl of `salsa::Database`:
If you want to permit accessing your database from multiple threads at once, then you also need to implement the `ParallelDatabase` trait:
```rust
{{#include ../../../calc-example/calc/src/db.rs:par_db_impl}}
{{#include ../../../examples-2022/calc/src/db.rs:par_db_impl}}
```
## Implementing the `Default` trait
@ -40,7 +40,7 @@ If you want to permit accessing your database from multiple threads at once, the
It's not required, but implementing the `Default` trait is often a convenient way to let users instantiate your database:
```rust
{{#include ../../../calc-example/calc/src/db.rs:default_impl}}
{{#include ../../../examples-2022/calc/src/db.rs:default_impl}}
```
## Implementing the traits for each jar

6
book/src/tutorial/debug.md
View file

@ -25,7 +25,7 @@ The `DebugWithDb` trait is automatically derived for all `#[input]`, `#[interned
For consistency, it is sometimes useful to have a `DebugWithDb` implementation even for types, like `Op`, that are just ordinary enums. You can do that like so:
```rust
{{#include ../../../calc-example/calc/src/ir.rs:op_debug_impl}}
{{#include ../../../examples-2022/calc/src/ir.rs:op_debug_impl}}
```
## Writing the unit test
@ -34,11 +34,11 @@ Now that we have our `DebugWithDb` impls in place, we can write a simple unit te
The `parse_string` function below creates a database, sets the source text, and then invokes the parser:
```rust
{{#include ../../../calc-example/calc/src/parser.rs:parse_string}}
{{#include ../../../examples-2022/calc/src/parser.rs:parse_string}}
```
Combined with the [`expect-test`](https://crates.io/crates/expect-test) crate, we can then write unit tests like this one:
```rust
{{#include ../../../calc-example/calc/src/parser.rs:parse_print}}
{{#include ../../../examples-2022/calc/src/parser.rs:parse_print}}
```

10
book/src/tutorial/ir.md
View file

@ -31,7 +31,7 @@ such that when those inputs change, we can try to efficiently recompute the new
Inputs are defined as Rust structs with a `#[salsa::input]` annotation:
```rust
{{#include ../../../calc-example/calc/src/ir.rs:input}}
{{#include ../../../examples-2022/calc/src/ir.rs:input}}
```
In our compiler, we have just one simple input, the `SourceProgram`, which has a `text` field (the string).
@ -77,7 +77,7 @@ Whereas inputs represent the *start* of a computation, tracked structs represent
In this case, the parser is going to take in the `SourceProgram` struct that we saw and return a `Program` that represents the fully parsed program:
```rust
{{#include ../../../calc-example/calc/src/ir.rs:program}}
{{#include ../../../examples-2022/calc/src/ir.rs:program}}
```
Like with an input, the fields of a tracked struct are also stored in the database.
@ -98,7 +98,7 @@ We will also use a tracked struct to represent each function:
The `Function` struct is going to be created by the parser to represent each of the functions defined by the user:
```rust
{{#include ../../../calc-example/calc/src/ir.rs:functions}}
{{#include ../../../examples-2022/calc/src/ir.rs:functions}}
```
If we had created some `Function` instance `f`, for example, we might find that `the f.body` field changes
@ -131,7 +131,7 @@ it's also inefficient to have to compare them for equality via string comparison
Therefore, we define two interned structs, `FunctionId` and `VariableId`, each with a single field that stores the string:
```rust
{{#include ../../../calc-example/calc/src/ir.rs:interned_ids}}
{{#include ../../../examples-2022/calc/src/ir.rs:interned_ids}}
```
When you invoke e.g. `FunctionId::new(&db, "my_string".to_string())`, you will get back a `FunctionId` that is just a newtype'd integer.
@ -158,7 +158,7 @@ If, however, you export interned identifiers outside the computation, and then c
We won't use any special "Salsa structs" for expressions and statements:
```rust
{{#include ../../../calc-example/calc/src/ir.rs:statements_and_expressions}}
{{#include ../../../examples-2022/calc/src/ir.rs:statements_and_expressions}}
```
Since statements and expressions are not tracked, this implies that we are only attempting to get incremental re-use at the granularity of functions --

6
book/src/tutorial/jar.md
View file

@ -19,7 +19,7 @@ This permits the crates to define private functions and other things that are me
To define a jar struct, you create a tuple struct with the `#[salsa::jar]` annotation:
```rust
{{#include ../../../calc-example/calc/src/main.rs:jar_struct}}
{{#include ../../../examples-2022/calc/src/main.rs:jar_struct}}
```
Although it's not required, it's highly recommended to put the `jar` struct at the root of your crate, so that it can be referred to as `crate::Jar`.
@ -36,7 +36,7 @@ This allows for separate compilation, where you have a database that contains th
The database trait for our `calc` crate is very simple:
```rust
{{#include ../../../calc-example/calc/src/main.rs:jar_db}}
{{#include ../../../examples-2022/calc/src/main.rs:jar_db}}
```
When you define a database trait like `Db`, the one thing that is required is that it must have a supertrait `salsa::DbWithJar<Jar>`,
@ -57,7 +57,7 @@ a common choice is to write a blanket impl for any type that implements `DbWithJ
and that's what we do here:
```rust
{{#include ../../../calc-example/calc/src/main.rs:jar_db_impl}}
{{#include ../../../examples-2022/calc/src/main.rs:jar_db_impl}}
```
## Summary

2
book/src/tutorial/parser.md
View file

@ -17,7 +17,7 @@ We're going to focus only on the Salsa-related aspects.
The starting point for the parser is the `parse_statements` function:
```rust
{{#include ../../../calc-example/calc/src/parser.rs:parse_statements}}
{{#include ../../../examples-2022/calc/src/parser.rs:parse_statements}}
```
This function is annotated as `#[salsa::tracked]`.

10
components/salsa-2022-macros/src/input.rs
View file

@ -136,20 +136,20 @@ impl InputStruct {
let __ingredients = <#jar_ty as salsa::storage::HasIngredientsFor< #ident >>::ingredient_mut(__jar);
let __id = __ingredients.#input_index.new_singleton_input(__runtime);
#(
__ingredients.#field_indices.store(__runtime, __id, #field_names, salsa::Durability::LOW);
__ingredients.#field_indices.store_mut(__runtime, __id, #field_names, salsa::Durability::LOW);
)*
__id
}
}
} else {
parse_quote! {
pub fn #constructor_name(__db: &mut #db_dyn_ty, #(#field_names: #field_tys,)*) -> Self
pub fn #constructor_name(__db: &#db_dyn_ty, #(#field_names: #field_tys,)*) -> Self
{
let (__jar, __runtime) = <_ as salsa::storage::HasJar<#jar_ty>>::jar_mut(__db);
let __ingredients = <#jar_ty as salsa::storage::HasIngredientsFor< #ident >>::ingredient_mut(__jar);
let (__jar, __runtime) = <_ as salsa::storage::HasJar<#jar_ty>>::jar(__db);
let __ingredients = <#jar_ty as salsa::storage::HasIngredientsFor< #ident >>::ingredient(__jar);
let __id = __ingredients.#input_index.new_input(__runtime);
#(
__ingredients.#field_indices.store(__runtime, __id, #field_names, salsa::Durability::LOW);
__ingredients.#field_indices.store_new(__runtime, __id, #field_names, salsa::Durability::LOW);
)*
__id
}

27
components/salsa-2022/src/input.rs
View file

@ -1,4 +1,7 @@
use std::fmt;
use std::{
fmt,
sync::atomic::{AtomicU32, Ordering},
};
use crate::{
cycle::CycleRecoveryStrategy,
@ -16,7 +19,7 @@ where
Id: InputId,
{
ingredient_index: IngredientIndex,
counter: u32,
counter: AtomicU32,
debug_name: &'static str,
_phantom: std::marker::PhantomData<Id>,
}
@ -41,24 +44,20 @@ where
}
}
pub fn new_input(&mut self, _runtime: &mut Runtime) -> Id {
let next_id = self.counter;
self.counter += 1;
pub fn new_input(&self, _runtime: &Runtime) -> Id {
let next_id = self.counter.fetch_add(1, Ordering::Relaxed);
Id::from_id(crate::Id::from_u32(next_id))
}
pub fn new_singleton_input(&mut self, _runtime: &mut Runtime) -> Id {
if self.counter >= 1 {
// already exists
Id::from_id(crate::Id::from_u32(self.counter - 1))
} else {
self.new_input(_runtime)
}
pub fn new_singleton_input(&mut self, _runtime: &Runtime) -> Id {
// There's only one singleton so record that we've created it
// and return the only id.
self.counter.store(1, Ordering::Relaxed);
Id::from_id(crate::Id::from_u32(0))
}
pub fn get_singleton_input(&self, _runtime: &Runtime) -> Option<Id> {
(self.counter > 0)
.then(|| Id::from_id(crate::Id::from_id(crate::Id::from_u32(self.counter - 1))))
(self.counter.load(Ordering::Relaxed) > 0).then(|| Id::from_id(crate::Id::from_u32(0)))
}
}

70
components/salsa-2022/src/input_field.rs
View file

@ -4,18 +4,23 @@ use crate::key::DependencyIndex;
use crate::runtime::local_state::QueryOrigin;
use crate::runtime::StampedValue;
use crate::{AsId, DatabaseKeyIndex, Durability, Id, IngredientIndex, Revision, Runtime};
use rustc_hash::FxHashMap;
use dashmap::mapref::entry::Entry;
use dashmap::DashMap;
use std::fmt;
use std::hash::Hash;
/// Ingredient used to represent the fields of a `#[salsa::input]`.
/// These fields can only be mutated by an explicit call to a setter
/// with an `&mut` reference to the database,
/// and therefore cannot be mutated during a tracked function or in parallel.
/// This makes the implementation considerably simpler.
///
/// These fields can only be mutated by a call to a setter with an `&mut`
/// reference to the database, and therefore cannot be mutated during a tracked
/// function or in parallel.
/// However for on-demand inputs to work the fields must be able to be set via
/// a shared reference, so some locking is required.
/// Altogether this makes the implementation somewhat simpler than tracked
/// structs.
pub struct InputFieldIngredient<K, F> {
index: IngredientIndex,
map: FxHashMap<K, StampedValue<F>>,
map: DashMap<K, Box<StampedValue<F>>>,
debug_name: &'static str,
}
@ -31,33 +36,52 @@ where
}
}
pub fn store(
pub fn store_mut(
&mut self,
runtime: &mut Runtime,
runtime: &Runtime,
key: K,
value: F,
durability: Durability,
) -> Option<F> {
let revision = runtime.current_revision();
let stamped_value = StampedValue {
let stamped_value = Box::new(StampedValue {
value,
durability,
changed_at: revision,
};
});
if let Some(old_value) = self.map.insert(key, stamped_value) {
Some(old_value.value)
} else {
None
self.map
.insert(key, stamped_value)
.map(|old_value| old_value.value)
}
/// Set the field of a new input.
///
/// This function panics if the field has ever been set before.
pub fn store_new(&self, runtime: &Runtime, key: K, value: F, durability: Durability) {
let revision = runtime.current_revision();
let stamped_value = Box::new(StampedValue {
value,
durability,
changed_at: revision,
});
match self.map.entry(key) {
Entry::Occupied(_) => {
panic!("attempted to set field of existing input using `store_new`, use `store_mut` instead");
}
Entry::Vacant(entry) => {
entry.insert(stamped_value);
}
}
}
pub fn fetch(&self, runtime: &Runtime, key: K) -> &F {
pub fn fetch<'db>(&'db self, runtime: &'db Runtime, key: K) -> &F {
let StampedValue {
value,
durability,
changed_at,
} = self.map.get(&key).unwrap();
} = &**self.map.get(&key).unwrap();
runtime.report_tracked_read(
self.database_key_index(key).into(),
@ -65,7 +89,11 @@ where
*changed_at,
);
value
// SAFETY:
// The value is stored in a box so internal moves in the dashmap don't
// invalidate the reference to the value inside the box.
// Values are only removed or altered when we have `&mut self`.
unsafe { transmute_lifetime(self, value) }
}
fn database_key_index(&self, key: K) -> DatabaseKeyIndex {
@ -76,6 +104,14 @@ where
}
}
// Returns `u` but with the lifetime of `t`.
//
// Safe if you know that data at `u` will remain shared
// until the reference `t` expires.
unsafe fn transmute_lifetime<'t, 'u, T, U>(_t: &'t T, u: &'u U) -> &'t U {
std::mem::transmute(u)
}
impl<DB: ?Sized, K, F> Ingredient<DB> for InputFieldIngredient<K, F>
where
K: AsId,

2
components/salsa-2022/src/setter.rs
View file

@ -33,7 +33,7 @@ where
pub fn to(self, value: F) -> F {
self.ingredient
.store(self.runtime, self.key, value, self.durability)
.store_mut(self.runtime, self.key, value, self.durability)
.unwrap()
}
}

0
calc-example/calc/Cargo.toml → examples-2022/calc/Cargo.toml
View file

0
calc-example/calc/src/compile.rs → examples-2022/calc/src/compile.rs
View file

0
calc-example/calc/src/db.rs → examples-2022/calc/src/db.rs
View file

0
calc-example/calc/src/ir.rs → examples-2022/calc/src/ir.rs
View file

4
calc-example/calc/src/main.rs → examples-2022/calc/src/main.rs
View file

@ -33,8 +33,8 @@ mod parser;
mod type_check;
pub fn main() {
let mut db = db::Database::default();
let source_program = SourceProgram::new(&mut db, String::new());
let db = db::Database::default();
let source_program = SourceProgram::new(&db, String::new());
compile::compile(&db, source_program);
let diagnostics = compile::compile::accumulated::<Diagnostics>(&db, source_program);
eprintln!("{diagnostics:?}");

4
calc-example/calc/src/parser.rs → examples-2022/calc/src/parser.rs
View file

@ -355,10 +355,10 @@ fn parse_string(source_text: &str) -> String {
use salsa::debug::DebugWithDb;
// Create the database
let mut db = crate::db::Database::default();
let db = crate::db::Database::default();
// Create the source program
let source_program = SourceProgram::new(&mut db, source_text.to_string());
let source_program = SourceProgram::new(&db, source_text.to_string());
// Invoke the parser
let statements = parse_statements(&db, source_program);

2
calc-example/calc/src/type_check.rs → examples-2022/calc/src/type_check.rs
View file

@ -97,7 +97,7 @@ fn check_string(
let mut db = Database::default().enable_logging();
// Create the source program
let source_program = SourceProgram::new(&mut db, source_text.to_string());
let source_program = SourceProgram::new(&db, source_text.to_string());
// Invoke the parser
let program = parse_statements(&db, source_program);

14
examples-2022/lazy-input/Cargo.toml Normal file
View file

@ -0,0 +1,14 @@
[package]
name = "lazy-input"
version = "0.1.0"
edition = "2021"
[dependencies]
crossbeam-channel = "0.5.6"
dashmap = "5.4.0"
eyre = "0.6.8"
notify-debouncer-mini = "0.2.1"
salsa = { path = "../../components/salsa-2022", package = "salsa-2022" }
[dev-dependencies]
expect-test = "1.4.0"

2
examples-2022/lazy-input/inputs/a Normal file
View file

@ -0,0 +1,2 @@
2
./aa

1
examples-2022/lazy-input/inputs/aa Normal file
View file

@ -0,0 +1 @@
8

1
examples-2022/lazy-input/inputs/b Normal file
View file

@ -0,0 +1 @@
4

3
examples-2022/lazy-input/inputs/start Normal file
View file

@ -0,0 +1,3 @@
1
./a
./b

225
examples-2022/lazy-input/src/main.rs Normal file
View file

@ -0,0 +1,225 @@
use std::{path::PathBuf, sync::Mutex, time::Duration};
use crossbeam_channel::{unbounded, Sender};
use dashmap::{mapref::entry::Entry, DashMap};
use eyre::{eyre, Context, Report, Result};
use notify_debouncer_mini::{
new_debouncer,
notify::{RecommendedWatcher, RecursiveMode},
DebounceEventResult, Debouncer,
};
use salsa::DebugWithDb;
// ANCHOR: main
fn main() -> Result<()> {
// Create the channel to receive file change events.
let (tx, rx) = unbounded();
let mut db = Database::new(tx);
let initial_file_path = std::env::args_os()
.nth(1)
.ok_or_else(|| eyre!("Usage: ./lazy-input <input-file>"))?;
// Create the initial input using the input method so that changes to it
// will be watched like the other files.
let initial = db.input(initial_file_path.into())?;
loop {
// Compile the code starting at the provided input, this will read other
// needed files using the on-demand mechanism.
let sum = compile(&db, initial);
let diagnostics = compile::accumulated::<Diagnostic>(&db, initial);
if diagnostics.is_empty() {
println!("Sum is: {}", sum);
} else {
for diagnostic in diagnostics {
println!("{}", diagnostic);
}
}
for log in db.logs.lock().unwrap().drain(..) {
eprintln!("{}", log);
}
// Wait for file change events, the output can't change unless the
// inputs change.
for event in rx.recv()?.unwrap() {
let path = event.path.canonicalize().wrap_err_with(|| {
format!("Failed to canonicalize path {}", event.path.display())
})?;
let file = match db.files.get(&path) {
Some(file) => *file,
None => continue,
};
// `path` has changed, so read it and update the contents to match.
// This creates a new revision and causes the incremental algorithm
// to kick in, just like any other update to a salsa input.
let contents = std::fs::read_to_string(path)
.wrap_err_with(|| format!("Failed to read file {}", event.path.display()))?;
file.set_contents(&mut db).to(contents);
}
}
}
// ANCHOR_END: main
#[salsa::jar(db = Db)]
struct Jar(Diagnostic, File, ParsedFile, compile, parse, sum);
// ANCHOR: db
#[salsa::input]
struct File {
path: PathBuf,
#[return_ref]
contents: String,
}
trait Db: salsa::DbWithJar<Jar> {
fn input(&self, path: PathBuf) -> Result<File>;
}
#[salsa::db(Jar)]
struct Database {
storage: salsa::Storage<Self>,
logs: Mutex<Vec<String>>,
files: DashMap<PathBuf, File>,
file_watcher: Mutex<Debouncer<RecommendedWatcher>>,
}
impl Database {
fn new(tx: Sender<DebounceEventResult>) -> Self {
let storage = Default::default();
Self {
storage,
logs: Default::default(),
files: DashMap::new(),
file_watcher: Mutex::new(new_debouncer(Duration::from_secs(1), None, tx).unwrap()),
}
}
}
impl Db for Database {
fn input(&self, path: PathBuf) -> Result<File> {
let path = path
.canonicalize()
.wrap_err_with(|| format!("Failed to read {}", path.display()))?;
Ok(match self.files.entry(path.clone()) {
// If the file already exists in our cache then just return it.
Entry::Occupied(entry) => *entry.get(),
// If we haven't read this file yet set up the watch, read the
// contents, store it in the cache, and return it.
Entry::Vacant(entry) => {
// Set up the watch before reading the contents to try to avoid
// race conditions.
let watcher = &mut *self.file_watcher.lock().unwrap();
watcher
.watcher()
.watch(&path, RecursiveMode::NonRecursive)
.unwrap();
let contents = std::fs::read_to_string(&path)
.wrap_err_with(|| format!("Failed to read {}", path.display()))?;
*entry.insert(File::new(self, path, contents))
}
})
}
}
// ANCHOR_END: db
impl salsa::Database for Database {
fn salsa_event(&self, event: salsa::Event) {
// don't log boring events
if let salsa::EventKind::WillExecute { .. } = event.kind {
self.logs
.lock()
.unwrap()
.push(format!("{:?}", event.debug(self)));
}
}
}
#[salsa::accumulator]
struct Diagnostic(String);
impl Diagnostic {
fn push_error(db: &dyn Db, file: File, error: Report) {
Diagnostic::push(
db,
format!(
"Error in file {}: {:?}\n",
file.path(db)
.file_name()
.unwrap_or_else(|| "<unknown>".as_ref())
.to_string_lossy(),
error,
),
)
}
}
#[salsa::tracked]
struct ParsedFile {
value: u32,
#[return_ref]
links: Vec<ParsedFile>,
}
#[salsa::tracked]
fn compile(db: &dyn Db, input: File) -> u32 {
let parsed = parse(db, input);
sum(db, parsed)
}
#[salsa::tracked]
fn parse(db: &dyn Db, input: File) -> ParsedFile {
let mut lines = input.contents(db).lines();
let value = match lines.next().map(|line| (line.parse::<u32>(), line)) {
Some((Ok(num), _)) => num,
Some((Err(e), line)) => {
Diagnostic::push_error(
db,
input,
Report::new(e).wrap_err(format!(
"First line ({}) could not be parsed as an integer",
line
)),
);
0
}
None => {
Diagnostic::push_error(db, input, eyre!("File must contain an integer"));
0
}
};
let links = lines
.filter_map(|path| {
let relative_path = match path.parse::<PathBuf>() {
Ok(path) => path,
Err(err) => {
Diagnostic::push_error(
db,
input,
Report::new(err).wrap_err(format!("Failed to parse path: {}", path)),
);
return None;
}
};
let link_path = input.path(db).parent().unwrap().join(relative_path);
match db.input(link_path) {
Ok(file) => Some(parse(db, file)),
Err(err) => {
Diagnostic::push_error(db, input, err);
None
}
}
})
.collect();
ParsedFile::new(db, value, links)
}
#[salsa::tracked]
fn sum(db: &dyn Db, input: ParsedFile) -> u32 {
input.value(db)
+ input
.links(db)
.iter()
.map(|&file| sum(db, file))
.sum::<u32>()
}

4
salsa-2022-tests/tests/accumulate-from-tracked-fn.rs
View file

@ -65,8 +65,8 @@ impl HasLogger for Database {
fn test1() {
let mut db = Database::default();
let l0 = List::new(&mut db, 1, None);
let l1 = List::new(&mut db, 10, Some(l0));
let l0 = List::new(&db, 1, None);
let l1 = List::new(&db, 10, Some(l0));
compute(&db, l1);
expect![[r#"

4
salsa-2022-tests/tests/accumulate-reuse-workaround.rs
View file

@ -69,8 +69,8 @@ impl HasLogger for Database {
fn test1() {
let mut db = Database::default();
let l1 = List::new(&mut db, 1, None);
let l2 = List::new(&mut db, 2, Some(l1));
let l1 = List::new(&db, 1, None);
let l2 = List::new(&db, 2, Some(l1));
assert_eq!(compute(&db, l2), 2);
db.assert_logs(expect![[r#"

4
salsa-2022-tests/tests/accumulate-reuse.rs
View file

@ -64,8 +64,8 @@ impl HasLogger for Database {
fn test1() {
let mut db = Database::default();
let l1 = List::new(&mut db, 1, None);
let l2 = List::new(&mut db, 2, Some(l1));
let l1 = List::new(&db, 1, None);
let l2 = List::new(&db, 2, Some(l1));
assert_eq!(compute(&db, l2), 2);
db.assert_logs(expect![[r#"

6
salsa-2022-tests/tests/accumulate.rs
View file

@ -85,7 +85,7 @@ fn accumulate_once() {
let mut db = Database::default();
// Just call accumulate on a base input to see what happens.
let input = MyInput::new(&mut db, 2, 3);
let input = MyInput::new(&db, 2, 3);
let logs = push_logs::accumulated::<Logs>(&db, input);
expect![[r#"
[
@ -109,7 +109,7 @@ fn change_a_and_reaccumulate() {
let mut db = Database::default();
// Accumulate logs for `a = 2` and `b = 3`
let input = MyInput::new(&mut db, 2, 3);
let input = MyInput::new(&db, 2, 3);
let logs = push_logs::accumulated::<Logs>(&db, input);
expect![[r#"
[
@ -148,7 +148,7 @@ fn get_a_logs_after_changing_b() {
let mut db = Database::default();
// Invoke `push_a_logs` with `a = 2` and `b = 3` (but `b` doesn't matter)
let input = MyInput::new(&mut db, 2, 3);
let input = MyInput::new(&db, 2, 3);
let logs = push_a_logs::accumulated::<Logs>(&db, input);
expect![[r#"
[

26
salsa-2022-tests/tests/cycles.rs
View file

@ -183,7 +183,7 @@ fn extract_cycle(f: impl FnOnce() + UnwindSafe) -> salsa::Cycle {
#[test]
fn cycle_memoized() {
let mut db = Database::default();
let input = MyInput::new(&mut db);
let input = MyInput::new(&db);
let cycle = extract_cycle(|| memoized_a(&db, input));
let expected = expect![[r#"
[
@ -197,7 +197,7 @@ fn cycle_memoized() {
#[test]
fn cycle_volatile() {
let mut db = Database::default();
let input = MyInput::new(&mut db);
let input = MyInput::new(&db);
let cycle = extract_cycle(|| volatile_a(&db, input));
let expected = expect![[r#"
[
@ -215,7 +215,7 @@ fn expect_cycle() {
// +-----+
let mut db = Database::default();
let abc = ABC::new(&mut db, CycleQuery::B, CycleQuery::A, CycleQuery::None);
let abc = ABC::new(&db, CycleQuery::B, CycleQuery::A, CycleQuery::None);
assert!(cycle_a(&db, abc).is_err());
}
@ -225,7 +225,7 @@ fn inner_cycle() {
// ^ |
// +-----+
let mut db = Database::default();
let abc = ABC::new(&mut db, CycleQuery::B, CycleQuery::A, CycleQuery::B);
let abc = ABC::new(&db, CycleQuery::B, CycleQuery::A, CycleQuery::B);
let err = cycle_c(&db, abc);
assert!(err.is_err());
let expected = expect![[r#"
@ -243,7 +243,7 @@ fn cycle_revalidate() {
// ^ |
// +-----+
let mut db = Database::default();
let abc = ABC::new(&mut db, CycleQuery::B, CycleQuery::A, CycleQuery::None);
let abc = ABC::new(&db, CycleQuery::B, CycleQuery::A, CycleQuery::None);
assert!(cycle_a(&db, abc).is_err());
abc.set_b(&mut db).to(CycleQuery::A); // same value as default
assert!(cycle_a(&db, abc).is_err());
@ -255,7 +255,7 @@ fn cycle_recovery_unchanged_twice() {
// ^ |
// +-----+
let mut db = Database::default();
let abc = ABC::new(&mut db, CycleQuery::B, CycleQuery::A, CycleQuery::None);
let abc = ABC::new(&db, CycleQuery::B, CycleQuery::A, CycleQuery::None);
assert!(cycle_a(&db, abc).is_err());
abc.set_c(&mut db).to(CycleQuery::A); // force new revision
@ -267,7 +267,7 @@ fn cycle_appears() {
let mut db = Database::default();
// A --> B
let abc = ABC::new(&mut db, CycleQuery::B, CycleQuery::None, CycleQuery::None);
let abc = ABC::new(&db, CycleQuery::B, CycleQuery::None, CycleQuery::None);
assert!(cycle_a(&db, abc).is_ok());
// A --> B
@ -284,7 +284,7 @@ fn cycle_disappears() {
// A --> B
// ^ |
// +-----+
let abc = ABC::new(&mut db, CycleQuery::B, CycleQuery::A, CycleQuery::None);
let abc = ABC::new(&db, CycleQuery::B, CycleQuery::A, CycleQuery::None);
assert!(cycle_a(&db, abc).is_err());
// A --> B
@ -334,7 +334,7 @@ fn cycle_mixed_1() {
// A --> B <-- C
// | ^
// +-----+
let abc = ABC::new(&mut db, CycleQuery::B, CycleQuery::C, CycleQuery::B);
let abc = ABC::new(&db, CycleQuery::B, CycleQuery::C, CycleQuery::B);
let expected = expect![[r#"
[
@ -354,7 +354,7 @@ fn cycle_mixed_2() {
// A --> B --> C
// ^ |
// +-----------+
let abc = ABC::new(&mut db, CycleQuery::B, CycleQuery::C, CycleQuery::A);
let abc = ABC::new(&db, CycleQuery::B, CycleQuery::C, CycleQuery::A);
let expected = expect![[r#"
[
"cycle_a(0)",
@ -374,7 +374,7 @@ fn cycle_deterministic_order() {
// A --> B
// ^ |
// +-----+
let abc = ABC::new(&mut db, CycleQuery::B, CycleQuery::A, CycleQuery::None);
let abc = ABC::new(&db, CycleQuery::B, CycleQuery::A, CycleQuery::None);
(db, abc)
};
let (db, abc) = f();
@ -411,7 +411,7 @@ fn cycle_multiple() {
//
// Here, conceptually, B encounters a cycle with A and then
// recovers.
let abc = ABC::new(&mut db, CycleQuery::B, CycleQuery::AthenC, CycleQuery::A);
let abc = ABC::new(&db, CycleQuery::B, CycleQuery::AthenC, CycleQuery::A);
let c = cycle_c(&db, abc);
let b = cycle_b(&db, abc);
@ -446,7 +446,7 @@ fn cycle_recovery_set_but_not_participating() {
// A --> C -+
// ^ |
// +--+
let abc = ABC::new(&mut db, CycleQuery::C, CycleQuery::None, CycleQuery::C);
let abc = ABC::new(&db, CycleQuery::C, CycleQuery::None, CycleQuery::C);
// Here we expect C to panic and A not to recover:
let r = extract_cycle(|| drop(cycle_a(&db, abc)));

6
salsa-2022-tests/tests/debug.rs
View file

@ -36,13 +36,13 @@ impl Db for Database {}
#[test]
fn input() {
let mut db = Database::default();
let db = Database::default();
let input = MyInput::new(&mut db, 22);
let input = MyInput::new(&db, 22);
let not_salsa = NotSalsa {
field: "it's salsa time".to_string(),
};
let complex_struct = ComplexStruct::new(&mut db, input, not_salsa);
let complex_struct = ComplexStruct::new(&db, input, not_salsa);
// default debug only includes identity fields
let actual = format!("{:?}", complex_struct.debug(&db));

2
salsa-2022-tests/tests/deletion.rs
View file

@ -84,7 +84,7 @@ fn basic() {
let mut db = Database::default();
// Creates 3 tracked structs
let input = MyInput::new(&mut db, 3);
let input = MyInput::new(&db, 3);
assert_eq!(final_result(&db, input), 2 * 2 + 2);
db.assert_logs(expect![[r#"
[

2
salsa-2022-tests/tests/expect_reuse_field_x_of_a_tracked_struct_changes_but_fn_depends_on_field_y.rs
View file

@ -74,7 +74,7 @@ fn execute() {
// intermediate results:
// x = (22 + 1) / 2 = 11
// y = 22 / 2 = 11
let input = MyInput::new(&mut db, 22);
let input = MyInput::new(&db, 22);
assert_eq!(final_result_depends_on_x(&db, input), 22);
db.assert_logs(expect![[r#"
[

2
salsa-2022-tests/tests/expect_reuse_field_x_of_an_input_changes_but_fn_depends_on_field_y.rs
View file

@ -53,7 +53,7 @@ fn execute() {
// result_depends_on_y = y - 1
let mut db = Database::default();
let input = MyInput::new(&mut db, 22, 33);
let input = MyInput::new(&db, 22, 33);
assert_eq!(result_depends_on_x(&db, input), 23);
db.assert_logs(expect![[r#"
[

6
salsa-2022-tests/tests/hello_world.rs
View file

@ -54,7 +54,7 @@ impl HasLogger for Database {
fn execute() {
let mut db = Database::default();
let input = MyInput::new(&mut db, 22);
let input = MyInput::new(&db, 22);
assert_eq!(final_result(&db, input), 22);
db.assert_logs(expect![[r#"
[
@ -85,7 +85,7 @@ fn execute() {
fn red_herring() {
let mut db = Database::default();
let input = MyInput::new(&mut db, 22);
let input = MyInput::new(&db, 22);
assert_eq!(final_result(&db, input), 22);
db.assert_logs(expect![[r#"
[
@ -96,7 +96,7 @@ fn red_herring() {
// Create a distinct input and mutate it.
// This will trigger a new revision in the database
// but shouldn't actually invalidate our existing ones.
let input2 = MyInput::new(&mut db, 44);
let input2 = MyInput::new(&db, 44);
input2.set_field(&mut db).to(66);
// Re-run the query on the original input. Nothing re-executes!

24
salsa-2022-tests/tests/lru.rs
View file

@ -82,28 +82,26 @@ fn load_n_potatoes() -> usize {
#[test]
fn lru_works() {
let mut db = DatabaseImpl::default();
let db = DatabaseImpl::default();
assert_eq!(load_n_potatoes(), 0);
for i in 0..128u32 {
let input = MyInput::new(&mut db, i);
let input = MyInput::new(&db, i);
let p = get_hot_potato(&db, input);
assert_eq!(p.0, i)
}
// Create a new input to change the revision, and trigger the GC
MyInput::new(&mut db, 0);
MyInput::new(&db, 0);
assert_eq!(load_n_potatoes(), 32);
}
#[test]
fn lru_doesnt_break_volatile_queries() {
let mut db = DatabaseImpl::default();
let db = DatabaseImpl::default();
// Create all inputs first, so that there are no revision changes among calls to `get_volatile`
let inputs: Vec<MyInput> = (0..128usize)
.map(|i| MyInput::new(&mut db, i as u32))
.collect();
let inputs: Vec<MyInput> = (0..128usize).map(|i| MyInput::new(&db, i as u32)).collect();
// Here, we check that we execute each volatile query at most once, despite
// LRU. That does mean that we have more values in DB than the LRU capacity,
@ -118,10 +116,10 @@ fn lru_doesnt_break_volatile_queries() {
#[test]
fn lru_can_be_changed_at_runtime() {
let mut db = DatabaseImpl::default();
let db = DatabaseImpl::default();
assert_eq!(load_n_potatoes(), 0);
let inputs: Vec<(u32, MyInput)> = (0..128).map(|i| (i, MyInput::new(&mut db, i))).collect();
let inputs: Vec<(u32, MyInput)> = (0..128).map(|i| (i, MyInput::new(&db, i))).collect();
for &(i, input) in inputs.iter() {
let p = get_hot_potato(&db, input);
@ -129,7 +127,7 @@ fn lru_can_be_changed_at_runtime() {
}
// Create a new input to change the revision, and trigger the GC
MyInput::new(&mut db, 0);
MyInput::new(&db, 0);
assert_eq!(load_n_potatoes(), 32);
get_hot_potato::set_lru_capacity(&db, 64);
@ -140,7 +138,7 @@ fn lru_can_be_changed_at_runtime() {
}
// Create a new input to change the revision, and trigger the GC
MyInput::new(&mut db, 0);
MyInput::new(&db, 0);
assert_eq!(load_n_potatoes(), 64);
// Special case: setting capacity to zero disables LRU
@ -152,7 +150,7 @@ fn lru_can_be_changed_at_runtime() {
}
// Create a new input to change the revision, and trigger the GC
MyInput::new(&mut db, 0);
MyInput::new(&db, 0);
assert_eq!(load_n_potatoes(), 128);
drop(db);
@ -167,7 +165,7 @@ fn lru_keeps_dependency_info() {
// Invoke `get_hot_potato2` 33 times. This will (in turn) invoke
// `get_hot_potato`, which will trigger LRU after 32 executions.
let inputs: Vec<MyInput> = (0..(capacity + 1))
.map(|i| MyInput::new(&mut db, i as u32))
.map(|i| MyInput::new(&db, i as u32))
.collect();
for (i, input) in inputs.iter().enumerate() {

2
salsa-2022-tests/tests/mutate_in_place.rs
View file

@ -36,7 +36,7 @@ impl HasLogger for Database {
fn execute() {
let mut db = Database::default();
let input = MyInput::new(&mut db, "Hello".to_string());
let input = MyInput::new(&db, "Hello".to_string());
// Overwrite field with an empty String
// and store the old value in my_string

4
salsa-2022-tests/tests/parallel/parallel_cycle_all_recover.rs
View file

@ -92,10 +92,10 @@ fn recover_b2(db: &dyn Db, _cycle: &salsa::Cycle, key: MyInput) -> i32 {
#[test]
fn execute() {
let mut db = Database::default();
let db = Database::default();
db.knobs().signal_on_will_block.set(3);
let input = MyInput::new(&mut db, 1);
let input = MyInput::new(&db, 1);
let thread_a = std::thread::spawn({
let db = db.snapshot();

4
salsa-2022-tests/tests/parallel/parallel_cycle_mid_recover.rs
View file

@ -87,10 +87,10 @@ fn recover_b3(db: &dyn Db, _cycle: &salsa::Cycle, key: MyInput) -> i32 {
#[test]
fn execute() {
let mut db = Database::default();
let db = Database::default();
db.knobs().signal_on_will_block.set(3);
let input = MyInput::new(&mut db, 1);
let input = MyInput::new(&db, 1);
let thread_a = std::thread::spawn({
let db = db.snapshot();

4
salsa-2022-tests/tests/parallel/parallel_cycle_none_recover.rs
View file

@ -43,10 +43,10 @@ pub(crate) fn b(db: &dyn Db, input: MyInput) -> i32 {
#[test]
fn execute() {
let mut db = Database::default();
let db = Database::default();
db.knobs().signal_on_will_block.set(3);
let input = MyInput::new(&mut db, -1);
let input = MyInput::new(&db, -1);
let thread_a = std::thread::spawn({
let db = db.snapshot();

4
salsa-2022-tests/tests/parallel/parallel_cycle_one_recover.rs
View file

@ -76,10 +76,10 @@ pub(crate) fn b2(db: &dyn Db, input: MyInput) -> i32 {
#[test]
fn execute() {
let mut db = Database::default();
let db = Database::default();
db.knobs().signal_on_will_block.set(3);
let input = MyInput::new(&mut db, 1);
let input = MyInput::new(&db, 1);
let thread_a = std::thread::spawn({
let db = db.snapshot();

2
salsa-2022-tests/tests/specify-only-works-if-the-key-is-created-in-the-current-query.rs
View file

@ -56,6 +56,6 @@ impl Db for Database {}
#[should_panic]
fn execute_when_specified() {
let mut db = Database::default();
let input = MyInput::new(&mut db, 22);
let input = MyInput::new(&db, 22);
let tracked = tracked_fn(&db, input);
}

14
salsa-2022-tests/tests/specify_tracked_fn_in_rev_1_but_not_2.rs
View file

@ -89,7 +89,7 @@ impl HasLogger for Database {
fn test_run_0() {
let mut db = Database::default();
let input = MyInput::new(&mut db, 0);
let input = MyInput::new(&db, 0);
assert_eq!(final_result(&db, input), 100);
db.assert_logs(expect![[r#"
[
@ -110,7 +110,7 @@ fn test_run_0() {
fn test_run_5() {
let mut db = Database::default();
let input = MyInput::new(&mut db, 5);
let input = MyInput::new(&db, 5);
assert_eq!(final_result(&db, input), 100);
db.assert_logs(expect![[r#"
[
@ -131,7 +131,7 @@ fn test_run_5() {
fn test_run_10() {
let mut db = Database::default();
let input = MyInput::new(&mut db, 10);
let input = MyInput::new(&db, 10);
assert_eq!(final_result(&db, input), 100);
db.assert_logs(expect![[r#"
[
@ -155,7 +155,7 @@ fn test_run_10() {
fn test_run_20() {
let mut db = Database::default();
let input = MyInput::new(&mut db, 20);
let input = MyInput::new(&db, 20);
assert_eq!(final_result(&db, input), 200);
db.assert_logs(expect![[r#"
[
@ -183,7 +183,7 @@ fn test_run_0_then_5_then_20() {
//
// * `create_tracked` specifies `10` for `maybe_specified`
// * final resuilt of `100` is derived by executing `read_maybe_specified`
let input = MyInput::new(&mut db, 0);
let input = MyInput::new(&db, 0);
assert_eq!(final_result(&db, input), 100);
db.assert_logs(expect![[r#"
[
@ -254,7 +254,7 @@ fn test_run_0_then_5_then_10_then_20() {
//
// * `create_tracked` specifies `10` for `maybe_specified`
// * final resuilt of `100` is derived by executing `read_maybe_specified`
let input = MyInput::new(&mut db, 0);
let input = MyInput::new(&db, 0);
assert_eq!(final_result(&db, input), 100);
db.assert_logs(expect![[r#"
[
@ -342,7 +342,7 @@ fn test_run_0_then_5_then_10_then_20() {
fn test_run_5_then_20() {
let mut db = Database::default();
let input = MyInput::new(&mut db, 5);
let input = MyInput::new(&db, 5);
assert_eq!(final_result(&db, input), 100);
db.assert_logs(expect![[r#"
[

2
salsa-2022-tests/tests/tracked_fn_on_input.rs
View file

@ -30,6 +30,6 @@ fn execute() {
impl Db for Database {}
let mut db = Database::default();
let input = MyInput::new(&mut db, 22);
let input = MyInput::new(&db, 22);
assert_eq!(tracked_fn(&db, input), 44);
}

4
salsa-2022-tests/tests/tracked_fn_on_tracked.rs
View file

@ -33,7 +33,7 @@ impl Db for Database {}
#[test]
fn execute() {
let mut db = Database::default();
let input = MyInput::new(&mut db, 22);
let db = Database::default();
let input = MyInput::new(&db, 22);
assert_eq!(tracked_fn(&db, input).field(&db), 44);
}

4
salsa-2022-tests/tests/tracked_fn_on_tracked_specify.rs
View file

@ -44,7 +44,7 @@ impl Db for Database {}
#[test]
fn execute_when_specified() {
let mut db = Database::default();
let input = MyInput::new(&mut db, 22);
let input = MyInput::new(&db, 22);
let tracked = tracked_fn(&db, input);
assert_eq!(tracked.field(&db), 44);
assert_eq!(tracked_fn_extra(&db, tracked), 2222);
@ -53,7 +53,7 @@ fn execute_when_specified() {
#[test]
fn execute_when_not_specified() {
let mut db = Database::default();
let input = MyInput::new(&mut db, 0);
let input = MyInput::new(&db, 0);
let tracked = tracked_fn(&db, input);
assert_eq!(tracked.field(&db), 0);
assert_eq!(tracked_fn_extra(&db, tracked), 0);

6
salsa-2022-tests/tests/tracked_fn_read_own_entity.rs
View file

@ -55,7 +55,7 @@ impl HasLogger for Database {
fn one_entity() {
let mut db = Database::default();
let input = MyInput::new(&mut db, 22);
let input = MyInput::new(&db, 22);
assert_eq!(final_result(&db, input), 22);
db.assert_logs(expect![[r#"
[
@ -86,7 +86,7 @@ fn one_entity() {
fn red_herring() {
let mut db = Database::default();
let input = MyInput::new(&mut db, 22);
let input = MyInput::new(&db, 22);
assert_eq!(final_result(&db, input), 22);
db.assert_logs(expect![[r#"
[
@ -97,7 +97,7 @@ fn red_herring() {
// Create a distinct input and mutate it.
// This will trigger a new revision in the database
// but shouldn't actually invalidate our existing ones.
let input2 = MyInput::new(&mut db, 44);
let input2 = MyInput::new(&db, 44);
input2.set_field(&mut db).to(66);
// Re-run the query on the original input. Nothing re-executes!