Closes
https://linear.app/zed-industries/issue/Z-352/high-memory-usage-from-fs-scanning-if-project-contains-symlinks-that
### Background
Currently, when you open a project, Zed eagerly scans the directory,
building an in-memory representation of all of the files and directories
within. This scanning includes all git-ignored files and follows any
symlinks. When any directory changes on disk, Zed recursively rescans it
in order to keep its in-memory representation up-to-date. When
collaborating, all of these files are replicated to all guests.
Right now, there are some performance problems associated with the
maintenance of this filesystem state:
* For various reasons, some projects contain symlinks that point out to
large folders like `$HOME`, which itself contains many symlinks that
point to the same large directory. When these projects are opened, the
worktree scans endlessly, using more and more memory.
* Some git-ignored directories (like `target` in a rust project) contain
*many* more files than are actually tracked in the git repository. These
files often change as a result of saving, (e.g. because the compiler
runs). Maintaining in memory all of these paths isn't useful to the
user, and causes significant CPU usage on every save. Most importantly,
when collaborating sending all of these changes to guests can be slow,
and can delay all other RPC messages.
### Change
This PR changes the worktree's filesystem-scanning logic to be *lazy*
about scanning two types of directories:
* git ignored directories
* "external" directories (those that are canonically located outside of
the worktree root, but accessed via symlinks)
The laziness works as follows. When, during a recursive scan, a
directory is found that falls into one of the above 2 categories, that
directory is marked as "unloaded". The directory might later be scanned,
if some explicit operation is performed within it (like opening a
buffer, or creating a file), if any collaborator expands that directory
in their project panel, or if an LSP requests that it be watched.
### Results
When collaborating on the `zed` folder:
| metric | before | after |
|-------|--------|------|
| # `worktree_entries` in collab db initially | 154,763 | 77,679 |
| # `worktree_entries` in collab db after 5 saves | 181,952 | 77,679
(nothing new to scan) |
| app memory footprint (host) | 260MB | 228.5 MB |
The db thing is a win, because reading and writing to the
`worktree_entries` table is one of the most expensive thing that the
`collab` server does.
There's also generally lower background CPU usage after every save,
because we don't need to recursively rescan directories inside of
`target`.
### Limitations
We still end up scanning some unnecessary directories (like
`target/debug/build/zed-b612db829aeac16e/out`) because the LSP instructs
us to watch those.
### To do:
* [x] Expand parent directories of any path opened via LSP
* [x] Avoid creating orphaned entries when FS events happen inside of
unscanned directories
* [x] Scan any newly-non-ignored directories after gitignore changes
* [x] Emit correct events for newly-discovered paths when expanding dirs
* [x] GC the set of expanded directory ids when dirs are removed
* [x] Don't include "external" entries in file-finder
* [x] Expand any directories watched by LSP
* [ ] manual testing and profiling
### Release Notes:
- Fixed a bug where Zed would use excessive memory when a project folder
contained symlinks pointing to directories outside of the project.
- Reduced Zed's memory and CPU usage when working in folders containing
many git-ignored files.
After #2641 we noticed that scrolling didn't take a count parameter, and
a few other issues with the way that we calculated the distance to
scroll.
Release Notes:
- Improved distance calculations for page-up/page-down
- vim: Allow counts to work with scrolling shortcuts.
Fixes: zed-industries/community#1690
I'm not sure this is the correct way to fix this...
* A simpler approach would be to just say `!showing_code_actions` in the
binding file (as `showing_completions` can only happen in insert mode -
and `VimControl` will be false). This seemed a little error prone if
more menus were added in the future.
* A more complicated approach would be to copy the way this is done from
the MouseContextMenu, which registers its own keyboard shortcuts, and as
such uses those when it's open. This seems "more correct", but is a
major refactoring for a very small reward.
Release Notes:
- vim: Fix code actions menu
([#1690](https://github.com/zed-industries/community/issues/1690))
After #2641 we noticed that scrolling didn't take a count parameter.
The PageDown/PageUp logic was also broken by an additional -1 (for both
vim mode and not).
The test was testing pretty straightforward logic, but for some strange reason
it was failing on CI (but passed locally). I think it's fine to delete it and
make progress, if zooming regresses we'll find out pretty quickly.
To turn any struct into a composite element, you can implement a render method
with the following signature:
fn render<V: View>(&mut self, view: &mut V, cx: &mut ViewContext<V>) -> AnyElement<V>;
Then add #[derive(Element)] to the struct definition.
This will make it easier to introduce higher-level components that are expressed in
terms of other elements.
These adapters have indicated some broader reason to the user why
they cannot be started, don't waste time/bandwidth attempting to
validate and reinstall them
Before this change code could not distinguish between a user providing a
count of 1 and no count at all.
Fixes: zed-industries/community#710
Release Notes:
- 1G now correctly goes to the first line in vim mode
([710](zed-industries/community#710))
Don't wait until populating that directory entry, for two reasons:
* In the case of submodules, .git is not a directory
* We don't eagerly populate .git directories, since their contents
are automatically ignored.
Release Notes:
- In vim mode you can now use count modifiers with arrow keys (`5<down>`
will go down 5 lines).
- In vim mode `ctrl+]` and `ctrl+o` now work to go to definition and
back
🚧 We have a couple more refinements to this to add 🚧
Release Notes:
You can now join lines with `ctrl-j` or `shift-J` in Vim normal mode.
🍐'd with @ConradIrwin
This is a part of the intensity driven theme rewrite.
It introduces the `toggle` and `interactive` helper functions to build
Toggle<T> and Interactive<T> styles for interactive elements in the
theme.
This PR also removes the `theme_testbench` crate and related actions.
Huge thanks to @osiewicz and @mikayla-maki for pushing this forward 🙏🏽
Release Notes:
- Updated the style of many interactive elements.
Improves latency for big inlay hints LSP responses for ~8k line files.
Before, the CPU usage sample for editing a single line inside
`edirot.rs` file in Zed contained serde inside the main thread traces:
<img width="1728" alt="Screenshot 2023-06-21 at 00 33 23"
src="https://github.com/zed-industries/zed/assets/2690773/d9789efe-8055-487f-bbe7-8beb49605bcb">
Release Notes:
- N/A
Since we don't want tabs, I think it would be better to render the toolbar
for ourselves directly and handle switching between conversations.
Co-Authored-By: Julia Risley <julia@zed.dev>
Fix mislocation of caller query in detach_and_log_error
Fix incorrect wording on livekit integration
Add share_mic action for manually enabling the microphone
Make mic sharing wait until the room has been fully established
This avoids a high cost which appears to be the system rasterizing the
cursor every time we call this, fixes a slowdown when scrolling rapidly
while mouse motion continually attempted to assign the style
Co-Authored-By: Antonio Scandurra <me@as-cii.com>
Z-2357
I've found a crate that handles both comments and trailing commas in
JSON. It is a fork of `serde_json`, but it is maintained & up-to-date.
Sadly RawValue seems to not play nicely with it; I've ran into
deserialisation issues around use of RawValue. For this PR I've migrated
to `Value` API.
Obviously this is just a point of discussion, not something I'd merge
straight away. There may be better solutions to this particular problem.
I've also noticed that `serde_json_lenient` does not handle trailing
commas after bindings array. I'm not sure how big of an issue that is.
Release Notes:
- Improved handling of trailing commas in settings files.
[#1322](https://github.com/zed-industries/community/issues/1322)
