mirror of
https://github.com/zed-industries/zed.git
synced 2025-02-09 12:01:58 +00:00
Code at the speed of thought – Zed is a high-performance, multiplayer code editor from the creators of Atom and Tree-sitter.
f128cf4a33
Closes https://linear.app/zed-industries/issue/Z-352/high-memory-usage-from-fs-scanning-if-project-contains-symlinks-that ### Background Currently, when you open a project, Zed eagerly scans the directory, building an in-memory representation of all of the files and directories within. This scanning includes all git-ignored files and follows any symlinks. When any directory changes on disk, Zed recursively rescans it in order to keep its in-memory representation up-to-date. When collaborating, all of these files are replicated to all guests. Right now, there are some performance problems associated with the maintenance of this filesystem state: * For various reasons, some projects contain symlinks that point out to large folders like `$HOME`, which itself contains many symlinks that point to the same large directory. When these projects are opened, the worktree scans endlessly, using more and more memory. * Some git-ignored directories (like `target` in a rust project) contain *many* more files than are actually tracked in the git repository. These files often change as a result of saving, (e.g. because the compiler runs). Maintaining in memory all of these paths isn't useful to the user, and causes significant CPU usage on every save. Most importantly, when collaborating sending all of these changes to guests can be slow, and can delay all other RPC messages. ### Change This PR changes the worktree's filesystem-scanning logic to be *lazy* about scanning two types of directories: * git ignored directories * "external" directories (those that are canonically located outside of the worktree root, but accessed via symlinks) The laziness works as follows. When, during a recursive scan, a directory is found that falls into one of the above 2 categories, that directory is marked as "unloaded". The directory might later be scanned, if some explicit operation is performed within it (like opening a buffer, or creating a file), if any collaborator expands that directory in their project panel, or if an LSP requests that it be watched. ### Results When collaborating on the `zed` folder: | metric | before | after | |-------|--------|------| | # `worktree_entries` in collab db initially | 154,763 | 77,679 | | # `worktree_entries` in collab db after 5 saves | 181,952 | 77,679 (nothing new to scan) | | app memory footprint (host) | 260MB | 228.5 MB | The db thing is a win, because reading and writing to the `worktree_entries` table is one of the most expensive thing that the `collab` server does. There's also generally lower background CPU usage after every save, because we don't need to recursively rescan directories inside of `target`. ### Limitations We still end up scanning some unnecessary directories (like `target/debug/build/zed-b612db829aeac16e/out`) because the LSP instructs us to watch those. ### To do: * [x] Expand parent directories of any path opened via LSP * [x] Avoid creating orphaned entries when FS events happen inside of unscanned directories * [x] Scan any newly-non-ignored directories after gitignore changes * [x] Emit correct events for newly-discovered paths when expanding dirs * [x] GC the set of expanded directory ids when dirs are removed * [x] Don't include "external" entries in file-finder * [x] Expand any directories watched by LSP * [ ] manual testing and profiling ### Release Notes: - Fixed a bug where Zed would use excessive memory when a project folder contained symlinks pointing to directories outside of the project. - Reduced Zed's memory and CPU usage when working in folders containing many git-ignored files. |
||
|---|---|---|
| .cargo | ||
| .github | ||
| .vscode | ||
| assets | ||
| crates | ||
| docs | ||
| plugins | ||
| script | ||
| styles | ||
| .dockerignore | ||
| .gitignore | ||
| .gitmodules | ||
| Cargo.lock | ||
| Cargo.toml | ||
| Dockerfile | ||
| Procfile | ||
| README.md | ||
| rust-toolchain.toml | ||
Zed
Welcome to Zed, a lightning-fast, collaborative code editor that makes your dreams come true.
Development tips
Dependencies
-
Install Postgres.app and start it.
-
Install the
LiveKitserver and theforemanprocess supervisor:brew install livekit brew install foreman -
Ensure the Zed.dev website is checked out in a sibling directory:
cd .. git clone https://github.com/zed-industries/zed.dev -
Initialize submodules
git submodule update --init --recursive -
Set up a local
zeddatabase and seed it with some initial users:Create a personal GitHub token to run
script/bootstraponce successfully: the token needs to have an access to private repositories for the script to work (repoOAuth scope). Then delete that token.GITHUB_TOKEN=<$token> script/bootstrap
Testing against locally-running servers
Start the web and collab servers:
foreman start
If you want to run Zed pointed at the local servers, you can run:
script/zed-with-local-servers
# or...
script/zed-with-local-servers --release
Dump element JSON
If you trigger cmd-alt-i, Zed will copy a JSON representation of the current window contents to the clipboard. You can paste this in a tool like DJSON to navigate the state of on-screen elements in a structured way.
Licensing
We use cargo-about to automatically comply with open source licenses. If CI is failing, check the following:
- Is it showing a
no license specifiederror for a crate you've created? If so, addpublish = falseunder[package]in your crate's Cargo.toml. - Is the error
failed to satisfy license requirementsfor a dependency? If so, first determine what license the project has and whether this system is sufficient to comply with this license's requirements. If you're unsure, ask a lawyer. Once you've verified that this system is acceptable add the license's SPDX identifier to theacceptedarray inscript/licenses/zed-licenses.toml. - Is
cargo-aboutunable to find the license for a dependency? If so, add a clarification field at the end ofscript/licenses/zed-licenses.toml, as specified in the cargo-about book.
Wasm Plugins
Zed has a Wasm-based plugin runtime which it currently uses to embed plugins. To compile Zed, you'll need to have the wasm32-wasi toolchain installed on your system. To install this toolchain, run:
rustup target add wasm32-wasi
Plugins can be found in the plugins folder in the root. For more information about how plugins work, check the Plugin Guide in crates/plugin_runtime/README.md.