summaryrefslogtreecommitdiff
path: root/cli/cache
AgeCommit message (Collapse)Author
2023-06-10chore(tests): test_util - Add `PathRef` (#19450)David Sherret
This adds a new `PathRef` struct to test_util for making it easier to work with paths in test code. I'm going to expand on this more in the future.
2023-05-30fix: do not show cache initialization errors if stderr is piped (#18920)David Sherret
Closes #18918
2023-05-25fix(compile): handle when DENO_DIR is readonly (#19257)David Sherret
Closes #19253
2023-05-14refactor(core): bake single-thread assumptions into spawn/spawn_blocking ↵Matt Mastracci
(#19056) Partially supersedes #19016. This migrates `spawn` and `spawn_blocking` to `deno_core`, and removes the requirement for `spawn` tasks to be `Send` given our single-threaded executor. While we don't need to technically do anything w/`spawn_blocking`, this allows us to have a single `JoinHandle` type that works for both cases, and allows us to more easily experiment with alternative `spawn_blocking` implementations that do not require tokio (ie: rayon). Async ops (+~35%): Before: ``` time 1310 ms rate 763358 time 1267 ms rate 789265 time 1259 ms rate 794281 time 1266 ms rate 789889 ``` After: ``` time 956 ms rate 1046025 time 954 ms rate 1048218 time 924 ms rate 1082251 time 920 ms rate 1086956 ``` HTTP serve (+~4.4%): Before: ``` Running 10s test @ http://localhost:4500 2 threads and 10 connections Thread Stats Avg Stdev Max +/- Stdev Latency 68.78us 19.77us 1.43ms 86.84% Req/Sec 68.78k 5.00k 73.84k 91.58% 1381833 requests in 10.10s, 167.36MB read Requests/sec: 136823.29 Transfer/sec: 16.57MB ``` After: ``` Running 10s test @ http://localhost:4500 2 threads and 10 connections Thread Stats Avg Stdev Max +/- Stdev Latency 63.12us 17.43us 1.11ms 85.13% Req/Sec 71.82k 3.71k 77.02k 79.21% 1443195 requests in 10.10s, 174.79MB read Requests/sec: 142921.99 Transfer/sec: 17.31MB ``` Suggested-By: alice@ryhl.io Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
2023-05-01refactor(cli): remove ProcState - add CliFactory (#18900)David Sherret
This removes `ProcState` and replaces it with a new `CliFactory` which initializes our "service structs" on demand. This isn't a performance improvement at the moment for `deno run`, but might unlock performance improvements in the future.
2023-04-26feat(cli): don't check permissions for statically analyzable dynamic imports ↵Nayeem Rahman
(#18713) Closes #17697 Closes #17658
2023-04-21refactor: move some CJS and ESM code analysis to ext/node (#18789)David Sherret
2023-04-14refactor: break up `ProcState` (#18707)David Sherret
1. Breaks up functionality within `ProcState` into several other structs to break out the responsibilities (`ProcState` is only a data struct now). 2. Moves towards being able to inject dependencies more easily and have functionality only require what it needs. 3. Exposes `Arc<T>` around the "service structs" instead of it being embedded within them. The idea behind embedding them was to reduce the verbosity of needing to pass around `Arc<...>`, but I don't think it was exactly working and as we move more of these structs to be more injectable I don't think the extra verbosity will be a big deal.
2023-03-30fix(lsp): include all diagnosable documents on initialize (#17979)David Sherret
Closes https://github.com/denoland/vscode_deno/issues/797 Closes https://github.com/denoland/deno/issues/11190 Closes https://github.com/denoland/vscode_deno/issues/811 Closes https://github.com/denoland/vscode_deno/issues/761 Closes https://github.com/denoland/vscode_deno/issues/585 Closes https://github.com/denoland/vscode_deno/issues/561 Closes https://github.com/denoland/vscode_deno/issues/410
2023-03-28fix(core): restore cache journal mode to TRUNCATE and tweak tokio test in ↵Matt Mastracci
CacheDB (#18469) Fast-follow on #18401 -- the reason that some tests were panicking in the `CacheDB` `impl Drop` was that the cache itself was being dropped during panic and the runtime may or may not still exist at that point. We can reduce the actual tokio runtime testing to where it's needed. In addition, we return the journal mode to `TRUNCATE` to avoid the risk of data corruption.
2023-03-27feat(core): initialize SQLite off-main-thread (#18401)Matt Mastracci
This gets SQLite off the flamegraph and reduces initialization time by somewhere between 0.2ms and 0.5ms. In addition, I took the opportunity to move all the cache management code to a single place and reduce duplication. While the PR has a net gain of lines, much of that is just being a bit more deliberate with how we're recovering from errors. The existing caches had various policies for dealing with cache corruption, so I've unified them and tried to isolate the decisions we make for recovery in a single place (see `open_connection` in `CacheDB`). The policy I chose was: 1. Retry twice to open on-disk caches 2. If that fails, try to delete the file and recreate it on-disk 3. If we fail to delete the file or re-create a new cache, use a fallback strategy that can be chosen per-cache: InMemory (temporary cache for the process run), BlackHole (ignore writes, return empty reads), or Error (fail on every operation). The caches all use the same general code now, and share the cache failure recovery policy. In addition, it cleans up a TODO in the `NodeAnalysisCache`.
2023-03-23refactor: make version and user_agent &'static str (#18400)Bartek Iwańczuk
These caused a bunch of unnecessary allocations on each startup.
2023-03-22fix: include error in message about not being able to create the TypeScript ↵David Sherret
cache (#18356)
2023-03-22feat(cli): --ext parameter for run, compile, and bundle (#17172)Cre3per
Adds `--ext` to `deno run`, closes #5088 Additionally - Adds `--ext` to `deno compile` and `deno bundle`
2023-03-22perf: disable WAL for transpiled source cache (#18084)Bartek Iwańczuk
Disable Write-Ahead Log for the cached module source database. This brings SQLite connection cost on startup from 2.5% to 1.6%.
2023-03-21Revert "fix(cli) Better error messages on corrupt databases that aren… ↵Bartek Iwańczuk
(#18337) …'t automatically re-created (#18330)" This reverts commit 2ef8269fdb395b0736153ff5fbb9696cbb976e42. Printing these messages by default (instead of requiring `-L debug` flag) caused various tests to start printing it and mismatch in output assertions.
2023-03-21fix(cli) Better error messages on corrupt databases that aren't ↵Matt Mastracci
automatically re-created (#18330)
2023-03-17feat(repl): add `DENO_REPL_HISTORY` to change history file path (#18047)Nick Hanley
2023-03-15refactor: remove usages of `map_or` / `map_or_else` (#18212)David Sherret
These methods are confusing because the arguments are backwards. I feel like they should have never been added to `Option<T>` and that clippy should suggest rewriting to `map(...).unwrap_or(...)`/`map(...).unwrap_or_else(|| ...)` https://github.com/rust-lang/rfcs/issues/1025
2023-02-24fix(npm): lazily install package.json dependencies only when necessary (#17931)David Sherret
This lazily does an "npm install" when any package name matches what's found in the package.json or when running a script from package.json with deno task. Part of #17916 Closes #17928
2023-02-23feat(npm): support bare specifiers from package.json in more subcommands and ↵David Sherret
language server (#17891)
2023-02-22feat: support bare specifier resolution with package.json (#17864)Bartek Iwańczuk
This commit enables resolution of "bare specifiers" (eg. "import express from 'express';") if a "package.json" file is discovered. It's a step towards being able to run projects authored for Node.js without any changes. With this commit we are able to successfully run Vite projects without any changes to the user code. --------- Co-authored-by: David Sherret <dsherret@gmail.com>
2023-02-22refactor: use deno_graph for npm specifiers (#17858)David Sherret
This changes npm specifiers to be handled by deno_graph and resolved to an npm package name and version when the specifier is encountered. It also slightly changes how npm specifier resolution occurs—previously it would collect all the npm specifiers and resolve them all at once, but now it resolves them on the fly as they are encountered in the module graph. https://github.com/denoland/deno_graph/pull/232 --------- Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
2023-02-21fix(npm): improve peer dependency resolution (#17835)David Sherret
This PR fixes peer dependency resolution to only resolve peers based on the current graph traversal path. Previously, it would resolve a peers by looking at a graph node's ancestors, which is not correct because graph nodes are shared by different resolutions. It also stores more information about peer dependency resolution in the lockfile.
2023-02-15feat: wire up ext/node to the Node compatibility layer (#17785)Bartek Iwańczuk
This PR changes Node.js/npm compatibility layer to use polyfills for built-in Node.js embedded in the snapshot (that are coming from "ext/node" extension). As a result loading `std/node`, either from "https://deno.land/std@<latest>/" or from "DENO_NODE_COMPAT_URL" env variable were removed. All code that is imported via "npm:" specifiers now uses code embedded in the snapshot. Several fixes were applied to various modules in "ext/node" to make tests pass. --------- Co-authored-by: Yoshiya Hinosawa <stibium121@gmail.com> Co-authored-by: Divy Srivastava <dj.srivastava23@gmail.com>
2023-02-15refactor: use deno_graph's semver and npm structs (#17791)David Sherret
2023-02-09perf: module info cache - avoid MediaType.to_string() allocation (#17699)David Sherret
Micro optimization because these allocations were coming up on a flame graph I was looking at (only 0.28% of total).
2023-02-08perf: disable fetching graph cache info except for `deno info` (#17698)David Sherret
2023-02-03refactor: reland "preserve ProcState::file_fetcher between restarts" (#17636)Nayeem Rahman
Just some watcher init step that I thought would be "cloned over" but needs to be done again on reset.
2023-01-30Revert "fix(watch): preserve `ProcState::file_fetcher` between restarts ↵Bartek Iwańczuk
(#15466) (#17591) This reverts commit 3545bff678f20c3fdf17fe6b26f96cf1b74f917c.
2023-01-27chore: upgrade to Rust 1.67 (#17548)David Sherret
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
2023-01-27chore(cli/cache): fewer string allocations (#17549)Geert-Jan Zwiers
2023-01-24feat: support node built-in module imports (#17264)Bartek Iwańczuk
Co-authored-by: David Sherret <dsherret@gmail.com>
2023-01-14chore: use rustfmt imports_granularity option (#17421)Divy Srivastava
Closes https://github.com/denoland/deno/issues/2699 Closes https://github.com/denoland/deno/issues/2347 Uses unstable rustfmt features. Since dprint invokes `rustfmt` we do not need to switch the cargo toolchain to nightly. Do we care about formatting stability of our codebase across Rust versions? (I don't)
2023-01-13chore: add `copyright_checker` tool and add the missing copyright (#17285)Yiyu Lin
2023-01-10fix(watch): preserve `ProcState::file_fetcher` between restarts (#15466)Nayeem Rahman
This commit changes "ProcState" to store "file_fetcher" field in an "Arc", allowing it to be preserved between restarts and thus keeping the state alive between the restarts. File watchers for "deno test" and "deno bench" now reset "ProcState" between restarts.
2023-01-07refactor(permissions): add PermissionsContainer struct for internal ↵Bartek Iwańczuk
mutability (#17134) Turns out we were cloning permissions which after prompting were discarded, so the state of permissions was never preserved. To handle that we need to store all permissions behind "Arc<Mutex<>>" (because there are situations where we need to send them to other thread). Testing and benching code still uses "Permissions" in most places - it's undesirable to share the same permission set between various test/bench files - otherwise granting or revoking permissions in one file would influence behavior of other test files.
2023-01-02chore: update copyright year to 2023 (#17247)David Sherret
Yearly tradition of creating extra noise in git.
2022-12-17chore: update to Rust 1.66.0 (#17078)linbingquan
2022-11-29fix(coverage): Error if the emit cache is invalid (#16850)sigmaSd
2022-11-28refactor: create util folder, move nap_sym to napi/sym, move http_cache to ↵David Sherret
cache folder (#16857)
2022-11-26refactor: `DenoDir` - move to cache folder and make `root_dir` private (#16823)David Sherret
2022-11-23chore: more debug logging and avoid allocating strings in ts logging when ↵David Sherret
not debug (#16689)
2022-11-18chore: use Rust 1.65.0 (#16688)Aaron O'Mullan
2022-10-01perf: node cjs & esm analysis cache (#16097)David Sherret
This commit adds a cache for CJS and ESM analysis that is backed by an SQLite file. The connection to the DB is lazily created on first use, so shouldn't have impact on the startup time. Benched with running Vite Deno v1.26: ``` $ deno task dev Warning deno task is unstable and may drastically change in the future Task dev deno run -A --unstable --node-modules-dir npm:vite VITE v3.1.4 ready in 961 ms ➜ Local: http://localhost:5173/ ➜ Network: use --host to expose ``` This branch: ``` ../deno/target/release/deno task dev Warning deno task is unstable and may drastically change in the future Task dev deno run -A --unstable --node-modules-dir npm:vite VITE v3.1.4 ready in 330 ms ➜ Local: http://localhost:5173/ ➜ Network: use --host to expose ``` Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
2022-09-07fix: upgrade deno_ast to 0.19 (#15808)David Sherret
2022-08-29fix(compile): panic when running with a populated dep analysis cache (#15672)David Sherret
Closes #15612
2022-08-22perf: cache swc dependency analysis and don't hold onto `ParsedSource`s in ↵David Sherret
memory (#15502)
2022-08-20feat(unstable): initial support for npm specifiers (#15484)David Sherret
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
2022-07-30refactor(emit/cache): move cli version into emit hash (#15348)David Sherret