summaryrefslogtreecommitdiff
path: root/cli/npm
AgeCommit message (Collapse)Author
2024-05-28perf(cli): Improve concurrency when setting up `node_modules` and loading ↵Nathan Whitaker
cached npm package info (#24018) The same issue in two different places - doing blocking FS work in an async task, limiting the amount of work that happens concurrently. - When setting up node_modules, where we try to set up entries concurrently but were blocking other tasks from actually running. - When loading package info from the npm registry file cache, loading and deserializing is expensive and prevents concurrency. This was especially noticeable when loading an npm resolution snapshot from a lockfile (`snapshot_from_lockfile` in `deno_npm`). Installing deps in `deno-docs`: ``` ❯ hyperfine -i -p 'rm -rf node_modules/' '../d7/deno-main i' '../d7/target/release/deno i' Benchmark 1: ../d7/deno-main i Time (mean ± σ): 2.193 s ± 0.027 s [User: 0.589 s, System: 1.033 s] Range (min … max): 2.151 s … 2.242 s 10 runs Benchmark 2: ../d7/target/release/deno i Time (mean ± σ): 1.597 s ± 0.021 s [User: 0.977 s, System: 1.337 s] Range (min … max): 1.550 s … 1.627 s 10 runs Summary ../d7/target/release/deno i ran 1.37 ± 0.02 times faster than ../d7/deno-main i ``` Caching `npm:@11ty/eleventy`: ``` ❯ hyperfine -i -p 'rm -rf node_modules/' --warmup 5 '../../d7/deno-main cache npm:@11ty/eleventy' '../../d7/target/release/deno cache npm:@11ty/eleventy' Benchmark 1: ../../d7/deno-main cache npm:@11ty/eleventy Time (mean ± σ): 129.9 ms ± 2.2 ms [User: 27.5 ms, System: 101.3 ms] Range (min … max): 127.5 ms … 135.8 ms 10 runs Benchmark 2: ../../d7/target/release/deno cache npm:@11ty/eleventy Time (mean ± σ): 100.6 ms ± 1.3 ms [User: 38.8 ms, System: 233.8 ms] Range (min … max): 99.3 ms … 103.2 ms 10 runs Summary ../../d7/target/release/deno cache npm:@11ty/eleventy ran 1.29 ± 0.03 times faster than ../../d7/deno-main cache npm:@11ty/eleventy ``` --------- Co-authored-by: David Sherret <dsherret@gmail.com>
2024-05-28perf(cli): Optimize setting up `node_modules` on macOS (#23980)Nathan Whitaker
Hard linking (`linkat`) is ridiculously slow on mac. `copyfile` is better, but what's even faster is `clonefile`. It doesn't have the space savings that comes with hardlinking, but the performance difference is worth it imo. ``` ❯ hyperfine -i -p 'rm -rf node_modules/' '../../d7/target/release/deno cache npm:@11ty/eleventy' 'deno cache npm:@11ty/eleventy' Benchmark 1: ../../d7/target/release/deno cache npm:@11ty/eleventy Time (mean ± σ): 115.4 ms ± 1.2 ms [User: 27.2 ms, System: 87.3 ms] Range (min … max): 113.7 ms … 117.5 ms 10 runs Benchmark 2: deno cache npm:@11ty/eleventy Time (mean ± σ): 619.3 ms ± 6.4 ms [User: 34.3 ms, System: 575.6 ms] Range (min … max): 612.2 ms … 633.3 ms 10 runs Summary ../../d7/target/release/deno cache npm:@11ty/eleventy ran 5.37 ± 0.08 times faster than deno cache npm:@11ty/eleventy ```
2024-05-28feat(vendor): support modifying remote files in vendor folder without ↵David Sherret
checksum errors (#23979) Includes: * https://github.com/denoland/deno_graph/pull/486 * https://github.com/denoland/deno_graph/pull/488 * https://github.com/denoland/deno_lockfile/pull/25 * https://github.com/denoland/deno_lockfile/pull/22 * https://github.com/denoland/deno_graph/pull/483 * https://github.com/denoland/deno_graph/pull/470
2024-05-23fix(npm): set up node_modules/.bin/ entries for package that provide bin ↵Bartek Iwańczuk
entrypoints (#23496) Closes https://github.com/denoland/deno/issues/23036 --------- Co-authored-by: Nathan Whitaker <nathan@deno.com>
2024-05-23FUTURE: initial support for .npmrc file (#23560)Bartek Iwańczuk
This commit adds initial support for ".npmrc" files. Currently we only discover ".npmrc" files next to "package.json" files and discovering these files in user home dir is left for a follow up. This pass supports "_authToken" and "_auth" configuration for providing authentication. LSP support has been left for a follow up PR. Towards https://github.com/denoland/deno/issues/16105
2024-05-21fix(task): do not error if node_modules folder not exists (#23920)David Sherret
Revealed https://github.com/denoland/deno/issues/23919 Closes #23914
2024-05-14fix(npm): make tarball extraction more reliable (#23759)David Sherret
1. Extracts to a directory beside the destination. 2. Renames to the destination with retries.
2024-04-25fix(npm): do not panic hitting root dir while resolving npm package (#23556)David Sherret
Closes https://github.com/denoland/deno/issues/23029
2024-04-19refactor: move redirect handling into deno_graph (#23444)David Sherret
2024-04-17perf: v8 code cache (#23081)Igor Zinkovsky
This PR enables V8 code cache for ES modules and for `require` scripts through `op_eval_context`. Code cache artifacts are transparently stored and fetched using sqlite db and are passed to V8. `--no-code-cache` can be used to disable. --------- Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
2024-04-15fix(check): cache bust when changing nodeModulesDir setting (#23355)David Sherret
2024-04-11fix(npm): local nodeModulesDir was sometimes resolving duplicates of same ↵David Sherret
package (#23320)
2024-04-10chore: update to Rust 1.77.2 (#23262)林炳权
update to Rust 1.77.2 --------- Co-authored-by: Matt Mastracci <matthew@mastracci.com>
2024-04-05perf(lsp): use lockfile to reduce npm pkg resolution time (#23247)David Sherret
This functionality was broken. The series of events was: 1. Load the npm resolution from the lockfile. 2. Discover only a subset of the specifiers in the documents. 3. Clear the npm snapshot. 4. Redo npm resolution with the new specifiers (~500ms). What this now does: 1. Load the npm resolution from the lockfile. 2. Discover only a subset of the specifiers in the documents and take into account the specifiers from the lockfile. 3. Do not redo resolution (~1ms).
2024-04-01perf(node): put pkg json into an `Rc` (#23156)David Sherret
Was doing a bit of debugging on why some stuff is not working in a personal project and ran a quick debug profile and saw it cloning the pkg json a lot. We should put this in an Rc.
2024-03-28fix(jsr): exclude yanked versions from 'deno add' and completions (#23113)Nayeem Rahman
2024-03-07perf: hard link npm cache (#22773)Divy Srivastava
2024-03-07perf(cli): use faster_hex (#22761)Matt Mastracci
`cli::util::checksum` was showing up on flame graphs because it was concatenating allocated strings. We can use `faster-hex` to improve it.
2024-03-06feat(unstable/pm): support npm packages in 'deno add' (#22715)Nayeem Rahman
2024-01-22feat(lockfile): track JSR and npm dependencies in config file (#22004)David Sherret
See overview in https://github.com/denoland/deno_lockfile/pull/13
2024-01-01chore: update copyright to 2024 (#21753)David Sherret
2023-12-06fix(npm): do not create symlink for non-system optional dep in node_modules ↵David Sherret
directory (#21478) Closes https://github.com/denoland/deno/issues/21476
2023-11-29feat(compile): support "bring your own node_modules" in deno compile (#21377)David Sherret
Not tested thoroughly. This is a good start. Closes #21350
2023-11-11perf: snapshot runtime ops (#21127)Divy Srivastava
Closes https://github.com/denoland/deno/issues/21135 ~1ms startup time improvement --------- Signed-off-by: Divy Srivastava <dj.srivastava23@gmail.com> Co-authored-by: David Sherret <dsherret@users.noreply.github.com>
2023-11-04fix(node): use closest package.json to resolve package.json imports (#21075)David Sherret
2023-10-26fix(unstable/byonm): improve error messages (#20987)David Sherret
This improves the error messages when a specifier can't be resolved from a deno module into an npm package.
2023-10-26chore: update base64 crate (#20877)Luca Casonato
2023-10-25feat(unstable): ability to `npm install` then `deno run main.ts` (#20967)David Sherret
This PR adds a new unstable "bring your own node_modules" (BYONM) functionality currently behind a `--unstable-byonm` flag (`"unstable": ["byonm"]` in a deno.json). This enables users to run a separate install command (ex. `npm install`, `pnpm install`) then run `deno run main.ts` and Deno will respect the layout of the node_modules directory as setup by the separate install command. It also works with npm/yarn/pnpm workspaces. For this PR, the behaviour is opted into by specifying `--unstable-byonm`/`"unstable": ["byonm"]`, but in the future we may make this the default behaviour as outlined in https://github.com/denoland/deno/issues/18967#issuecomment-1761248941 This is an extremely rough initial implementation. Errors are terrible in this and the LSP requires frequent restarts. Improvements will be done in follow up PRs.
2023-10-03refactor(npm): break up `NpmModuleLoader` and move more methods into the ↵David Sherret
managed `CliNpmResolver` (#20777) Part of https://github.com/denoland/deno/issues/18967
2023-10-02refactor(npm): make `NpmCache`, `CliNpmRegistryApi`, and `NpmResolution` ↵David Sherret
internal to `npm::managed` (#20764)
2023-09-30refactor(npm): create `cli::npm::managed` module (#20740)David Sherret
Creates the `cli::npm::managed` module and starts moving more functionality into it.
2023-09-29refactor(cli): make `CliNpmResolver` a trait (#20732)David Sherret
This makes `CliNpmResolver` a trait. The terminology used is: - **managed** - Deno manages the node_modules folder and does an auto-install (ex. `ManagedCliNpmResolver`) - **byonm** - "Bring your own node_modules" (ex. `ByonmCliNpmResolver`, which is in this PR, but unimplemented at the moment) Part of #18967
2023-09-28refactor(ext/node): remove dependency on deno_npm and deno_semver (#20718)David Sherret
This is required from BYONM (bring your own node_modules). Part of #18967
2023-09-26chore: slight cleanup in npm resolvers (#20692)David Sherret
2023-09-18fix(npm): properly handle legacy shasum of package (#20557)David Sherret
Closes #20554
2023-09-14refactor: remove `DENO_UNSTABLE_NPM_SYNC_DOWNLOAD` and custom sync ↵David Sherret
functionality (#20504) https://github.com/denoland/deno/pull/20488 enables us to remove this functionality. This is better because our test suite is now not testing a separate code path.
2023-09-13refactor: move `deno_core::TaskQueue` to `cli::util::sync` (#20481)David Sherret
TaskQueue is being removed from `deno_core` and replaced with an unsync version in deno_unsyc. https://github.com/denoland/deno_core/pull/193 This is a change in preparation for that. The remaining `deno_core::TaskQueue` usage in this repo should be replaced with `deno_core::unsync::TaskQueue` once upgraded.
2023-09-08feat: lockfile v3 (#20424)David Sherret
Details: https://github.com/denoland/deno_lockfile/pull/8
2023-08-26chore: update to Rust 1.72 (#20258)林炳权
<!-- Before submitting a PR, please read https://deno.com/manual/contributing 1. Give the PR a descriptive title. Examples of good title: - fix(std/http): Fix race condition in server - docs(console): Update docstrings - feat(doc): Handle nested reexports Examples of bad title: - fix #7123 - update docs - fix bugs 2. Ensure there is a related issue and it is referenced in the PR text. 3. Ensure there are tests that cover the changes. 4. Ensure `cargo test` passes. 5. Ensure `./tools/format.js` passes without changing files. 6. Ensure `./tools/lint.js` passes. 7. Open as a draft PR if your work is still in progress. The CI won't run all steps, but you can add '[ci]' to a commit message to force it to. 8. If you would like to run the benchmarks on the CI, add the 'ci-bench' label. --> As the title. --------- Co-authored-by: Matt Mastracci <matthew@mastracci.com>
2023-08-23fix(ext/web): add stream tests to detect v8slice split bug (#20253)Matt Mastracci
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
2023-08-21refactor: upgrade deno_ast 0.28 and deno_semver 0.4 (#20193)David Sherret
2023-08-16feat(ext/node): eagerly bootstrap node (#20153)Matt Mastracci
To fix bugs around detection of when node emulation is required, we will just eagerly initialize it. The improvements we make to reduce the impact of the startup time: - [x] Process stdin/stdout/stderr are lazily created - [x] node.js global proxy no longer allocates on each access check - [x] Process checks for `beforeExit` listeners before doing expensive shutdown work - [x] Process should avoid adding global event handlers until listeners are added Benchmarking this PR (`89de7e1ff`) vs main (`41cad2179`) ``` 12:36 $ third_party/prebuilt/mac/hyperfine --warmup 100 -S none './deno-41cad2179 run ./empty.js' './deno-89de7e1ff run ./empty.js' Benchmark 1: ./deno-41cad2179 run ./empty.js Time (mean ± σ): 24.3 ms ± 1.6 ms [User: 16.2 ms, System: 6.0 ms] Range (min … max): 21.1 ms … 29.1 ms 115 runs Benchmark 2: ./deno-89de7e1ff run ./empty.js Time (mean ± σ): 24.0 ms ± 1.4 ms [User: 16.3 ms, System: 5.6 ms] Range (min … max): 21.3 ms … 28.6 ms 126 runs ``` Fixes https://github.com/denoland/deno/issues/20142 Fixes https://github.com/denoland/deno/issues/15826 Fixes https://github.com/denoland/deno/issues/20028
2023-08-08chore: rename some helpers on the Fs trait (#20097)Luca Casonato
Rename some of the helper methods on the Fs trait to be suffixed with `_sync` / `_async`, in preparation of the introduction of more async methods for some helpers. Also adds a `read_text_file_async` helper to complement the renamed `read_text_file_sync` helper.
2023-08-08refactor(cli): move `snapshot_from_lockfile` function to `deno_npm` (#20024)Yusuke Tanaka
This commit moves `snapshot_from_lockfile` function to [deno_npm crate](https://github.com/denoland/deno_npm). This allows this function to be called outside Deno CLI (in particular, Deno Deploy).
2023-08-02feat(unstable): optional `deno_modules` directory (#19977)David Sherret
Closes #15633
2023-07-26fix(check): should bust check cache when json module or npm resolution ↵David Sherret
changes (#19941) A small part of #19928.
2023-07-17fix(node): improve require esm error messages (#19853)David Sherret
Part of #19842. Closes #19583 Closes #16913
2023-07-14fix(npm): improve error message importing non-existent file in a ↵David Sherret
node_modules npm package (#19835)
2023-07-10perf: add setup cache for node_modules folder (#19787)David Sherret
Part of #19774. This makes it twice as fast on my machine. Stores a file at `node_modules/.deno/setup-cache.bin`, which contains information about how the node_modules folder is currently setup. Obviously there is a risk that this information will get out of date with the current folder structure.
2023-07-04fix: bump default @types/node version range to 18.16.19 (#19706)David Sherret
generated by cgit v1.2.3 (git 2.25.1) at 2025-08-07 22:35:18 +0000