summaryrefslogtreecommitdiff
path: root/cli/npm
AgeCommit message (Collapse)Author
2024-06-07fix: make writing to the deps cache more reliable (#24135)David Sherret
I was able to reproduce this locally. ``` [error] Failed to execute snippet: import { validate } from "@std/uuid"; import { assert, assertFalse } from "@std/assert"; assert(validate("6ec0bd7f-11c0-43da-975e-2a8ad9ebae0b")); assertFalse(validate("not a UUID")); Download https://jsr.io/@std/uuid/meta.json Download https://jsr.io/@std/uuid/1.0.0-rc.1_meta.json Download https://jsr.io/@std/uuid/1.0.0-rc.1/mod.ts Download https://jsr.io/@std/uuid/1.0.0-rc.1/common.ts Download https://jsr.io/@std/uuid/1.0.0-rc.1/constants.ts Download https://jsr.io/@std/uuid/1.0.0-rc.1/v1.ts Download https://jsr.io/@std/uuid/1.0.0-rc.1/v3.ts Download https://jsr.io/@std/uuid/1.0.0-rc.1/v4.ts Download https://jsr.io/@std/uuid/1.0.0-rc.1/v5.ts Download https://jsr.io/@std/uuid/1.0.0-rc.1/_common.ts error: Access is denied. (os error 5) (for 'V:\.cache\deno\deps\https\jsr.io\2ae5bb614c7526d0876be0b76da1372fd51304ae27d6202ee94df720b3523d08') at file:///V:/deno_std/uuid/common.ts:43 [error] Failed to execute snippet: import { v5, NAMESPACE_DNS, NIL_UUID } from "@std/uuid"; import { assert, assertFalse } from "@std/assert"; const data = new TextEncoder().encode("deno.land"); const uuid = await v5.generate(NAMESPACE_DNS, data); assert(v5.validate(uuid)); assertFalse(v5.validate(NIL_UUID)); Download https://jsr.io/@std/uuid/meta.json Download https://jsr.io/@std/uuid/1.0.0-rc.1_meta.json Download https://jsr.io/@std/uuid/1.0.0-rc.1/mod.ts Download https://jsr.io/@std/uuid/1.0.0-rc.1/common.ts Download https://jsr.io/@std/uuid/1.0.0-rc.1/constants.ts Download https://jsr.io/@std/uuid/1.0.0-rc.1/v1.ts Download https://jsr.io/@std/uuid/1.0.0-rc.1/v3.ts Download https://jsr.io/@std/uuid/1.0.0-rc.1/v4.ts Download https://jsr.io/@std/uuid/1.0.0-rc.1/v5.ts Download https://jsr.io/@std/uuid/1.0.0-rc.1/_common.ts error: Access is denied. (os error 5) (for 'V:\.cache\deno\deps\https\jsr.io\63dd818c5fc1ac39c04df9b42bd9dd4bbc07f7d1b174e405d003731125778da1') at https://jsr.io/@std/uuid/1.0.0-rc.1/mod.ts:30:15 at file:///V:/deno_std/uuid/mod.ts:4 [error] Failed to execute snippet: import { isNil } from "@std/uuid"; import { assert, assertFalse } from "@std/assert"; assert(isNil("00000000-0000-0000-0000-000000000000")); assertFalse(isNil(crypto.randomUUID())); Download https://jsr.io/@std/uuid/meta.json Download https://jsr.io/@std/uuid/1.0.0-rc.1_meta.json Download https://jsr.io/@std/uuid/1.0.0-rc.1/mod.ts Download https://jsr.io/@std/uuid/1.0.0-rc.1/common.ts Download https://jsr.io/@std/uuid/1.0.0-rc.1/constants.ts Download https://jsr.io/@std/uuid/1.0.0-rc.1/v1.ts Download https://jsr.io/@std/uuid/1.0.0-rc.1/v3.ts Download https://jsr.io/@std/uuid/1.0.0-rc.1/v4.ts Download https://jsr.io/@std/uuid/1.0.0-rc.1/v5.ts Download https://jsr.io/@std/uuid/1.0.0-rc.1/_common.ts error: Access is denied. (os error 5) (for 'V:\.cache\deno\deps\https\jsr.io\fd3a12fc091d16ee29f10fa7a05eeeb8bd6c3cc014642e72478c757f00e7261e') at https://jsr.io/@std/uuid/1.0.0-rc.1/mod.ts:34:40 at file:///V:/deno_std/uuid/common.ts:23 [error] Failed to execute snippet: import { version } from "@std/uuid"; import { assertEquals } from "@std/assert/assert-equals"; assertEquals(version("d9428888-122b-11e1-b85c-61cd3cbb3210"), 1); assertEquals(version("6ec0bd7f-11c0-43da-975e-2a8ad9ebae0b"), 4); Download https://jsr.io/@std/uuid/meta.json Download https://jsr.io/@std/uuid/1.0.0-rc.1_meta.json Download https://jsr.io/@std/uuid/1.0.0-rc.1/mod.ts Download https://jsr.io/@std/uuid/1.0.0-rc.1/common.ts Download https://jsr.io/@std/uuid/1.0.0-rc.1/constants.ts Download https://jsr.io/@std/uuid/1.0.0-rc.1/v1.ts Download https://jsr.io/@std/uuid/1.0.0-rc.1/v3.ts Download https://jsr.io/@std/uuid/1.0.0-rc.1/v4.ts Download https://jsr.io/@std/uuid/1.0.0-rc.1/v5.ts Download https://jsr.io/@std/uuid/1.0.0-rc.1/_common.ts error: Access is denied. (os error 5) (for 'V:\.cache\deno\deps\https\jsr.io\2ae5bb614c7526d0876be0b76da1372fd51304ae27d6202ee94df720b3523d08') at file:///V:/deno_std/uuid/common.ts:66 4 errors found ``` It occurs when many Deno processes are writing to the deps cache at the same time. Fix is to use `atomic_write_with_retries` which is much more reliable (and the function that helped make the ecosystem tests more reliable too). After this change I no longer have this issue. Closes https://github.com/denoland/deno/issues/24073
2024-06-06refactor: remove `PermissionsContainer` in deno_runtime (#24119)David Sherret
Also removes permissions being passed in for node resolution. It was completely useless because we only checked it for reading package.json files, but Deno reading package.json files for resolution is perfectly fine. My guess is this is also a perf improvement because Deno is doing less work.
2024-06-06refactor: `NpmRegistryApi` - `#[async_trait(?Send)]` (#24126)David Sherret
2024-06-06fix(cli): Overwrite existing bin entries in `node_modules` (#24123)Nathan Whitaker
Previously we warned on unix and didn't touch them on windows, now we unconditionally overwrite them. This matches what npm does.
2024-06-05fix(npm): use configured auth for tarball urls instead of scope auth (#24111)David Sherret
Deno was using the scope auth for the tarball urls, which is not always correct. We are going to do a release immediately for this issue.
2024-06-05refactor(npm): improve locking around updating npm resolution (#24104)David Sherret
Introduces a `SyncReadAsyncWriteLock` to make it harder to write to the npm resolution without first waiting async in a queue. For the npm resolution, reading synchronously is fine, but when updating, someone should wait async, clone the data, then write the data at the end back.
2024-06-05fix: better handling of npm resolution occurring on workers (#24094)David Sherret
Closes https://github.com/denoland/deno/issues/24063
2024-06-03refactor: don't share `reqwest::HttpClient` across tokio runtimes (#24092)David Sherret
This also fixes several issues where we weren't properly creating http clients with the user's settings.
2024-06-03refactor: extract structs for downloading tarballs and npm registry ↵David Sherret
packuments (#24067)
2024-05-29fix(cli): Prefer npm bin entries provided by packages closer to the root ↵Nathan Whitaker
(#24024) Fixes #24012. In the case of multiple packages providing a binary with a same name, we were basically leaving the results undefined (since we set up things in parallel, and whichever got set up first won). In addition, we were warning about these cases, even though it's a situation that's expected to occur. Instead, in the case of a collision in the binary names, we prefer the binary provided by the package with the least depth in the dependency tree. While I was at it, I also took moved more code to `bin_entries.rs` since it was starting to get a bit cluttered.
2024-05-29fix: bump cache sqlite dbs to v2 for WAL journal mode change (#24030)David Sherret
In https://github.com/denoland/deno/pull/23955 we changed the sqlite db journal mode to WAL. This causes issues when someone is running an old version of Deno using TRUNCATE and a new version because the two fight against each other.
2024-05-28perf(cli): Improve concurrency when setting up `node_modules` and loading ↵Nathan Whitaker
cached npm package info (#24018) The same issue in two different places - doing blocking FS work in an async task, limiting the amount of work that happens concurrently. - When setting up node_modules, where we try to set up entries concurrently but were blocking other tasks from actually running. - When loading package info from the npm registry file cache, loading and deserializing is expensive and prevents concurrency. This was especially noticeable when loading an npm resolution snapshot from a lockfile (`snapshot_from_lockfile` in `deno_npm`). Installing deps in `deno-docs`: ``` ❯ hyperfine -i -p 'rm -rf node_modules/' '../d7/deno-main i' '../d7/target/release/deno i' Benchmark 1: ../d7/deno-main i Time (mean ± σ): 2.193 s ± 0.027 s [User: 0.589 s, System: 1.033 s] Range (min … max): 2.151 s … 2.242 s 10 runs Benchmark 2: ../d7/target/release/deno i Time (mean ± σ): 1.597 s ± 0.021 s [User: 0.977 s, System: 1.337 s] Range (min … max): 1.550 s … 1.627 s 10 runs Summary ../d7/target/release/deno i ran 1.37 ± 0.02 times faster than ../d7/deno-main i ``` Caching `npm:@11ty/eleventy`: ``` ❯ hyperfine -i -p 'rm -rf node_modules/' --warmup 5 '../../d7/deno-main cache npm:@11ty/eleventy' '../../d7/target/release/deno cache npm:@11ty/eleventy' Benchmark 1: ../../d7/deno-main cache npm:@11ty/eleventy Time (mean ± σ): 129.9 ms ± 2.2 ms [User: 27.5 ms, System: 101.3 ms] Range (min … max): 127.5 ms … 135.8 ms 10 runs Benchmark 2: ../../d7/target/release/deno cache npm:@11ty/eleventy Time (mean ± σ): 100.6 ms ± 1.3 ms [User: 38.8 ms, System: 233.8 ms] Range (min … max): 99.3 ms … 103.2 ms 10 runs Summary ../../d7/target/release/deno cache npm:@11ty/eleventy ran 1.29 ± 0.03 times faster than ../../d7/deno-main cache npm:@11ty/eleventy ``` --------- Co-authored-by: David Sherret <dsherret@gmail.com>
2024-05-28perf(cli): Optimize setting up `node_modules` on macOS (#23980)Nathan Whitaker
Hard linking (`linkat`) is ridiculously slow on mac. `copyfile` is better, but what's even faster is `clonefile`. It doesn't have the space savings that comes with hardlinking, but the performance difference is worth it imo. ``` ❯ hyperfine -i -p 'rm -rf node_modules/' '../../d7/target/release/deno cache npm:@11ty/eleventy' 'deno cache npm:@11ty/eleventy' Benchmark 1: ../../d7/target/release/deno cache npm:@11ty/eleventy Time (mean ± σ): 115.4 ms ± 1.2 ms [User: 27.2 ms, System: 87.3 ms] Range (min … max): 113.7 ms … 117.5 ms 10 runs Benchmark 2: deno cache npm:@11ty/eleventy Time (mean ± σ): 619.3 ms ± 6.4 ms [User: 34.3 ms, System: 575.6 ms] Range (min … max): 612.2 ms … 633.3 ms 10 runs Summary ../../d7/target/release/deno cache npm:@11ty/eleventy ran 5.37 ± 0.08 times faster than deno cache npm:@11ty/eleventy ```
2024-05-28feat(vendor): support modifying remote files in vendor folder without ↵David Sherret
checksum errors (#23979) Includes: * https://github.com/denoland/deno_graph/pull/486 * https://github.com/denoland/deno_graph/pull/488 * https://github.com/denoland/deno_lockfile/pull/25 * https://github.com/denoland/deno_lockfile/pull/22 * https://github.com/denoland/deno_graph/pull/483 * https://github.com/denoland/deno_graph/pull/470
2024-05-23fix(npm): set up node_modules/.bin/ entries for package that provide bin ↵Bartek Iwańczuk
entrypoints (#23496) Closes https://github.com/denoland/deno/issues/23036 --------- Co-authored-by: Nathan Whitaker <nathan@deno.com>
2024-05-23FUTURE: initial support for .npmrc file (#23560)Bartek Iwańczuk
This commit adds initial support for ".npmrc" files. Currently we only discover ".npmrc" files next to "package.json" files and discovering these files in user home dir is left for a follow up. This pass supports "_authToken" and "_auth" configuration for providing authentication. LSP support has been left for a follow up PR. Towards https://github.com/denoland/deno/issues/16105
2024-05-21fix(task): do not error if node_modules folder not exists (#23920)David Sherret
Revealed https://github.com/denoland/deno/issues/23919 Closes #23914
2024-05-14fix(npm): make tarball extraction more reliable (#23759)David Sherret
1. Extracts to a directory beside the destination. 2. Renames to the destination with retries.
2024-04-25fix(npm): do not panic hitting root dir while resolving npm package (#23556)David Sherret
Closes https://github.com/denoland/deno/issues/23029
2024-04-19refactor: move redirect handling into deno_graph (#23444)David Sherret
2024-04-17perf: v8 code cache (#23081)Igor Zinkovsky
This PR enables V8 code cache for ES modules and for `require` scripts through `op_eval_context`. Code cache artifacts are transparently stored and fetched using sqlite db and are passed to V8. `--no-code-cache` can be used to disable. --------- Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
2024-04-15fix(check): cache bust when changing nodeModulesDir setting (#23355)David Sherret
2024-04-11fix(npm): local nodeModulesDir was sometimes resolving duplicates of same ↵David Sherret
package (#23320)
2024-04-10chore: update to Rust 1.77.2 (#23262)林炳权
update to Rust 1.77.2 --------- Co-authored-by: Matt Mastracci <matthew@mastracci.com>
2024-04-05perf(lsp): use lockfile to reduce npm pkg resolution time (#23247)David Sherret
This functionality was broken. The series of events was: 1. Load the npm resolution from the lockfile. 2. Discover only a subset of the specifiers in the documents. 3. Clear the npm snapshot. 4. Redo npm resolution with the new specifiers (~500ms). What this now does: 1. Load the npm resolution from the lockfile. 2. Discover only a subset of the specifiers in the documents and take into account the specifiers from the lockfile. 3. Do not redo resolution (~1ms).
2024-04-01perf(node): put pkg json into an `Rc` (#23156)David Sherret
Was doing a bit of debugging on why some stuff is not working in a personal project and ran a quick debug profile and saw it cloning the pkg json a lot. We should put this in an Rc.
2024-03-28fix(jsr): exclude yanked versions from 'deno add' and completions (#23113)Nayeem Rahman
2024-03-07perf: hard link npm cache (#22773)Divy Srivastava
2024-03-07perf(cli): use faster_hex (#22761)Matt Mastracci
`cli::util::checksum` was showing up on flame graphs because it was concatenating allocated strings. We can use `faster-hex` to improve it.
2024-03-06feat(unstable/pm): support npm packages in 'deno add' (#22715)Nayeem Rahman
2024-01-22feat(lockfile): track JSR and npm dependencies in config file (#22004)David Sherret
See overview in https://github.com/denoland/deno_lockfile/pull/13
2024-01-01chore: update copyright to 2024 (#21753)David Sherret
2023-12-06fix(npm): do not create symlink for non-system optional dep in node_modules ↵David Sherret
directory (#21478) Closes https://github.com/denoland/deno/issues/21476
2023-11-29feat(compile): support "bring your own node_modules" in deno compile (#21377)David Sherret
Not tested thoroughly. This is a good start. Closes #21350
2023-11-11perf: snapshot runtime ops (#21127)Divy Srivastava
Closes https://github.com/denoland/deno/issues/21135 ~1ms startup time improvement --------- Signed-off-by: Divy Srivastava <dj.srivastava23@gmail.com> Co-authored-by: David Sherret <dsherret@users.noreply.github.com>
2023-11-04fix(node): use closest package.json to resolve package.json imports (#21075)David Sherret
2023-10-26fix(unstable/byonm): improve error messages (#20987)David Sherret
This improves the error messages when a specifier can't be resolved from a deno module into an npm package.
2023-10-26chore: update base64 crate (#20877)Luca Casonato
2023-10-25feat(unstable): ability to `npm install` then `deno run main.ts` (#20967)David Sherret
This PR adds a new unstable "bring your own node_modules" (BYONM) functionality currently behind a `--unstable-byonm` flag (`"unstable": ["byonm"]` in a deno.json). This enables users to run a separate install command (ex. `npm install`, `pnpm install`) then run `deno run main.ts` and Deno will respect the layout of the node_modules directory as setup by the separate install command. It also works with npm/yarn/pnpm workspaces. For this PR, the behaviour is opted into by specifying `--unstable-byonm`/`"unstable": ["byonm"]`, but in the future we may make this the default behaviour as outlined in https://github.com/denoland/deno/issues/18967#issuecomment-1761248941 This is an extremely rough initial implementation. Errors are terrible in this and the LSP requires frequent restarts. Improvements will be done in follow up PRs.
2023-10-03refactor(npm): break up `NpmModuleLoader` and move more methods into the ↵David Sherret
managed `CliNpmResolver` (#20777) Part of https://github.com/denoland/deno/issues/18967
2023-10-02refactor(npm): make `NpmCache`, `CliNpmRegistryApi`, and `NpmResolution` ↵David Sherret
internal to `npm::managed` (#20764)
2023-09-30refactor(npm): create `cli::npm::managed` module (#20740)David Sherret
Creates the `cli::npm::managed` module and starts moving more functionality into it.
2023-09-29refactor(cli): make `CliNpmResolver` a trait (#20732)David Sherret
This makes `CliNpmResolver` a trait. The terminology used is: - **managed** - Deno manages the node_modules folder and does an auto-install (ex. `ManagedCliNpmResolver`) - **byonm** - "Bring your own node_modules" (ex. `ByonmCliNpmResolver`, which is in this PR, but unimplemented at the moment) Part of #18967
2023-09-28refactor(ext/node): remove dependency on deno_npm and deno_semver (#20718)David Sherret
This is required from BYONM (bring your own node_modules). Part of #18967
2023-09-26chore: slight cleanup in npm resolvers (#20692)David Sherret
2023-09-18fix(npm): properly handle legacy shasum of package (#20557)David Sherret
Closes #20554
2023-09-14refactor: remove `DENO_UNSTABLE_NPM_SYNC_DOWNLOAD` and custom sync ↵David Sherret
functionality (#20504) https://github.com/denoland/deno/pull/20488 enables us to remove this functionality. This is better because our test suite is now not testing a separate code path.
2023-09-13refactor: move `deno_core::TaskQueue` to `cli::util::sync` (#20481)David Sherret
TaskQueue is being removed from `deno_core` and replaced with an unsync version in deno_unsyc. https://github.com/denoland/deno_core/pull/193 This is a change in preparation for that. The remaining `deno_core::TaskQueue` usage in this repo should be replaced with `deno_core::unsync::TaskQueue` once upgraded.
2023-09-08feat: lockfile v3 (#20424)David Sherret
Details: https://github.com/denoland/deno_lockfile/pull/8
2023-08-26chore: update to Rust 1.72 (#20258)林炳权
<!-- Before submitting a PR, please read https://deno.com/manual/contributing 1. Give the PR a descriptive title. Examples of good title: - fix(std/http): Fix race condition in server - docs(console): Update docstrings - feat(doc): Handle nested reexports Examples of bad title: - fix #7123 - update docs - fix bugs 2. Ensure there is a related issue and it is referenced in the PR text. 3. Ensure there are tests that cover the changes. 4. Ensure `cargo test` passes. 5. Ensure `./tools/format.js` passes without changing files. 6. Ensure `./tools/lint.js` passes. 7. Open as a draft PR if your work is still in progress. The CI won't run all steps, but you can add '[ci]' to a commit message to force it to. 8. If you would like to run the benchmarks on the CI, add the 'ci-bench' label. --> As the title. --------- Co-authored-by: Matt Mastracci <matthew@mastracci.com>
generated by cgit v1.2.3 (git 2.25.1) at 2025-08-08 10:33:33 +0000