| Age | Commit message (Collapse) | Author |
|---|
|
This PR caches `_iterableHeaders` for immutable `Headers` increasing the
performance of `fetch` & server if headers are iterated.
Should close #19466
I only cached immutable headers to address this comment
https://github.com/denoland/deno/issues/19466#issuecomment-1589892373
since I didn't find any occurrence of header mutation on immutable
headers. We can discuss caching for non-immutable, but I think this is a
great first step.
## BENCHMARK
### Server
```js
const addr = Deno.args[0] ?? "127.0.0.1:4500";
const [hostname, port] = addr.split(":");
const { serve } = Deno;
serve({ hostname, port: Number(port), reusePort: true }, (req) => {
const headers = [...req.headers]; // req.headers are immutable, cannot set/append/delete
return new Response("ok");
});
```
Used `wrk` with 5 headers
```
wrk -d 10s --latency -H "X-Deno: true" -H "Accept: application/json" -H "X-Foo: bar" -H "User-Agent: wrk" -H "Accept-Encoding: gzip, br" http://127.0.0.1:4500
```
**This patch**
```
Running 10s test @ http://127.0.0.1:4500
2 threads and 10 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 70.18us 22.89us 679.00us 81.37%
Req/Sec 71.55k 9.69k 82.18k 89.60%
Latency Distribution
50% 59.00us
75% 89.00us
90% 98.00us
99% 159.00us
1437891 requests in 10.10s, 193.35MB read
Requests/sec: 142369.83
Transfer/sec: 19.14MB
```
**main**
```
Running 10s test @ http://127.0.0.1:4500
2 threads and 10 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 112.78us 36.47us 2.09ms 77.99%
Req/Sec 44.30k 1.65k 49.14k 74.26%
Latency Distribution
50% 99.00us
75% 136.00us
90% 162.00us
99% 213.00us
890588 requests in 10.10s, 118.91MB read
Requests/sec: 88176.37
Transfer/sec: 11.77MB
```
### fetch
```js
const res = await fetch('http://127.0.0.1:4500');
Deno.bench("Headers iterator", () => {
const i = [...res.headers]; // res.headers are immutable, cannot set/append/delete
});
```
**this patch**
```
cpu: 13th Gen Intel(R) Core(TM) i9-13900H
runtime: deno 1.36.1 (x86_64-unknown-linux-gnu)
benchmark time (avg) iter/s (min … max) p75 p99 p995
---------------------------------------------------------------------- -----------------------------
Headers iterator 329.5 ns/iter 3,034,909.0 (318.55 ns … 364.34 ns) 331.1 ns 355.72 ns 364.34 ns
```
**main**
```
cpu: 13th Gen Intel(R) Core(TM) i9-13900H
runtime: deno 1.36.1 (x86_64-unknown-linux-gnu)
benchmark time (avg) iter/s (min … max) p75 p99 p995
---------------------------------------------------------------------- -----------------------------
Headers iterator 2.59 µs/iter 386,372.1 (2.56 µs … 2.68 µs) 2.59 µs 2.68 µs 2.68 µs
```
|
|
This PR optimizes `Request` constructor when `init` is not empty. This
path is also used by `fetch` when `options` argument is used
```js
fetch("https://deno.land", {
method: "POST",
body: 'land'
});
```
- Removed 3 extra calls to `headerListFromHeaders`
- Avoid `Object.keys` & `headerList` clone if `init.headers` is set
- Only empty `headersList` (`.splice`) if it's not already empty.
## Benchmarks
**this patch**
```
cpu: 13th Gen Intel(R) Core(TM) i9-13900H
runtime: deno 1.36.1 (x86_64-unknown-linux-gnu)
benchmark time (avg) iter/s (min … max) p75 p99 p995
----------------------------------------------------------------------------- -----------------------------
Request without headers 1.86 µs/iter 536,440.7 (1.67 µs … 2.76 µs) 1.89 µs 2.76 µs 2.76 µs
Request with headers 1.96 µs/iter 509,440.5 (1.83 µs … 2.17 µs) 1.99 µs 2.17 µs 2.17 µs
```
**main**
```
cpu: 13th Gen Intel(R) Core(TM) i9-13900H
runtime: deno 1.36.1 (x86_64-unknown-linux-gnu)
benchmark time (avg) iter/s (min … max) p75 p99 p995
----------------------------------------------------------------------------- -----------------------------
Request without headers 1.96 µs/iter 510,201.5 (1.81 µs … 2.64 µs) 2 µs 2.64 µs 2.64 µs
Request with headers 2.03 µs/iter 493,526.6 (1.84 µs … 2.31 µs) 2.08 µs 2.31 µs 2.31 µs
```
```js
Deno.bench("Request without headers", () => {
const r = new Request("https://deno.land", {
method: "POST",
body: '{"foo": "bar"}',
});
});
Deno.bench("Request with headers", () => {
const r = new Request("https://deno.land", {
method: "POST",
body: '{"foo": "bar"}',
headers: {
"Content-Type": "application/json",
},
});
});
```
|
|
Closes https://github.com/denoland/deno/issues/19983
Closes https://github.com/denoland/deno/issues/18303
Closes https://github.com/denoland/deno/issues/16681
Closes https://github.com/denoland/deno/issues/19978
|
|
Fixes https://github.com/denoland/deno/issues/19540
|
|
Fixes https://github.com/denoland/deno/issues/19935
|
|
|
|
Closes https://github.com/denoland/deno/issues/20075
|
|
This PR improves performance of `Deno.Serve` when providing `info`
argument by creating `ServeHandlerInfo` class instead of creating an
object literal with a getter on every request.
```js
Deno.serve((_req, info) => new Response(info.remoteAddr.transport) });
```
### Benchmarks
```
wrk -d 10s --latency http://127.0.0.1:4500
Running 10s test @ http://127.0.0.1:4500
2 threads and 10 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 42.34us 16.30us 1.66ms 95.88%
Req/Sec 118.17k 2.95k 127.38k 76.73%
Latency Distribution
50% 38.00us
75% 41.00us
90% 56.00us
99% 83.00us
2375298 requests in 10.10s, 319.40MB read
Requests/sec: 235177.04
Transfer/sec: 31.62MB
```
**main**
```
wrk -d 10s --latency http://127.0.0.1:4500
Running 10s test @ http://127.0.0.1:4500
2 threads and 10 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 78.86us 211.06us 3.58ms 96.52%
Req/Sec 105.90k 4.35k 117.41k 78.22%
Latency Distribution
50% 41.00us
75% 53.00us
90% 62.00us
99% 1.18ms
2127534 requests in 10.10s, 286.09MB read
Requests/sec: 210647.49
Transfer/sec: 28.33MB
```
```
cpu: 13th Gen Intel(R) Core(TM) i9-13900H
runtime: deno 1.36.0 (x86_64-unknown-linux-gnu)
benchmark time (avg) iter/s (min … max) p75 p99 p995
-------------------------------------------------------------------------- -----------------------------
new ServeHandlerInfo 3.43 ns/iter 291,508,889.3 (3.07 ns … 12.21 ns) 3.42 ns 3.84 ns 3.87 ns
{} with getter 133.84 ns/iter 7,471,528.9 (92.9 ns … 458.95 ns) 132.45 ns 364.96 ns 429.43 ns
```
----
### Drawbacks:
`.remoteAddr` is now not enumerable
```
ServeHandlerInfo {}
```
vs
```
{ remoteAddr: [Getter] }
```
It'll break any code trying to iterate through `info` keys (Doubt
there's anyone doing it though)
```js
Deno.serve((req, info) => {
console.log(Object.keys(info).length === 0) // true;
return new Response("yes");
});
|
|
This PR optimizes the `getHeader` function by replacing `.filter` and
`.map` with a `for` loop
**this patch**
```
cpu: 13th Gen Intel(R) Core(TM) i9-13900H
runtime: deno 1.36.0 (x86_64-unknown-linux-gnu)
benchmark time (avg) iter/s (min … max) p75 p99 p995
----------------------------------------------------------------- -----------------------------
Headers.get 132.2 ns/iter 7,564,093.4 (125.81 ns … 147.66 ns) 133.79 ns 144.92 ns 145.36 ns
```
**main**
```
cpu: 13th Gen Intel(R) Core(TM) i9-13900H
runtime: deno 1.36.0 (x86_64-unknown-linux-gnu)
benchmark time (avg) iter/s (min … max) p75 p99 p995
----------------------------------------------------------------- -----------------------------
Headers.get 191.48 ns/iter 5,222,523.6 (182.75 ns … 212.22 ns) 193.5 ns 205.96 ns 211.51 ns
```
```js
const headers = new Headers({
"Content-Type": "application/json",
"Date": "Thu, 10 Aug 2023 07:45:10 GMT",
"X-Deno": "Deno",
"Powered-By": "Deno",
"Content-Encoding": "gzip",
"Set-Cookie": "__Secure-ID=123; Secure; Domain=example.com",
"Content-Length": "150",
"Vary": "Accept-Encoding, Accept, X-Requested-With",
});
Deno.bench("Headers.get", () => {
const i = headers.get("x-deno");
});
```
|
|
Co-authored-by: denobot <33910674+denobot@users.noreply.github.com>
Co-authored-by: littledivy <littledivy@users.noreply.github.com>
|
|
Fixes https://github.com/denoland/deno/issues/19041
|
|
Fixes https://github.com/denoland/deno/issues/19866
|
|
Closes https://github.com/denoland/deno/issues/20076
|
|
Handles ASCCI espace chars in test and bench name making
test and bench reporting more reliable. This one is also tested
in the fixture of "node:test" module.
|
|
|
|
Mac builds are occasionally flaking out on these.
|
|
The original implementation of `Cache` used a custom `shutdown` method
on the resource, but to simplify fast streams work we're going to move
this to an op of its own.
While we're in here, we're going to replace `opAsync` with
`ensureFastOps`. `op2` work will have to wait because of some
limitations to our async support, however.
|
|
interface (#20100)
This is unused and will allow us to remove `FetchRequestBodyResource` in
a future PR.
|
|
Rename some of the helper methods on the Fs trait to be suffixed with
`_sync` / `_async`, in preparation of the introduction of more async
methods for some helpers.
Also adds a `read_text_file_async` helper to complement the renamed
`read_text_file_sync` helper.
|
|
This commit moves `snapshot_from_lockfile` function to [deno_npm
crate](https://github.com/denoland/deno_npm). This allows this function
to be called outside Deno CLI (in particular, Deno Deploy).
|
|
Uses https://github.com/denoland/deno_cache/pull/26
|
|
(#20093)
This is for when semiColons: false
Closes #20089
|
|
This PR ensures that the original signal event is fired before any
dependent signal events.
---
The enabled tests fail on `main`:
```
assert_array_equals: Abort events fired in correct order expected property 0 to be
"original-aborted" but got "clone-aborted" (expected array ["original-aborted", "clone-aborted"]
got ["clone-aborted", "original-aborted"])
```
|
|
Not sure why `40_testing.js` is there. The other two `00_typescript.js`
and `99_main_compiler.js` should be covered by
`files_loaded_during_snapshot` at the end.
This helps with `__runtime_js_sources` wrt changing `40_testing.js`.
|
|
Renames the unstable `deno_modules` directory and corresponding settings
to `vendor` after feedback. Also causes the vendoring of the
`node_modules` directory which can be disabled via
`--node-modules-dir=false` or `"nodeModulesDir": false`.
|
|
directories (#20070)
This changes the design of the manifest.json file to have a separate
"folders" map for mapping hashed directories. This allows, for example,
to add files in a folder like `http_localhost_8000/#testing_5de71/` and
have them be resolved automatically as long as their remaining
components are identity-mappable to the file system (not hashed). It
also saves space in the manifest.json file by only including the hashed
directory instead of each descendant file.
```
// manifest.json
{
"folders": {
"https://localhost/NOT_MAPPABLE/": "localhost/#not_mappable_5cefgh"
},
"modules": {
"https://localhost/folder/file": {
"headers": {
"content-type": "application/javascript"
}
},
}
}
// folder structure
localhost
- folder
- #file_2defn (note: I've made up the hashes in these examples)
- #not_mappable_5cefgh
- mod.ts
- etc.ts
- more_files.ts
```
|
|
Closes #19399 (running without snapshots at all was suggested as an
alternative solution).
Adds a `__runtime_js_sources` pseudo-private feature to load extension
JS sources at runtime for faster development, instead of building and
loading snapshots or embedding sources in the binary. Will only work in
a development environment obviously.
Try running `cargo test --features __runtime_js_sources
integration::node_unit_tests::os_test`. Then break some behaviour in
`ext/node/polyfills/os.ts` e.g. make `function cpus() {}` return an
empty array, and run it again. Fix and then run again. No more build
time in between.
|
|
|
|
Extracted from PR #16011
|
|
This PR exposes garbage collector for WPT
see:
https://github.com/web-platform-tests/wpt/blob/3d80f7e87928e2d0da25d1c60a13dab001332139/common/gc.js#L34-L36
```
/streams/readable-streams/garbage-collection.any.html
test stderr:
Tests are running without the ability to do manual garbage collection. They will still work, but coverage will be suboptimal.
Tests are running without the ability to do manual garbage collection. They will still work, but coverage will be suboptimal.
Tests are running without the ability to do manual garbage collection. They will still work, but coverage will be suboptimal.
Tests are running without the ability to do manual garbage collection. They will still work, but coverage will be suboptimal.
file result: ok. 4 passed; 0 failed; 0 expected failure; total 4 (255ms)
----------------------------------------
/streams/readable-streams/garbage-collection.any.worker.html
test stderr:
Tests are running without the ability to do manual garbage collection. They will still work, but coverage will be suboptimal.
Tests are running without the ability to do manual garbage collection. They will still work, but coverage will be suboptimal.
Tests are running without the ability to do manual garbage collection. They will still work, but coverage will be suboptimal.
Tests are running without the ability to do manual garbage collection. They will still work, but coverage will be suboptimal.
file result: ok. 4 passed; 0 failed; 0 expected failure; total 4 (277ms)
```
This PR removes that warning and improves coverage.
|
|
directory names (#20069)
Meant to do this earlier.
|
|
Use brotli's fastest mode rather than default mode
|
|
This bumps `async-compression` dependency in `deno_http` to latest, in
order to avoid having multiple duplicate versions.
Related, it also unpin a stale `flate2` dependency so that the whole
chain of `async-compression` -> `flate2` -> `miniz_oxide` can surface up
to current versions.
The lockfile entries for all of the above crates have been update
accordingly; the new tree of dependencies looks like this:
```
$ cargo tree -i -p miniz_oxide
miniz_oxide v0.7.1
└── flate2 v1.0.26
└── async-compression v0.4.1
```
|
|
This tweaks the HTTP response-writer in order to align the two possible
execution flows into using the same gzip default compression level, that
is `1` (otherwise the implicit default level is `6`).
|
|
|
|
Closes https://github.com/denoland/deno/issues/19777
|
|
Ref https://github.com/denoland/deno/issues/19733
|
|
This PR fixes some crashing WPT tests due to an unresolved promise.
---
This could be a [stream spec](https://streams.spec.whatwg.org) bug
When `controller.close` is called on a byob stream, there's no cleanup
of pending `readIntoRequests`. The only cleanup of pending
`readIntoRequests` happen when `.byobRequest.respond(0)` is called, it
happens
here:https://github.com/denoland/deno/blob/6ba245fe2570b29e35a4fd296a196a58870b1e3c/ext/web/06_streams.js#L2026
which ends up calling `readIntoRequest.closeSteps(chunk);` in
https://github.com/denoland/deno/blob/6ba245fe2570b29e35a4fd296a196a58870b1e3c/ext/web/06_streams.js#L2070
To reproduce:
```js
async function byobRead() {
const input = [new Uint8Array([8, 241, 48, 123, 151])];
const stream = new ReadableStream({
type: "bytes",
async pull(controller) {
if(input.length === 0) {
controller.close();
// controller.byobRequest.respond(0); // uncomment for fix
return
}
controller.enqueue(input.shift())
},
});
const reader = stream.getReader({ mode: 'byob' });
const r1 = await reader.read(new Uint8Array(64));
console.log(r1);
const r2 = await reader.read(new Uint8Array(64));
console.log(r2);
}
await byobRead();
```
Running the script triggers:
```
error: Top-level await promise never resolved
```
|
|
Chipping away at making tests faster. Appears we don't need double
timeout before sanitizing ops. This should cut baseline cost of running a test
by half.
|
|
Ref #19915
---------
Co-authored-by: Matt Mastracci <matthew@mastracci.com>
|
|
This is a prerequisite for fast streams work -- this particular resource
used a custom `mpsc`-style stream, and this work will allow us to unify
it with the streams in `ext/http` in time.
Instead of using Option as an internal semaphore for "correctly
completed EOF", we allow code to propagate errors into the channel which
can be picked up by downstream sinks like Hyper. EOF is signalled using
a more standard sender drop.
|
|
Many of the CI tests have been failing on my M2 Pro mac (Ventura 13.4)
when running inside of a vscode terminal (a strange `ENOTTY` error).
This modifies the pty-handling code to use libc directly rather than the
older pty library that appears mostly unmaintained (outside of
@littledivy's fork).
As a bonus, this should allow us to run pty tests on the mac CI runner.
After this PR, the tests now complete with 100% success on my local
machine. Before this PR, I needed to pass `CI=true` to get my local test
suite to pass.
|
|
Co-authored-by: bartlomieju <bartlomieju@users.noreply.github.com>
|
|
This commit adds new "--deny-*" permission flags. These are complimentary to
"--allow-*" flags.
These flags can be used to restrict access to certain resources, even if they
were granted using "--allow-*" flags or the "--allow-all" ("-A") flag.
Eg. specifying "--allow-read --deny-read" will result in a permission error,
while "--allow-read --deny-read=/etc" will allow read access to all FS but the
"/etc" directory.
Runtime permissions APIs ("Deno.permissions") were adjusted as well, mainly
by adding, a new "PermissionStatus.partial" field. This field denotes that
while permission might be granted to requested resource, it's only partial (ie.
a "--deny-*" flag was specified that excludes some of the requested resources).
Eg. specifying "--allow-read=foo/ --deny-read=foo/bar" and then querying for
permissions like "Deno.permissions.query({ name: "read", path: "foo/" })"
will return "PermissionStatus { state: "granted", onchange: null, partial: true }",
denoting that some of the subpaths don't have read access.
Closes #18804.
---------
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
Co-authored-by: Nayeem Rahman <nayeemrmn99@gmail.com>
|
|
This commit adds "--reporter" and "--junit-path" flags to "deno test"
subcommand instead of using "--dot" and "--junit" flags.
|
|
Closes #20015
Closes https://github.com/denoland/vscode_deno/issues/850 (only for
deno_modules, but I don't think this will be possible for the global
cache)
|
|
|
|
This commit adds a "dot" reporter to "deno test" subcommand,
that can be activated using "--dot" flag.
It provides a concise output using:
- "." for passing test
- "," for ignored test
- "!" for failing test
User output is silenced and not printed to the console.
In non-TTY environments each result is printed on a separate line.
|
|
Also removed some noisy output that caused test flakiness.
|
|
Ref https://github.com/denoland/deno/pull/19994#discussion_r1281267606
Ref https://github.com/denoland/deno/pull/19994#discussion_r1281243602
|
