summaryrefslogtreecommitdiff
path: root/ext/fetch/lib.rs
AgeCommit message (Collapse)Author
2024-01-01chore: update copyright to 2024 (#21753)David Sherret
2023-12-31fix(http_client): Fix Deno.createHttpClient to accept poolIdleTimeout ↵Raashid Anwar
parameter (#21603) Fixed the bug `Deno.createHttpClient` to accept `poolIdleTimeout` parameter. Fixes https://github.com/denoland/deno/issues/21546
2023-12-27refactor: simplify hyper, http, h2 deps (#21715)Bartek Iwańczuk
Main change is that: - "hyper" has been renamed to "hyper_v014" to signal that it's legacy - "hyper1" has been renamed to "hyper" and should be the default
2023-12-23chore(ext/node): use BufView natively in http2 (#21688)Matt Mastracci
Node HTTP/2 was using the default h2 `Bytes` datatype when we can be making using of `BufView` like we do in `Deno.serve`. `fetch` and `Deno.serverHttp` can't make use of `BufView` because they are using `reqwest` which is stuck on hyper 0.x at this time.
2023-12-01refactor: use resourceForReadableStream for fetch (#20217)Matt Mastracci
Switch `ext/fetch` over to `resourceForReadableStream` to simplify and unify implementation with `ext/serve`. This allows us to work in Rust with resources only. Two additional changes made to `resourceForReadableStream` were required: - Add an optional length to `resourceForReadableStream` which translates to `size_hint` - Fix a bug where writing to a closed stream that was full would panic
2023-11-01feat(ext/websocket): websockets over http2 (#21040)Matt Mastracci
Implements `WebSocket` over http/2. This requires a conformant http/2 server supporting the extended connect protocol. Passes approximately 100 new WPT tests (mostly `?wpt_flags=h2` versions of existing websockets APIs). This is implemented as a fallback when http/1.1 fails, so a server that supports both h1 and h2 WebSockets will still end up on the http/1.1 upgrade path. The patch also cleas up the websockets handshake to split it up into http, https+http1 and https+http2, making it a little less intertwined. This uncovered a likely bug in the WPT test server: https://github.com/web-platform-tests/wpt/issues/42896
2023-10-31feat(ext/web): EventSource (#14730)Leo Kettmeir
Closes #10298 --------- Co-authored-by: Aapo Alasuutari <aapo.alasuutari@gmail.com>
2023-09-23refactor: rewrite ops using i64/usize to op2 (#20647)Bartek Iwańczuk
2023-09-21refactor: rewrite some ops to op2 macro (#20603)Bartek Iwańczuk
2023-08-23fix(ext/web): add stream tests to detect v8slice split bug (#20253)Matt Mastracci
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
2023-08-09refactor(ext/fetch): Remove FetchRequestBodyResource from FetchHandler ↵Matt Mastracci
interface (#20100) This is unused and will allow us to remove `FetchRequestBodyResource` in a future PR.
2023-08-03refactor(ext/fetch): refactor fetch to use new write_error method (#20029)Matt Mastracci
This is a prerequisite for fast streams work -- this particular resource used a custom `mpsc`-style stream, and this work will allow us to unify it with the streams in `ext/http` in time. Instead of using Option as an internal semaphore for "correctly completed EOF", we allow code to propagate errors into the channel which can be picked up by downstream sinks like Hyper. EOF is signalled using a more standard sender drop.
2023-07-28feat: Deno.createHttpClient allowHost (#19689)Leo Kettmeir
This adds an option to allow using the host header in a fetch call. Closes https://github.com/denoland/deno/issues/16840 Ref https://github.com/denoland/deno/issues/11017
2023-06-22refactor(serde_v8): split ZeroCopyBuf into JsBuffer and ToJsBuffer (#19566)Bartek Iwańczuk
`ZeroCopyBuf` was convenient to use, but sometimes it did hide details that some copies were necessary in certain cases. Also it made it way to easy for the caller to pass around and convert into different values. This commit splits `ZeroCopyBuf` into `JsBuffer` (an array buffer coming from V8) and `ToJsBuffer` (a Rust buffer that will be converted into a V8 array buffer). As a result some magical conversions were removed (they were never used) limiting the API surface and preparing for changes in #19534.
2023-06-15refactor(ext/fetch): simplify fetch ops (#19494)Bartek Iwańczuk
Addresses feedback from https://github.com/denoland/deno/pull/19412#discussion_r1227912676
2023-06-13fix(ext/node): handle 'upgrade' responses (#19412)Bartek Iwańczuk
This commit adds support for "upgrade" events in "node:http" "ClientRequest". Currently only "Websocket" upgrades are handled. Thanks to this change package like "npm:puppeteer" and "npm:discord" should work. Closes https://github.com/denoland/deno/issues/18913 Closes https://github.com/denoland/deno/issues/17847
2023-05-29fix: use proper ALPN protocols if HTTP client is HTTP/1.1 only (#19303)Bartek Iwańczuk
Closes https://github.com/denoland/deno/issues/16923 --------- Co-authored-by: crowlkats <crowlkats@toaxl.com> Co-authored-by: Matt Mastracci <matthew@mastracci.com>
2023-05-21feat(unstable): add more options to Deno.createHttpClient (#17385)Leo Kettmeir
2023-05-17refactor(node): reimplement http client (#19122)Leo Kettmeir
This commit reimplements most of "node:http" client APIs using "ext/fetch". There is some duplicated code and two removed Node compat tests that will be fixed in follow up PRs. --------- Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
2023-05-01perf: lazily create RootCertStore (#18938)David Sherret
2023-03-23refactor: make version and user_agent &'static str (#18400)Bartek Iwańczuk
These caused a bunch of unnecessary allocations on each startup.
2023-03-17perf(core) Reduce copying and cloning in extension initialization (#18252)Matt Mastracci
Follow-up to #18210: * we are passing the generated `cfg` object into the state function rather than passing individual config fields * reduce cloning dramatically by making the state_fn `FnOnce` * `take` for `ExtensionBuilder` to avoid more unnecessary copies * renamed `config` to `options`
2023-03-17feat(core) deno_core::extension! macro to simplify extension registration ↵Matt Mastracci
(#18210) This implements two macros to simplify extension registration and centralize a lot of the boilerplate as a base for future improvements: * `deno_core::ops!` registers a block of `#[op]`s, optionally with type parameters, useful for places where we share lists of ops * `deno_core::extension!` is used to register an extension, and creates two methods that can be used at runtime/snapshot generation time: `init_ops` and `init_ops_and_esm`. --------- Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
2023-03-13feat(ext/fetch): support fallible request-builder hooks (#18116)Luca Bruno
This tweaks the signature of `request_builder_hook` in order to support fallible hooks. The rationale for this is mostly on two sides: * it allows a hook to inspect and possibly drop an outgoing request (e.g. for policying purposes), bubbling up a detailed error message to the user. * it wires into newer `reqwest` API which allows to split and then reassemble a `RequestBuilder`, although only in a fallible way (https://github.com/seanmonstar/reqwest/pull/1770)
2023-03-09refactor(core): remove RuntimeOptions::extensions_with_js (#18099)Bartek Iwańczuk
This commit removes "deno_core::RuntimeOptions::extensions_with_js". Now it's embedders' responsibility to properly register extensions that will not contains JavaScript sources when running from an existing snapshot. Prerequisite for https://github.com/denoland/deno/pull/18080
2023-03-09refactor: Split extension registration for runtime and snapshotting (#18095)Bartek Iwańczuk
This commit splits "<ext_name>::init" functions into "init_ops" and "init_ops_and_esm". That way we don't have to construct list of ESM sources on each startup if we're running with a snapshot. In a follow up commit "deno_core" will be changed to not have a split between "extensions" and "extensions_with_js" - it will be embedders' responsibility to pass appropriately configured extensions. Prerequisite for https://github.com/denoland/deno/pull/18080
2023-03-09refactor(core): Extension::builder_with_deps (#18093)Bartek Iwańczuk
Prerequisite for https://github.com/denoland/deno/pull/18080
2023-03-07refactor(core): don't use Result in ExtensionBuilder::state (#18066)Bartek Iwańczuk
There's no point for this API to expect result. If something fails it should result in a panic during build time to signal to embedder that setup is wrong.
2023-02-07 refactor: remove prefix from include_js_files & use extension name (#17683)Leo Kettmeir
2023-02-07refactor: Use ES modules for internal runtime code (#17648)Leo Kettmeir
This PR refactors all internal js files (except core) to be written as ES modules. `__bootstrap`has been mostly replaced with static imports in form in `internal:[path to file from repo root]`. To specify if files are ESM, an `esm` method has been added to `Extension`, similar to the `js` method. A new ModuleLoader called `InternalModuleLoader` has been added to enable the loading of internal specifiers, which is used in all situations except when a snapshot is only loaded, and not a new one is created from it. --------- Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
2023-02-05 refactor: rename `deno` specifiers to `internal` (#17655)Leo Kettmeir
2023-01-27chore: upgrade to Rust 1.67 (#17548)David Sherret
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
2023-01-14chore: use rustfmt imports_granularity option (#17421)Divy Srivastava
Closes https://github.com/denoland/deno/issues/2699 Closes https://github.com/denoland/deno/issues/2347 Uses unstable rustfmt features. Since dprint invokes `rustfmt` we do not need to switch the cargo toolchain to nightly. Do we care about formatting stability of our codebase across Rust versions? (I don't)
2023-01-08feat(core): allow specifying name and dependencies of an Extension (#17301)Leo Kettmeir
2023-01-02chore: update copyright year to 2023 (#17247)David Sherret
Yearly tradition of creating extra noise in git.
2022-12-23fix(ext/fetch): Guard against invalid URL before its used by reqwest (#17164)Kamil Ogórek
2022-12-19fix(ext/fetch): handle errors in req body stream (#17081)Luca Casonato
Right now an error in a request body stream causes an uncatchable global promise rejection. This PR fixes this to instead propagate the error correctly into the promise returned from `fetch`. It additionally fixes errored readable stream bodies being treated as successfully completed bodies by Rust.
2022-10-24fix(ext/fetch): fix `size_hint` on response body resource (#16254)Marcos Casagrande
2022-10-17fix(ext/fetch): set accept-encoding: identity if range header is present ↵Marcos Casagrande
(#16197) https://fetch.spec.whatwg.org/#http-network-or-cache-fetch > If httpRequest’s header list contains `Range`, then append (`Accept-Encoding`, `identity`) > to httpRequest’s header list. > > This avoids a failure when handling content codings with a part of an encoded response. > Additionally, many servers mistakenly ignore `Range` headers if a non-identity encoding is accepted.
2022-10-12fix(ext/fetch): throw TypeError on read failure (#16219)Marcos Casagrande
2022-10-09feat(core): improve resource read & write traits (#16115)Luca Casonato
This commit introduces two new buffer wrapper types to `deno_core`. The main benefit of these new wrappers is that they can wrap a number of different underlying buffer types. This allows for a more flexible read and write API on resources that will require less copying of data between different buffer representations. - `BufView` is a read-only view onto a buffer. It can be backed by `ZeroCopyBuf`, `Vec<u8>`, and `bytes::Bytes`. - `BufViewMut` is a read-write view onto a buffer. It can be cheaply converted into a `BufView`. It can be backed by `ZeroCopyBuf` or `Vec<u8>`. Both new buffer views have a cursor. This means that the start point of the view can be constrained to write / read from just a slice of the view. Only the start point of the slice can be adjusted. The end point is fixed. To adjust the end point, the underlying buffer needs to be truncated. Readable resources have been changed to better cater to resources that do not support BYOB reads. The basic `read` method now returns a `BufView` instead of taking a `ZeroCopyBuf` to fill. This allows the operation to return buffers that the resource has already allocated, instead of forcing the caller to allocate the buffer. BYOB reads are still very useful for resources that support them, so a new `read_byob` method has been added that takes a `BufViewMut` to fill. `op_read` attempts to use `read_byob` if the resource supports it, which falls back to `read` and performs an additional copy if it does not. For Rust->JS reads this change should have no impact, but for Rust->Rust reads, this allows the caller to avoid an additional copy in many scenarios. This combined with the support for `BufView` to be backed by `bytes::Bytes` allows us to avoid one data copy when piping from a `fetch` response into an `ext/http` response. Writable resources have been changed to take a `BufView` instead of a `ZeroCopyBuf` as an argument. This allows for less copying of data in certain scenarios, as described above. Additionally a new `Resource::write_all` method has been added that takes a `BufView` and continually attempts to write the resource until the entire buffer has been written. Certain resources like files can override this method to provide a more efficient `write_all` implementation.
2022-10-04perf(ext/fetch): consume body using ops (#16038)Marcos Casagrande
This commit adds a fast path to `Request` and `Response` that make consuming request bodies much faster when using `Body#text`, `Body#arrayBuffer`, and `Body#blob`, if the body is a FastStream. Because the response bodies for `fetch` are FastStream, this speeds up consuming `fetch` response bodies significantly.
2022-09-27feat: Add requesting API name to permission prompt (#15936)Bartek Iwańczuk
Co-authored-by: Leo Kettmeir <crowlkats@toaxl.com>
2022-09-26perf(ext/fetch): use content-length in InnerBody.consume (#15925)Marcos Casagrande
This fast path prevents repeated allocations when receiving a fetch body with a known size. Co-authored-by: Luca Casonato <hello@lcas.dev>
2022-08-23fix(ext/fetch): ignore user content-length header (#15555)Luca Casonato
Previously if a user specified a content-length header for an POST request without a body, the request would contain two `content-length` headers. One added by us, and one added by the user. This commit ignores all content-length headers coming from the user, because we need to have the sole authority on the content-length because we transmit the body.
2022-04-23chore(ext/fetch): custom arity (#14198)Divy Srivastava
2022-04-22Reland "perf(http): optimize ReadableStreams backed by a resource" (#14346)Divy Srivastava
2022-04-21Revert various PRs related to "ext/http" (#14339)Bartek Iwańczuk
* Revert "feat(ext/http): stream auto resp body compression (#14325)" * Revert "core: introduce `resource.read_return` (#14331)" * Revert "perf(http): optimize `ReadableStream`s backed by a resource (#14284)"
2022-04-20core: introduce `resource.read_return` (#14331)Divy Srivastava
2022-04-02experiment(serde_v8): derive_more enabled opaque wrappers (#14096)Aaron O'Mullan