From dcbbcd23f5dd8601e2851aded4cabc6557164363 Mon Sep 17 00:00:00 2001 From: Matt Mastracci Date: Fri, 9 Feb 2024 13:33:05 -0700 Subject: refactor: split integration tests from CLI (part 1) (#22308) This PR separates integration tests from CLI tests into a new project named `cli_tests`. This is a prerequisite for an integration test runner that can work with either the CLI binary in the current project, or one that is built ahead of time. ## Background Rust does not have the concept of artifact dependencies yet (https://github.com/rust-lang/cargo/issues/9096). Because of this, the only way we can ensure a binary is built before running associated tests is by hanging tests off the crate with the binary itself. Unfortunately this means that to run those tests, you _must_ build the binary and in the case of the deno executable that might be a 10 minute wait in release mode. ## Implementation To allow for tests to run with and without the requirement that the binary is up-to-date, we split the integration tests into a project of their own. As these tests would not require the binary to build itself before being run as-is, we add a stub integration `[[test]]` target in the `cli` project that invokes these tests using `cargo test`. The stub test runner we add has `harness = false` so that we can get access to a `main` function. This `main` function's sole job is to `execvp` the command `cargo test -p deno_cli`, effectively "calling" another cargo target. This ensures that the deno executable is always correctly rebuilt before running the stub test runner from `cli`, and gets us closer to be able to run the entire integration test suite on arbitrary deno executables (and therefore split the build into multiple phases). The new `cli_tests` project lives within `cli` to avoid a large PR. In later PRs, the test data will be split from the `cli` project. As there are a few thousand files, it'll be better to do this as a completely separate PR to avoid noise. --- .github/workflows/ci.generate.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to '.github/workflows/ci.generate.ts') diff --git a/.github/workflows/ci.generate.ts b/.github/workflows/ci.generate.ts index e47435b7e..26ca89316 100755 --- a/.github/workflows/ci.generate.ts +++ b/.github/workflows/ci.generate.ts @@ -791,7 +791,7 @@ const ci = { // Run unit then integration tests. Skip doc tests here // since they are sometimes very slow on Mac. "cargo test --locked --lib", - "cargo test --locked --test '*'", + "cargo test --locked --tests", ].join("\n"), env: { CARGO_PROFILE_DEV_DEBUG: 0 }, }, -- cgit v1.2.3