feat(bench): update API, new console reporter (#14305)

This commit changes "deno bench" subcommand, by updating
the "Deno.bench" API as follows:
- remove "Deno.BenchDefinition.n"
- remove "Deno.BenchDefintion.warmup"
- add "Deno.BenchDefinition.group"
- add "Deno.BenchDefintion.baseline"

This is done because bench cases are no longer run fixed amount
of iterations, but instead they are run until there is difference between
subsequent runs that is statistically insiginificant.

Additionally, console reporter was rewritten completely, to looks
similar to "hyperfine" reporter.

25 files changed, 229 insertions, 300 deletions

diff --git a/cli/tests/integration/bench_tests.rs b/cli/tests/integration/bench_tests.rs
index e889a8224..928f08bea 100644
--- a/cli/tests/integration/bench_tests.rs
+++ b/cli/tests/integration/bench_tests.rs
@@ -111,6 +111,12 @@ itest!(finally_timeout {
   output: "bench/finally_timeout.out",
 });
 
+itest!(group_baseline {
+  args: "bench --unstable bench/group_baseline.ts",
+  exit_code: 0,
+  output: "bench/group_baseline.out",
+});
+
 itest!(unresolved_promise {
   args: "bench --unstable bench/unresolved_promise.ts",
   exit_code: 1,
diff --git a/cli/tests/testdata/bench/allow_all.out b/cli/tests/testdata/bench/allow_all.out
index eb7d2005c..c4a60fcf5 100644
--- a/cli/tests/testdata/bench/allow_all.out
+++ b/cli/tests/testdata/bench/allow_all.out
@@ -1,18 +1,21 @@
-[WILDCARD]
-running 14 benches from [WILDCARD]
-bench read false ... 1000 iterations [WILDCARD] ns/iter ([WILDCARD]..[WILDCARD] ns/iter) ok [WILDCARD]
-bench read true ... 1000 iterations [WILDCARD] ns/iter ([WILDCARD]..[WILDCARD] ns/iter) ok [WILDCARD]
-bench write false ... 1000 iterations [WILDCARD] ns/iter ([WILDCARD]..[WILDCARD] ns/iter) ok [WILDCARD]
-bench write true ... 1000 iterations [WILDCARD] ns/iter ([WILDCARD]..[WILDCARD] ns/iter) ok [WILDCARD]
-bench net false ... 1000 iterations [WILDCARD] ns/iter ([WILDCARD]..[WILDCARD] ns/iter) ok [WILDCARD]
-bench net true ... 1000 iterations [WILDCARD] ns/iter ([WILDCARD]..[WILDCARD] ns/iter) ok [WILDCARD]
-bench env false ... 1000 iterations [WILDCARD] ns/iter ([WILDCARD]..[WILDCARD] ns/iter) ok [WILDCARD]
-bench env true ... 1000 iterations [WILDCARD] ns/iter ([WILDCARD]..[WILDCARD] ns/iter) ok [WILDCARD]
-bench run false ... 1000 iterations [WILDCARD] ns/iter ([WILDCARD]..[WILDCARD] ns/iter) ok [WILDCARD]
-bench run true ... 1000 iterations [WILDCARD] ns/iter ([WILDCARD]..[WILDCARD] ns/iter) ok [WILDCARD]
-bench ffi false ... 1000 iterations [WILDCARD] ns/iter ([WILDCARD]..[WILDCARD] ns/iter) ok [WILDCARD]
-bench ffi true ... 1000 iterations [WILDCARD] ns/iter ([WILDCARD]..[WILDCARD] ns/iter) ok [WILDCARD]
-bench hrtime false ... 1000 iterations [WILDCARD] ns/iter ([WILDCARD]..[WILDCARD] ns/iter) ok [WILDCARD]
-bench hrtime true ... 1000 iterations [WILDCARD] ns/iter ([WILDCARD]..[WILDCARD] ns/iter) ok [WILDCARD]
+Check [WILDCARD]/bench/allow_all.ts
+cpu: [WILDCARD]
+runtime: deno [WILDCARD] ([WILDCARD])
 
-bench result: ok. 14 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out [WILDCARD]
+[WILDCARD]/bench/allow_all.ts
+benchmark         time (avg)             (min … max)       p75       p99      p995
+---------------------------------------------------- -----------------------------
+read false [WILDCARD] [WILDCARD]/iter[WILDCARD]([WILDCARD] … [WILDCARD]) [WILDCARD]
+read true [WILDCARD] [WILDCARD]/iter[WILDCARD]([WILDCARD] … [WILDCARD]) [WILDCARD]
+write false [WILDCARD] [WILDCARD]/iter[WILDCARD]([WILDCARD] … [WILDCARD]) [WILDCARD]
+write true [WILDCARD] [WILDCARD]/iter[WILDCARD]([WILDCARD] … [WILDCARD]) [WILDCARD]
+net false [WILDCARD] [WILDCARD]/iter[WILDCARD]([WILDCARD] … [WILDCARD]) [WILDCARD]
+net true [WILDCARD] [WILDCARD]/iter[WILDCARD]([WILDCARD] … [WILDCARD]) [WILDCARD]
+env false [WILDCARD] [WILDCARD]/iter[WILDCARD]([WILDCARD] … [WILDCARD]) [WILDCARD]
+env true [WILDCARD] [WILDCARD]/iter[WILDCARD]([WILDCARD] … [WILDCARD]) [WILDCARD]
+run false [WILDCARD] [WILDCARD]/iter[WILDCARD]([WILDCARD] … [WILDCARD]) [WILDCARD]
+run true [WILDCARD] [WILDCARD]/iter[WILDCARD]([WILDCARD] … [WILDCARD]) [WILDCARD]
+ffi false [WILDCARD] [WILDCARD]/iter[WILDCARD]([WILDCARD] … [WILDCARD]) [WILDCARD]
+ffi true [WILDCARD] [WILDCARD]/iter[WILDCARD]([WILDCARD] … [WILDCARD]) [WILDCARD]
+hrtime false [WILDCARD] [WILDCARD]/iter[WILDCARD]([WILDCARD] … [WILDCARD]) [WILDCARD]
+hrtime true [WILDCARD] [WILDCARD]/iter[WILDCARD]([WILDCARD] … [WILDCARD]) [WILDCARD]
diff --git a/cli/tests/testdata/bench/allow_none.out b/cli/tests/testdata/bench/allow_none.out
index 0eb2ba5a3..3fd649ebe 100644
--- a/cli/tests/testdata/bench/allow_none.out
+++ b/cli/tests/testdata/bench/allow_none.out
@@ -1,51 +1,22 @@
-[WILDCARD]
-running 7 benches from [WILDCARD]
-bench read ... 1000 iterations FAILED [WILDCARD]
-bench write ... 1000 iterations FAILED [WILDCARD]
-bench net ... 1000 iterations FAILED [WILDCARD]
-bench env ... 1000 iterations FAILED [WILDCARD]
-bench run ... 1000 iterations FAILED [WILDCARD]
-bench ffi ... 1000 iterations FAILED [WILDCARD]
-bench hrtime ... 1000 iterations FAILED [WILDCARD]
-
-failures:
+Check [WILDCARD]/bench/allow_none.ts
+cpu: [WILDCARD]
+runtime: deno [WILDCARD] ([WILDCARD])
 
-read
-PermissionDenied: Can't escalate parent thread permissions
+[WILDCARD]/bench/allow_none.ts
+benchmark      time (avg)             (min … max)       p75       p99      p995
+------------------------------------------------- -----------------------------
+read       error: PermissionDenied: Can't escalate parent thread permissions
 [WILDCARD]
-
-write
-PermissionDenied: Can't escalate parent thread permissions
+write      error: PermissionDenied: Can't escalate parent thread permissions
 [WILDCARD]
-
-net
-PermissionDenied: Can't escalate parent thread permissions
+net        error: PermissionDenied: Can't escalate parent thread permissions
 [WILDCARD]
-
-env
-PermissionDenied: Can't escalate parent thread permissions
+env        error: PermissionDenied: Can't escalate parent thread permissions
 [WILDCARD]
-
-run
-PermissionDenied: Can't escalate parent thread permissions
+run        error: PermissionDenied: Can't escalate parent thread permissions
 [WILDCARD]
-
-ffi
-PermissionDenied: Can't escalate parent thread permissions
+ffi        error: PermissionDenied: Can't escalate parent thread permissions
 [WILDCARD]
-
-hrtime
-PermissionDenied: Can't escalate parent thread permissions
+hrtime     error: PermissionDenied: Can't escalate parent thread permissions
 [WILDCARD]
-
-failures:
-
-	read
-	write
-	net
-	env
-	run
-	ffi
-	hrtime
-
-bench result: FAILED. 0 passed; 7 failed; 0 ignored; 0 measured; 0 filtered out [WILDCARD]
+error: Bench failed
diff --git a/cli/tests/testdata/bench/clear_timeout.out b/cli/tests/testdata/bench/clear_timeout.out
index 10aa47d75..b66c49718 100644
--- a/cli/tests/testdata/bench/clear_timeout.out
+++ b/cli/tests/testdata/bench/clear_timeout.out
@@ -1,8 +1,10 @@
 Check [WILDCARD]/bench/clear_timeout.ts
-running 3 benches from [WILDCARD]/bench/clear_timeout.ts
-bench bench1 ... 1000 iterations [WILDCARD] ns/iter ([WILDCARD]..[WILDCARD] ns/iter) ok ([WILDCARD])
-bench bench2 ... 1000 iterations [WILDCARD] ns/iter ([WILDCARD]..[WILDCARD] ns/iter) ok ([WILDCARD])
-bench bench3 ... 1000 iterations [WILDCARD] ns/iter ([WILDCARD]..[WILDCARD] ns/iter) ok ([WILDCARD])
-
-bench result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out ([WILDCARD])
+cpu: [WILDCARD]
+runtime: deno [WILDCARD] ([WILDCARD])
 
+[WILDCARD]/bench/clear_timeout.ts
+benchmark      time (avg)             (min … max)       p75       p99      p995
+------------------------------------------------- -----------------------------
+bench1 [WILDCARD] [WILDCARD]/iter[WILDCARD]([WILDCARD] … [WILDCARD]) [WILDCARD]
+bench2 [WILDCARD] [WILDCARD]/iter[WILDCARD]([WILDCARD] … [WILDCARD]) [WILDCARD]
+bench3 [WILDCARD] [WILDCARD]/iter[WILDCARD]([WILDCARD] … [WILDCARD]) [WILDCARD]
diff --git a/cli/tests/testdata/bench/collect.out b/cli/tests/testdata/bench/collect.out
index 570b2e4f2..18b27f8b9 100644
--- a/cli/tests/testdata/bench/collect.out
+++ b/cli/tests/testdata/bench/collect.out
@@ -1,5 +1,7 @@
 Check [WILDCARD]/bench/collect/bench.ts
-running 0 benches from [WILDCARD]/bench/collect/bench.ts
-
-bench result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out ([WILDCARD])
+cpu: [WILDCARD]
+runtime: deno [WILDCARD] ([WILDCARD])
 
+[WILDCARD]/bench/collect/bench.ts
+benchmark      time (avg)             (min … max)       p75       p99      p995
+------------------------------------------------- -----------------------------
diff --git a/cli/tests/testdata/bench/exit_sanitizer.out b/cli/tests/testdata/bench/exit_sanitizer.out
index 23ce871fc..ba063642d 100644
--- a/cli/tests/testdata/bench/exit_sanitizer.out
+++ b/cli/tests/testdata/bench/exit_sanitizer.out
@@ -1,35 +1,14 @@
 Check [WILDCARD]/bench/exit_sanitizer.ts
-running 3 benches from [WILDCARD]/bench/exit_sanitizer.ts
-bench exit(0) ... 1000 iterations FAILED ([WILDCARD])
-bench exit(1) ... 1000 iterations FAILED ([WILDCARD])
-bench exit(2) ... 1000 iterations FAILED ([WILDCARD])
-
-failures:
-
-exit(0)
-AssertionError: Bench attempted to exit with exit code: 0
-    at [WILDCARD]
-    at [WILDCARD]/bench/exit_sanitizer.ts:2:8
-    at [WILDCARD]
-
-exit(1)
-AssertionError: Bench attempted to exit with exit code: 1
-    at [WILDCARD]
-    at [WILDCARD]/bench/exit_sanitizer.ts:6:8
-    at [WILDCARD]
-
-exit(2)
-AssertionError: Bench attempted to exit with exit code: 2
-    at [WILDCARD]
-    at [WILDCARD]/bench/exit_sanitizer.ts:10:8
-    at [WILDCARD]
-
-failures:
-
-	exit(0)
-	exit(1)
-	exit(2)
-
-bench result: FAILED. 0 passed; 3 failed; 0 ignored; 0 measured; 0 filtered out ([WILDCARD])
-
+cpu: [WILDCARD]
+runtime: deno [WILDCARD] ([WILDCARD])
+
+[WILDCARD]/bench/exit_sanitizer.ts
+benchmark      time (avg)             (min … max)       p75       p99      p995
+------------------------------------------------- -----------------------------
+exit(0)    error: AssertionError: Bench attempted to exit with exit code: 0
+[WILDCARD]
+exit(1)    error: AssertionError: Bench attempted to exit with exit code: 1
+[WILDCARD]
+exit(2)    error: AssertionError: Bench attempted to exit with exit code: 2
+[WILDCARD]
 error: Bench failed
diff --git a/cli/tests/testdata/bench/fail.out b/cli/tests/testdata/bench/fail.out
index 9779a27fe..cc92a08b4 100644
--- a/cli/tests/testdata/bench/fail.out
+++ b/cli/tests/testdata/bench/fail.out
@@ -1,81 +1,28 @@
 Check [WILDCARD]/bench/fail.ts
-running 10 benches from [WILDCARD]/bench/fail.ts
-bench bench0 ... 1000 iterations FAILED ([WILDCARD])
-bench bench1 ... 1000 iterations FAILED ([WILDCARD])
-bench bench2 ... 1000 iterations FAILED ([WILDCARD])
-bench bench3 ... 1000 iterations FAILED ([WILDCARD])
-bench bench4 ... 1000 iterations FAILED ([WILDCARD])
-bench bench5 ... 1000 iterations FAILED ([WILDCARD])
-bench bench6 ... 1000 iterations FAILED ([WILDCARD])
-bench bench7 ... 1000 iterations FAILED ([WILDCARD])
-bench bench8 ... 1000 iterations FAILED ([WILDCARD])
-bench bench9 ... 1000 iterations FAILED ([WILDCARD])
-
-failures:
-
-bench0
-Error
-    at [WILDCARD]/bench/fail.ts:2:9
-    at [WILDCARD]
-
-bench1
-Error
-    at [WILDCARD]/bench/fail.ts:5:9
-    at [WILDCARD]
-
-bench2
-Error
-    at [WILDCARD]/bench/fail.ts:8:9
-    at [WILDCARD]
-
-bench3
-Error
-    at [WILDCARD]/bench/fail.ts:11:9
-    at [WILDCARD]
-
-bench4
-Error
-    at [WILDCARD]/bench/fail.ts:14:9
-    at [WILDCARD]
-
-bench5
-Error
-    at [WILDCARD]/bench/fail.ts:17:9
-    at [WILDCARD]
-
-bench6
-Error
-    at [WILDCARD]/bench/fail.ts:20:9
-    at [WILDCARD]
-
-bench7
-Error
-    at [WILDCARD]/bench/fail.ts:23:9
-    at [WILDCARD]
-
-bench8
-Error
-    at [WILDCARD]/bench/fail.ts:26:9
-    at [WILDCARD]
-
-bench9
-Error
-    at [WILDCARD]/bench/fail.ts:29:9
-    at [WILDCARD]
-
-failures:
-
-	bench0
-	bench1
-	bench2
-	bench3
-	bench4
-	bench5
-	bench6
-	bench7
-	bench8
-	bench9
-
-bench result: FAILED. 0 passed; 10 failed; 0 ignored; 0 measured; 0 filtered out ([WILDCARD])
-
+cpu: [WILDCARD]
+runtime: deno [WILDCARD] ([WILDCARD])
+
+[WILDCARD]/bench/fail.ts
+benchmark      time (avg)             (min … max)       p75       p99      p995
+------------------------------------------------- -----------------------------
+bench0     error: Error
+[WILDCARD]
+bench1     error: Error
+[WILDCARD]
+bench2     error: Error
+[WILDCARD]
+bench3     error: Error
+[WILDCARD]
+bench4     error: Error
+[WILDCARD]
+bench5     error: Error
+[WILDCARD]
+bench6     error: Error
+[WILDCARD]
+bench7     error: Error
+[WILDCARD]
+bench8     error: Error
+[WILDCARD]
+bench9     error: Error
+[WILDCARD]
 error: Bench failed
diff --git a/cli/tests/testdata/bench/filter.out b/cli/tests/testdata/bench/filter.out
index 8657e56cc..3356ded99 100644
--- a/cli/tests/testdata/bench/filter.out
+++ b/cli/tests/testdata/bench/filter.out
@@ -1,12 +1,20 @@
 Check [WILDCARD]/bench/filter/a_bench.ts
 Check [WILDCARD]/bench/filter/b_bench.ts
 Check [WILDCARD]/bench/filter/c_bench.ts
-running 1 bench from [WILDCARD]/bench/filter/a_bench.ts
-bench foo ... 1000 iterations [WILDCARD] ns/iter ([WILDCARD]..[WILDCARD] ns/iter) ok ([WILDCARD])
-running 1 bench from [WILDCARD]/bench/filter/b_bench.ts
-bench foo ... 1000 iterations [WILDCARD] ns/iter ([WILDCARD]..[WILDCARD] ns/iter) ok ([WILDCARD])
-running 1 bench from [WILDCARD]/bench/filter/c_bench.ts
-bench foo ... 1000 iterations [WILDCARD] ns/iter ([WILDCARD]..[WILDCARD] ns/iter) ok ([WILDCARD])
+cpu: [WILDCARD]
+runtime: deno [WILDCARD] ([WILDCARD])
 
-bench result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 6 filtered out ([WILDCARD])
+[WILDCARD]/bench/filter/a_bench.ts
+benchmark      time (avg)             (min … max)       p75       p99      p995
+------------------------------------------------- -----------------------------
+foo [WILDCARD] [WILDCARD]/iter[WILDCARD]([WILDCARD] … [WILDCARD]) [WILDCARD]
 
+[WILDCARD]/bench/filter/b_bench.ts
+benchmark      time (avg)             (min … max)       p75       p99      p995
+------------------------------------------------- -----------------------------
+foo [WILDCARD] [WILDCARD]/iter[WILDCARD]([WILDCARD] … [WILDCARD]) [WILDCARD]
+
+[WILDCARD]/bench/filter/c_bench.ts
+benchmark      time (avg)             (min … max)       p75       p99      p995
+------------------------------------------------- -----------------------------
+foo [WILDCARD] [WILDCARD]/iter[WILDCARD]([WILDCARD] … [WILDCARD]) [WILDCARD]
diff --git a/cli/tests/testdata/bench/finally_timeout.out b/cli/tests/testdata/bench/finally_timeout.out
index dfae4607d..ec91900aa 100644
--- a/cli/tests/testdata/bench/finally_timeout.out
+++ b/cli/tests/testdata/bench/finally_timeout.out
@@ -1,19 +1,11 @@
 Check [WILDCARD]/bench/finally_timeout.ts
-running 2 benches from [WILDCARD]/bench/finally_timeout.ts
-bench error ... 1000 iterations FAILED ([WILDCARD])
-bench success ... 1000 iterations [WILDCARD] ns/iter ([WILDCARD]..[WILDCARD] ns/iter) ok ([WILDCARD])
-
-failures:
-
-error
-Error: fail
-    at [WILDCARD]/bench/finally_timeout.ts:4:11
-    at [WILDCARD]
-
-failures:
-
-	error
-
-bench result: FAILED. 1 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out ([WILDCARD])
-
+cpu: [WILDCARD]
+runtime: deno [WILDCARD] ([WILDCARD])
+
+[WILDCARD]/bench/finally_timeout.ts
+benchmark      time (avg)             (min … max)       p75       p99      p995
+------------------------------------------------- -----------------------------
+error      error: Error: fail
+[WILDCARD]
+success [WILDCARD] [WILDCARD]/iter[WILDCARD]([WILDCARD] … [WILDCARD]) [WILDCARD]
 error: Bench failed
diff --git a/cli/tests/testdata/bench/group_baseline.out b/cli/tests/testdata/bench/group_baseline.out
new file mode 100644
index 000000000..da7157b5b
--- /dev/null
+++ b/cli/tests/testdata/bench/group_baseline.out
@@ -0,0 +1,18 @@
+[WILDCARD]/bench/group_baseline.ts
+benchmark         time (avg)             (min … max)       p75       p99      p995
+---------------------------------------------------- -----------------------------
+noop [WILDCARD] [WILDCARD]/iter[WILDCARD]([WILDCARD] … [WILDCARD]) [WILDCARD]
+noop2 [WILDCARD] [WILDCARD]/iter[WILDCARD]([WILDCARD] … [WILDCARD]) [WILDCARD]
+
+summary
+  noo[WILDCARD]
+   [WILDCARD]x times [WILDCARD] than noo[WILDCARD]
+
+noop3 [WILDCARD] [WILDCARD]/iter[WILDCARD]([WILDCARD] … [WILDCARD]) [WILDCARD]
+parse url 2x [WILDCARD] [WILDCARD]/iter[WILDCARD]([WILDCARD] … [WILDCARD]) [WILDCARD]
+parse url 6x [WILDCARD] [WILDCARD]/iter[WILDCARD]([WILDCARD] … [WILDCARD]) [WILDCARD]
+
+summary
+  parse url 2x
+   [WILDCARD]x times slower than noop3
+   [WILDCARD]x times faster than parse url 6x
diff --git a/cli/tests/testdata/bench/group_baseline.ts b/cli/tests/testdata/bench/group_baseline.ts
new file mode 100644
index 000000000..86c545116
--- /dev/null
+++ b/cli/tests/testdata/bench/group_baseline.ts
@@ -0,0 +1,18 @@
+Deno.bench("noop", () => {});
+Deno.bench("noop2", { baseline: true }, () => {});
+
+Deno.bench("noop3", { group: "url" }, () => {});
+
+Deno.bench("parse url 2x", { group: "url", baseline: true }, () => {
+  new URL("https://deno.land/std/http/server.ts");
+  new URL("https://deno.land/std/http/server.ts");
+});
+
+Deno.bench("parse url 6x", { group: "url" }, () => {
+  new URL("https://deno.land/std/http/server.ts");
+  new URL("https://deno.land/std/http/server.ts");
+  new URL("https://deno.land/std/http/server.ts");
+  new URL("https://deno.land/std/http/server.ts");
+  new URL("https://deno.land/std/http/server.ts");
+  new URL("https://deno.land/std/http/server.ts");
+});
diff --git a/cli/tests/testdata/bench/ignore.out b/cli/tests/testdata/bench/ignore.out
index cda77ea52..e2a35621d 100644
--- a/cli/tests/testdata/bench/ignore.out
+++ b/cli/tests/testdata/bench/ignore.out
@@ -1,15 +1,7 @@
 Check [WILDCARD]/bench/ignore.ts
-running 10 benches from [WILDCARD]/bench/ignore.ts
-bench bench0 ... 1000 iterations ignored ([WILDCARD])
-bench bench1 ... 1000 iterations ignored ([WILDCARD])
-bench bench2 ... 1000 iterations ignored ([WILDCARD])
-bench bench3 ... 1000 iterations ignored ([WILDCARD])
-bench bench4 ... 1000 iterations ignored ([WILDCARD])
-bench bench5 ... 1000 iterations ignored ([WILDCARD])
-bench bench6 ... 1000 iterations ignored ([WILDCARD])
-bench bench7 ... 1000 iterations ignored ([WILDCARD])
-bench bench8 ... 1000 iterations ignored ([WILDCARD])
-bench bench9 ... 1000 iterations ignored ([WILDCARD])
-
-bench result: ok. 0 passed; 0 failed; 10 ignored; 0 measured; 0 filtered out ([WILDCARD])
+cpu: [WILDCARD]
+runtime: deno [WILDCARD] ([WILDCARD])
 
+[WILDCARD]/bench/ignore.ts
+benchmark      time (avg)             (min … max)       p75       p99      p995
+------------------------------------------------- -----------------------------
diff --git a/cli/tests/testdata/bench/ignore_permissions.out b/cli/tests/testdata/bench/ignore_permissions.out
index c55ccaa21..a518c572c 100644
--- a/cli/tests/testdata/bench/ignore_permissions.out
+++ b/cli/tests/testdata/bench/ignore_permissions.out
@@ -1,6 +1,7 @@
 Check [WILDCARD]/bench/ignore_permissions.ts
-running 1 bench from [WILDCARD]/bench/ignore_permissions.ts
-bench ignore ... 1000 iterations ignored ([WILDCARD])
-
-bench result: ok. 0 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out ([WILDCARD])
+cpu: [WILDCARD]
+runtime: deno [WILDCARD] ([WILDCARD])
 
+[WILDCARD]/bench/ignore_permissions.ts
+benchmark      time (avg)             (min … max)       p75       p99      p995
+------------------------------------------------- -----------------------------
diff --git a/cli/tests/testdata/bench/interval.out b/cli/tests/testdata/bench/interval.out
index dec5549ef..882821837 100644
--- a/cli/tests/testdata/bench/interval.out
+++ b/cli/tests/testdata/bench/interval.out
@@ -1,5 +1,7 @@
 Check [WILDCARD]/bench/interval.ts
-running 0 benches from [WILDCARD]/bench/interval.ts
-
-bench result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out ([WILDCARD])
+cpu: [WILDCARD]
+runtime: deno [WILDCARD] ([WILDCARD])
 
+[WILDCARD]/bench/interval.ts
+benchmark      time (avg)             (min … max)       p75       p99      p995
+------------------------------------------------- -----------------------------
diff --git a/cli/tests/testdata/bench/load_unload.out b/cli/tests/testdata/bench/load_unload.out
index 9b73341d4..6537f47fa 100644
--- a/cli/tests/testdata/bench/load_unload.out
+++ b/cli/tests/testdata/bench/load_unload.out
@@ -1,6 +1,8 @@
 Check [WILDCARD]/bench/load_unload.ts
-running 1 bench from [WILDCARD]/bench/load_unload.ts
-bench bench ... 1000 iterations [WILDCARD] ns/iter ([WILDCARD]..[WILDCARD] ns/iter) ok ([WILDCARD])
-
-bench result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out ([WILDCARD])
+cpu: [WILDCARD]
+runtime: deno [WILDCARD] ([WILDCARD])
 
+[WILDCARD]/bench/load_unload.ts
+benchmark      time (avg)             (min … max)       p75       p99      p995
+------------------------------------------------- -----------------------------
+bench [WILDCARD] [WILDCARD]/iter[WILDCARD]([WILDCARD] … [WILDCARD]) [WILDCARD]
diff --git a/cli/tests/testdata/bench/meta.out b/cli/tests/testdata/bench/meta.out
index e62172eb3..85b898e7e 100644
--- a/cli/tests/testdata/bench/meta.out
+++ b/cli/tests/testdata/bench/meta.out
@@ -1,7 +1,9 @@
 Check [WILDCARD]/bench/meta.ts
 import.meta.main: false
 import.meta.url: [WILDCARD]/bench/meta.ts
-running 0 benches from [WILDCARD]/bench/meta.ts
-
-bench result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out ([WILDCARD])
+cpu: [WILDCARD]
+runtime: deno [WILDCARD] ([WILDCARD])
 
+[WILDCARD]/bench/meta.ts
+benchmark      time (avg)             (min … max)       p75       p99      p995
+------------------------------------------------- -----------------------------
diff --git a/cli/tests/testdata/bench/no_check.out b/cli/tests/testdata/bench/no_check.out
index ceb8b22fc..1f90836eb 100644
--- a/cli/tests/testdata/bench/no_check.out
+++ b/cli/tests/testdata/bench/no_check.out
@@ -1,6 +1,3 @@
-
-bench result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out ([WILDCARD])
-
 error: Uncaught TypeError: Cannot read properties of undefined (reading 'fn')
 Deno.bench();
      ^
diff --git a/cli/tests/testdata/bench/no_prompt_by_default.out b/cli/tests/testdata/bench/no_prompt_by_default.out
index d47198d85..a73971a9b 100644
--- a/cli/tests/testdata/bench/no_prompt_by_default.out
+++ b/cli/tests/testdata/bench/no_prompt_by_default.out
@@ -1,16 +1,9 @@
-running 1 bench from [WILDCARD]no_prompt_by_default.ts
-bench no prompt ... 1000 iterations FAILED ([WILDCARD]ms)
+[WILDCARD]cpu: [WILDCARD]
+runtime: deno [WILDCARD] ([WILDCARD])
 
-failures:
-
-no prompt
-PermissionDenied: Requires read access to "./some_file.txt", run again with the --allow-read flag
+[WILDCARD]/bench/no_prompt_by_default.ts
+benchmark      time (avg)             (min … max)       p75       p99      p995
+------------------------------------------------- -----------------------------
+no prompt  error: PermissionDenied: Requires read access to "./some_file.txt", run again with the --allow-read flag
 [WILDCARD]
-
-failures:
-
-	no prompt
-
-bench result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out ([WILDCARD]ms)
-
 error: Bench failed
diff --git a/cli/tests/testdata/bench/no_prompt_with_denied_perms.out b/cli/tests/testdata/bench/no_prompt_with_denied_perms.out
index efe9fa6dc..fe8f0c29c 100644
--- a/cli/tests/testdata/bench/no_prompt_with_denied_perms.out
+++ b/cli/tests/testdata/bench/no_prompt_with_denied_perms.out
@@ -1,16 +1,9 @@
-running 1 bench from [WILDCARD]/no_prompt_with_denied_perms.ts
-bench no prompt ... 1000 iterations FAILED ([WILDCARD]ms)
+[WILDCARD]cpu: [WILDCARD]
+runtime: deno [WILDCARD] ([WILDCARD])
 
-failures:
-
-no prompt
-PermissionDenied: Requires read access to "./some_file.txt", run again with the --allow-read flag
+[WILDCARD]/bench/no_prompt_with_denied_perms.ts
+benchmark      time (avg)             (min … max)       p75       p99      p995
+------------------------------------------------- -----------------------------
+no prompt  error: PermissionDenied: Requires read access to "./some_file.txt", run again with the --allow-read flag
 [WILDCARD]
-
-failures:
-
-	no prompt
-
-bench result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out ([WILDCARD]ms)
-
 error: Bench failed
diff --git a/cli/tests/testdata/bench/only.out b/cli/tests/testdata/bench/only.out
index 3c9855560..b874c9a96 100644
--- a/cli/tests/testdata/bench/only.out
+++ b/cli/tests/testdata/bench/only.out
@@ -1,7 +1,9 @@
 Check [WILDCARD]/bench/only.ts
-running 1 bench from [WILDCARD]/bench/only.ts
-bench only ... 1000 iterations [WILDCARD] ns/iter ([WILDCARD]..[WILDCARD] ns/iter) ok ([WILDCARD])
-
-bench result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 2 filtered out ([WILDCARD])
+cpu: [WILDCARD]
+runtime: deno [WILDCARD] ([WILDCARD])
 
+[WILDCARD]/bench/only.ts
+benchmark      time (avg)             (min … max)       p75       p99      p995
+------------------------------------------------- -----------------------------
+only [WILDCARD] [WILDCARD]/iter[WILDCARD]([WILDCARD] … [WILDCARD]) [WILDCARD]
 error: Bench failed because the "only" option was used
diff --git a/cli/tests/testdata/bench/overloads.out b/cli/tests/testdata/bench/overloads.out
index a736b2e98..347a8392d 100644
--- a/cli/tests/testdata/bench/overloads.out
+++ b/cli/tests/testdata/bench/overloads.out
@@ -1,11 +1,12 @@
 Check [WILDCARD]/bench/overloads.ts
-running 6 benches from [WILDCARD]/bench/overloads.ts
-bench bench0 ... 1000 iterations [WILDCARD] ns/iter ([WILDCARD]..[WILDCARD] ns/iter) ok ([WILDCARD])
-bench bench1 ... 1000 iterations [WILDCARD] ns/iter ([WILDCARD]..[WILDCARD] ns/iter) ok ([WILDCARD])
-bench bench2 ... 1000 iterations [WILDCARD] ns/iter ([WILDCARD]..[WILDCARD] ns/iter) ok ([WILDCARD])
-bench bench3 ... 1000 iterations [WILDCARD] ns/iter ([WILDCARD]..[WILDCARD] ns/iter) ok ([WILDCARD])
-bench bench4 ... 1000 iterations [WILDCARD] ns/iter ([WILDCARD]..[WILDCARD] ns/iter) ok ([WILDCARD])
-bench bench5 ... 1000 iterations ignored ([WILDCARD])
-
-bench result: ok. 5 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out ([WILDCARD])
+cpu: [WILDCARD]
+runtime: deno [WILDCARD] ([WILDCARD])
 
+[WILDCARD]/bench/overloads.ts
+benchmark      time (avg)             (min … max)       p75       p99      p995
+------------------------------------------------- -----------------------------
+bench0 [WILDCARD] [WILDCARD]/iter[WILDCARD]([WILDCARD] … [WILDCARD]) [WILDCARD]
+bench1 [WILDCARD] [WILDCARD]/iter[WILDCARD]([WILDCARD] … [WILDCARD]) [WILDCARD]
+bench2 [WILDCARD] [WILDCARD]/iter[WILDCARD]([WILDCARD] … [WILDCARD]) [WILDCARD]
+bench3 [WILDCARD] [WILDCARD]/iter[WILDCARD]([WILDCARD] … [WILDCARD]) [WILDCARD]
+bench4 [WILDCARD] [WILDCARD]/iter[WILDCARD]([WILDCARD] … [WILDCARD]) [WILDCARD]
diff --git a/cli/tests/testdata/bench/pass.out b/cli/tests/testdata/bench/pass.out
index 99320e666..9090b5535 100644
--- a/cli/tests/testdata/bench/pass.out
+++ b/cli/tests/testdata/bench/pass.out
@@ -1,15 +1,17 @@
 Check [WILDCARD]/bench/pass.ts
-running 10 benches from [WILDCARD]/bench/pass.ts
-bench bench0 ... 1000 iterations [WILDCARD] ns/iter ([WILDCARD]..[WILDCARD] ns/iter) ok ([WILDCARD])
-bench bench1 ... 1000 iterations [WILDCARD] ns/iter ([WILDCARD]..[WILDCARD] ns/iter) ok ([WILDCARD])
-bench bench2 ... 1000 iterations [WILDCARD] ns/iter ([WILDCARD]..[WILDCARD] ns/iter) ok ([WILDCARD])
-bench bench3 ... 1000 iterations [WILDCARD] ns/iter ([WILDCARD]..[WILDCARD] ns/iter) ok ([WILDCARD])
-bench bench4 ... 1000 iterations [WILDCARD] ns/iter ([WILDCARD]..[WILDCARD] ns/iter) ok ([WILDCARD])
-bench bench5 ... 1000 iterations [WILDCARD] ns/iter ([WILDCARD]..[WILDCARD] ns/iter) ok ([WILDCARD])
-bench bench6 ... 1000 iterations [WILDCARD] ns/iter ([WILDCARD]..[WILDCARD] ns/iter) ok ([WILDCARD])
-bench bench7 ... 1000 iterations [WILDCARD] ns/iter ([WILDCARD]..[WILDCARD] ns/iter) ok ([WILDCARD])
-bench bench8 ... 1000 iterations [WILDCARD] ns/iter ([WILDCARD]..[WILDCARD] ns/iter) ok ([WILDCARD])
-bench bench9 ... 1000 iterations [WILDCARD] ns/iter ([WILDCARD]..[WILDCARD] ns/iter) ok ([WILDCARD])
-
-bench result: ok. 10 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out ([WILDCARD])
+cpu: [WILDCARD]
+runtime: deno [WILDCARD] ([WILDCARD])
 
+[WILDCARD]/bench/pass.ts
+benchmark      time (avg)             (min … max)       p75       p99      p995
+------------------------------------------------- -----------------------------
+bench0 [WILDCARD] [WILDCARD]/iter[WILDCARD]([WILDCARD] … [WILDCARD]) [WILDCARD]
+bench1 [WILDCARD] [WILDCARD]/iter[WILDCARD]([WILDCARD] … [WILDCARD]) [WILDCARD]
+bench2 [WILDCARD] [WILDCARD]/iter[WILDCARD]([WILDCARD] … [WILDCARD]) [WILDCARD]
+bench3 [WILDCARD] [WILDCARD]/iter[WILDCARD]([WILDCARD] … [WILDCARD]) [WILDCARD]
+bench4 [WILDCARD] [WILDCARD]/iter[WILDCARD]([WILDCARD] … [WILDCARD]) [WILDCARD]
+bench5 [WILDCARD] [WILDCARD]/iter[WILDCARD]([WILDCARD] … [WILDCARD]) [WILDCARD]
+bench6 [WILDCARD] [WILDCARD]/iter[WILDCARD]([WILDCARD] … [WILDCARD]) [WILDCARD]
+bench7 [WILDCARD] [WILDCARD]/iter[WILDCARD]([WILDCARD] … [WILDCARD]) [WILDCARD]
+bench8 [WILDCARD] [WILDCARD]/iter[WILDCARD]([WILDCARD] … [WILDCARD]) [WILDCARD]
+bench9 [WILDCARD] [WILDCARD]/iter[WILDCARD]([WILDCARD] … [WILDCARD]) [WILDCARD]
diff --git a/cli/tests/testdata/bench/quiet.out b/cli/tests/testdata/bench/quiet.out
index e214980e6..aad853189 100644
--- a/cli/tests/testdata/bench/quiet.out
+++ b/cli/tests/testdata/bench/quiet.out
@@ -1,8 +1,10 @@
-running 4 benches from [WILDCARD]/bench/quiet.ts
-bench console.log ... 1000 iterations [WILDCARD] ns/iter ([WILDCARD]..[WILDCARD] ns/iter) ok [WILDCARD]
-bench console.error ... 1000 iterations [WILDCARD] ns/iter ([WILDCARD]..[WILDCARD] ns/iter) ok [WILDCARD]
-bench console.info ... 1000 iterations [WILDCARD] ns/iter ([WILDCARD]..[WILDCARD] ns/iter) ok [WILDCARD]
-bench console.warn ... 1000 iterations [WILDCARD] ns/iter ([WILDCARD]..[WILDCARD] ns/iter) ok [WILDCARD]
-
-bench result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out ([WILDCARD])
+cpu: [WILDCARD]
+runtime: deno [WILDCARD] ([WILDCARD])
 
+[WILDCARD]/bench/quiet.ts
+benchmark          time (avg)             (min … max)       p75       p99      p995
+----------------------------------------------------- -----------------------------
+console.log [WILDCARD] [WILDCARD]/iter[WILDCARD]([WILDCARD] … [WILDCARD]) [WILDCARD]
+console.error [WILDCARD] [WILDCARD]/iter[WILDCARD]([WILDCARD] … [WILDCARD]) [WILDCARD]
+console.info [WILDCARD] [WILDCARD]/iter[WILDCARD]([WILDCARD] … [WILDCARD]) [WILDCARD]
+console.warn [WILDCARD] [WILDCARD]/iter[WILDCARD]([WILDCARD] … [WILDCARD]) [WILDCARD]
diff --git a/cli/tests/testdata/bench/unhandled_rejection.out b/cli/tests/testdata/bench/unhandled_rejection.out
index 0e2b03c3e..98c9e68b5 100644
--- a/cli/tests/testdata/bench/unhandled_rejection.out
+++ b/cli/tests/testdata/bench/unhandled_rejection.out
@@ -1,7 +1,4 @@
 Check [WILDCARD]/bench/unhandled_rejection.ts
-
-bench result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out ([WILDCARD])
-
 error: Uncaught (in promise) Error: rejection
   reject(new Error("rejection"));
          ^
diff --git a/cli/tests/testdata/bench/unresolved_promise.out b/cli/tests/testdata/bench/unresolved_promise.out
index b3c3d65f9..d544d77e7 100644
--- a/cli/tests/testdata/bench/unresolved_promise.out
+++ b/cli/tests/testdata/bench/unresolved_promise.out
@@ -1,5 +1,2 @@
 Check [WILDCARD]/bench/unresolved_promise.ts
-
-bench result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out ([WILDCARD])
-
 error: Module evaluation is still pending but there are no pending ops or dynamic imports. This situation is often caused by unresolved promises.