High-Performance Sandbox Benchmarks

Compare top sandbox providers on the same pinned machine shape for real developer and CI/CD workloads.

Same target everywhere: 2 vCPU · 8 GiB RAM · 40 GB disk. One headline metric per dimension, ranked with honest statistics.

Synthetic scores tell you what the hardware can do. We measure what developers actually experience — clone a repo, install dependencies, lint, build, test, etc.

A sandbox provider can top a creation time or CPU performance chart and still lose badly on:

• dependency installation is thousands of small, random file writes, and a network-attached or bandwidth-capped disk turns that into the longest step of your run.
• cloning a repo has the opposite profile: mostly sequential writes, bounded by network.
• single-threaded developer tools are limited by single-thread CPU not threads

Live provider benches and toolchain releases are maintainer-only (GitHub Environment privileged). Pull requests never receive provider secrets — see CI & secrets.

This repo is a Bun workspace monorepo with a strict, enforced dependency DAG and a uniform package shape. The guiding rule: "can I import this?" is answered by the path alone, and boundary violations fail CI.

Every package's exports map points at TypeScript source (./src/index.ts), and Bun resolves workspace sources natively. There is no compile step: bun install → typecheck → test → lint are all green with zero compilation. The committed bun.lock pins the whole graph.

packages/   importable libraries   — scope @sandbox-benchmarks/*
  schema/       shared types + arktype schemas (bottom of the DAG)
  providers/    provider adapters → schema + computesdk
  templates/    per-provider template builders (one export subpath each)
  harness/      benchmark timing → providers + schema
  results/      normalization → schema only (no provider SDKs)
apps/
  cli/          entrypoint with bin commands → all five packages
tooling/        dev-only            — scope @repo/*
  tsconfig/     shared source-first TS configs (config-only)
  test-utils/   provider conformance suite factory
  repo-checks/  boundary + package-meta invariant tests

results deliberately depends on schema only — it must normalize without any provider SDK, and @repo/repo-checks enforces that no package reaches across boundaries or into another package's private lib/.

Run a single bin during development: bun apps/cli/src/bin/plan-matrix.ts.

Non-Bun tools are version-pinned in mise.toml and managed with mise. Today that's typos, the spell checker behind bun run spell. After cloning, run mise install (and mise trust once) so the pinned binaries are available; bun run spell invokes typos through mise exec, so it always uses the pinned version. mise fetches from official release sources with checksum verification — no npm republisher and no install-time postinstall.

.github/workflows/ci.yml runs the command contract on every pull request and every push to main: bun install --frozen-lockfile --ignore-scripts → bun run lint (the Biome gate) → bun run typecheck → bun run test → bun run spell (typos, set up via mise). The same checks run locally, so green-on-your-machine means green-in-CI.

Fork PRs get the hosted CI gate only. Self-hosted jobs and anything that needs provider credentials run only from this repository, on main, behind Environment privileged.

Lefthook runs a fast local mirror of CI on every commit, configured in lefthook.yml:

• Biome on staged files (biome check --write, restaging any auto-fixes; unfixable issues or warnings block the commit).
• Typos repo-wide (bun run spell) — read-only, so run bun run spell:fix to apply corrections.
• Lockfile check (bun install --frozen-lockfile) when a manifest or bun.lock is staged, so package.json and bun.lock can't drift apart.

bun install wires the hooks automatically via the project's own prepare script (lefthook install) — no third-party postinstall runs. Re-install them with bunx lefthook install, and bypass a single commit with LEFTHOOK=0 git commit.

bunfig.toml sets minimumReleaseAge = 604800 (7 days) so freshly published — possibly compromised — releases are not installed, and no third-party lifecycle scripts run (empty trustedDependencies). The git hooks above are wired by the project's own first-party prepare script, not a dependency's postinstall, and CI installs with --ignore-scripts so it runs none either. Lint and formatting are root-only via a single biome.json.

• Contributing — local gate; how to add a provider, suite, or metric
• Security — vulnerability reporting; never paste secrets into issues or PRs