CI needs to be ready when I am

Depot finished our typecheck benchmark in 49 seconds. Boat took 5 minutes and 1 second. Both restored all 101 tasks from cache.

Depot was much faster. Then I looked at how I actually work.

I'm often working across several chat sessions. One is implementing something, another is fixing a test, another is reviewing a change. I come back and merge things as a batch. I don't sit in front of every CI run waiting for it to turn green.

That changes the question I want the benchmark to answer: how much validated work is ready when I come back?

CI still needs to be fast. If I return to review and half the changes are waiting on checks, it is too slow. If an agent needs a test result before it can make its next edit, that delay is part of its working loop. But a result arriving four minutes earlier doesn't automatically give me four productive minutes back.

After working on making CI faster, we started looking at Boat because Char's Depot bill felt too high. Char has JavaScript, Rust, desktop and mobile code. We moved Linux CI and deployment jobs to Boat, kept Apple jobs on Depot, and kept Windows builds on GitHub. The migration passed candidate CI and merged on November 1. We haven't measured a comparable period of invoices after the switch yet.

The warm benchmark gives us a useful starting point:

These are observations from one warm run per provider, not a distribution of production CI times.

Most of the gap came before the task results were used. Depot automatically routes standard Actions cache operations through its own cache service and connects Turbo to its remote cache. On Boat, we added explicit persistence for Turbo results, but still spent minutes restoring dependencies and preparing the environment. Depot cache integration, Turbo integration.

We also had to lower typecheck concurrency from 16 to eight. Both machines had 16 vCPUs, but Boat had 32 GB of advertised RAM versus Depot's 64 GB. Our first Boat attempt exhausted memory and swap. Matching the CPU count hadn't reproduced the environment.

Those differences matter. This is a comparison of the configurations we would actually use, not an isolated CPU test.

Boat's dashboard attributed $0.017 of usage to the successful warm run. Depot's published 16-vCPU Linux rate is $0.048 per minute, which puts the 49-second run at $0.0392 at list price. That Depot number is calculated, not taken from our invoice. Depot runner pricing.

On those inputs, a dollar of compute corresponds to roughly 59 Boat benchmark runs or 26 Depot runs. That is an illustrative budget comparison using Boat's rounded dashboard figure and Depot's public rate. It excludes subscription floors, included credits, discounts, extra start charges and failed attempts. It also counts this one cached check, not complete PRs or shipped features.

Depot's speed narrows the price gap because we pay for fewer running seconds. The remaining premium can be easy to justify when someone is blocked on the result. The interesting part is what happens when the work is independent.

Suppose we have 20 independent checks like this one, ready at the same time. Assume the observed durations hold under load, with no failures, startup delay or shared bottlenecks. This is a model, not a load test:

At ten slots, if I return to review after 15 minutes, both setups have finished the whole batch. Depot was ready much earlier. In this example, that earlier finish didn't change when I reviewed or how many results were ready.

If I return after three minutes, Depot has finished and Boat hasn't finished its first wave. The same infrastructure now creates a real interruption. If the checks arrive gradually, a change pushed just before I return may still be waiting even when the earlier ones are done.

The review cadence determines whether the latency is visible.

Parallelism also changes how we should talk about saved time. Twenty jobs each finishing four minutes earlier does not mean I got 80 minutes back. They may all run while I am doing something else. Their compute costs still add up: under the model above, the batch costs about $0.784 on Depot versus $0.340 on Boat, regardless of whether we use one slot or twenty, provided durations stay constant.

The faster setup releases each slot earlier, though. With the same number of continuously busy slots, Depot can finish more checks per hour. Boat can buy more of this particular check per dollar of compute. Those are different constraints, and both can matter to the same team.

More parallel jobs cannot accelerate a chain where each step needs the previous result. An agent doing edit → test → inspect failure → edit still benefits from fast feedback. So does a person waiting to fix a release. A fast typecheck also matters if it unlocks later jobs. We need to measure the dependency path to a mergeable change, not add every job's duration together.

There are limits to simply running more things at once. Providers have start quotas and concurrency limits; jobs can compete for cache bandwidth or other shared resources. Our Boat plan has a $100 monthly floor with compute credit, and its included starts are limited per minute, hour and day. Our 20-job illustration assumes those limits have headroom. We haven't purchased extra starts. Boat limits and billing.

A matrix of test shards can consume many starts for one PR. Twenty independent checks are not twenty complete CI pipelines. And 101 cached Turbo tasks are not 101 new pieces of work accomplished.

This is why we changed job selection as well as providers. Our old workflow already skipped native mobile builds on PRs and selected them for affected pushes to main. The new selector distinguishes ordinary mobile JavaScript from native inputs, selects iOS and Android separately, and chooses affected Rust platforms before allocating their runners. Selected native checks now participate in the PR gate.

For frontend-only work, we keep the affected checks and mobile bundle exports on Linux. Native changes still get the appropriate platform builds. Skipping an unnecessary native job reduces compute demand and the chance that an unrelated long-running build holds up the batch. A green Linux check still doesn't prove Apple-specific code or device behavior works.

The provider choice and the selection policy can be evaluated independently. We can retain the selector even if we change runners again.

What I want to measure after the switch is the fraction of changes whose required checks are ready when I return to review. Alongside that: actual blocked time, the tail of CI completion times, and total cost per successfully validated change, including retries. That will tell us more than multiplying every saved runner-minute by an engineer's hourly rate.

We still need a comparable period of real workloads and billing before claiming monthly savings. That accounting needs to include queue time, failed attempts, complete PR pipelines, the Apple jobs we kept on Depot, and both providers' plan allowances.

I want CI fast enough that the batch is ready when I come back, and faster where another person or agent is waiting for its answer.