The State of Test Parallelization 2026

How teams shard their test suites in 2026: the speedup sharding actually buys, the runner cost it adds, and the point where the curve flattens and each new shard stops earning its minute.

Executive summary

Test parallelization is the most direct lever a team has on pipeline wall-clock time, and in 2026 sharding has moved from a large-org luxury to a mainstream practice for any suite that runs longer than a coffee break. A suite that takes twenty minutes on one runner is a tax on every push, and splitting it across several runners is the obvious way to claw that time back. But the speedup the tooling implies and the speedup teams actually realize diverge quickly, and the gap is where most of the wasted money in sharded CI hides.

The reason is Amdahl's law wearing a CI costume. Fixed per-shard overhead, namely runner startup, dependency install, and the serial setup that every shard repeats, does not parallelize. Each added shard returns less wall-clock time than the last while still billing the same per-minute rate, so the speedup curve bends down at exactly the moment the cost curve bends up. This report quantifies that crossover: where the speedup is nearly linear, where it bends, and where adding shards mostly adds cost without adding much speed.

The teams that get parallelization right treat sharding as one piece of a system rather than a dial to turn to eleven. They pair it with caching to shrink the un-parallelizable overhead that caps the speedup, they balance shards by historical timing rather than file count so no single shard dictates the whole run, and they stop adding shards once the marginal minute saved costs more than it is worth. Done well, sharding turns a slow gate into a fast one for a manageable premium. Done carelessly, it doubles a CI bill to shave a couple of minutes that caching would have removed for free.

Three numbers frame the year. Roughly half of teams running large suites now shard across more than one runner, which makes parallelization a mainstream default rather than a power-user trick. The median suite realizes about six times wall-clock speedup at eight-way sharding, well short of the eightfold the matrix implies. And the marginal speedup per added shard falls below a third of a runner around the eight-shard mark for a typical suite, which is the practical stopping point where the economics stop favoring more parallelism.

The encouraging part for engineering leaders is that none of the levers here are exotic. Caching, timing-based shard balancing, right-sized runners, and a cheap per-minute rate are pipeline hygiene, and together they move both the speedup ceiling and the cost crossover in the team's favor. The teams that internalize this stop asking 'how many shards' as if more were always better, and start asking 'what does the next shard cost and what does it actually buy', which is the question that keeps a sharded pipeline fast and affordable as the suite grows.

Speedup is near-linear early and bends hard by eight shards

Through about four shards most suites realize close to the ideal speedup, because actual test execution still dominates the minute and there is plenty of parallelizable work to spread across runners. Two shards return roughly 1.9 times, four return roughly 3.4 times, and the curve tracks the linear ideal closely enough that the difference is easy to ignore. This is the regime where sharding feels like magic, and it is the regime most teams are in when they first turn it on.

By eight shards the picture changes. The un-parallelizable setup that every shard repeats, runner startup plus dependency install plus the serial fixture work that does not split, becomes the floor under the wall-clock time. The modeled median realizes about six times at eight-way sharding rather than the eight times the matrix implies, and at sixteen shards it reaches only about 8.7 times. The execution that parallelizes keeps shrinking per shard while the fixed overhead per shard does not, so the curve asymptotes toward a ceiling set by that overhead.

The donut below shows why. On a representative shard, roughly six in ten minutes are parallelizable test execution and the rest is overhead that every shard pays in full. As you add shards you keep dividing the parallelizable slice while re-paying the fixed slice on each new runner, so the realized speedup can never exceed the ratio of total work to per-shard fixed cost. The practical reading is that the bend is not a tooling defect to be fixed, it is the arithmetic of the suite's overhead, and the only way to move the ceiling up is to shrink that overhead.

Cost grows faster than speed, and the gap is the overhead

Because per-shard startup and install are paid once per shard, billed minutes climb super-linearly while speedup climbs sub-linearly. Eight shards in the model cost about 9.2 times a single runner's minutes while delivering about six times the speed, and sixteen shards cost nearly twenty times while delivering under nine. The two curves diverge from the very first split and the gap widens with every shard, because every new runner re-pays the full fixed overhead before it does any useful parallel work.

The crossover that matters is the point where each new shard costs more in money than the wall-clock minute it saves is worth to the team. For most suites that arrives around the eight-shard mark, which is why eight shows up repeatedly as the practical ceiling rather than a hard technical limit. Past it you are buying smaller and smaller time savings at a larger and larger per-shard premium, and the spend stops being justifiable on speed alone.

Reading the two teaser charts together is the whole lesson of this report. The speedup chart bends down and the cost-multiple chart bends up, and the vertical distance between them at any shard count is the overhead you are paying for parallelism. Teams that watch only the speedup chart keep adding shards because the line is still rising, while the cost line is rising faster the entire time. The right number of shards is the one just before the cost line pulls decisively away from the value of the time saved.

Caching is what unlocks the next shard cheaply

The thing that flattens the speedup curve is the repeated, un-parallelizable overhead, and the largest single piece of that overhead is dependency install. Every shard that starts cold re-fetches and re-installs the same dependency tree that every other shard is installing at the same moment, which is pure duplicated work that scales with shard count rather than with the change being tested.

A warm cache shrinks that fixed cost on every shard simultaneously. When dependencies restore from cache in seconds instead of installing from scratch in minutes, the un-parallelizable slice of each shard gets smaller, which does two things at once: it raises the realized speedup, because more of each shard's minute is now parallelizable execution, and it pushes the cost crossover further out, because each added shard now re-pays a smaller fixed overhead. Caching and sharding are therefore complements, not alternatives, and the order of operations matters: cache first, then shard.

This is why a team that shards without caching often sees disappointing numbers and blames parallelism, when the real culprit is that every shard is paying full cold-install cost. Fixing the cache frequently buys more speedup than adding several shards would have, at a fraction of the cost, and it makes whatever sharding the team does add far more effective.

• Repeated dependency install is the largest slice of un-parallelizable per-shard overhead in the modeled split.
• A warm cache shrinks the fixed cost on every shard at once, raising realized speedup and pushing the cost crossover out.
• Cache before you shard: a cache fix often buys more speed than several added shards, for far less money.

Balance shards by timing, not by file count

Splitting tests evenly by file or alphabetically is the default in most sharding setups, and it is also the quiet reason realized speedup falls short of the model. Tests are not uniform: a handful of integration tests can take longer than thousands of unit tests, so an even-by-count split routinely lands all the slow tests on one shard. That shard becomes the longest pole, and because the run is only done when the slowest shard finishes, the whole matrix waits on it.

When one shard runs twice as long as the others, you are paying for full parallelism and realizing half of it. The team bought eight runners and got the wall-clock time of four, while still being billed for eight. This is the single most common way sharding underdelivers, and it is invisible unless someone looks at per-shard durations rather than the aggregate.

Balancing by historical per-test duration fixes it directly. When the splitter assigns tests to shards using recorded timing so that every shard is predicted to finish at roughly the same moment, the shards complete together and the realized speedup moves back toward the model. This timing-based balancing is the difference between the realized speedup in this report and the much worse number teams often see in practice, and it costs nothing but a record of how long each test took last time.

• Even-by-count splits put all the slow tests on one shard, creating a longest pole that caps the whole run.
• A single 2x-longer shard means you pay for full parallelism and realize roughly half of it.
• Balancing by recorded per-test duration keeps shards finishing together and recovers most of the lost speedup.

Adoption tracks suite duration almost perfectly

Sharding adoption is not uniform across teams, and the variable that predicts it best is simply how long the suite takes on one runner. Teams whose suite runs in under five minutes mostly do not bother, because the wall-clock pain is small and the overhead of setting up and maintaining a shard matrix is not worth it. The modeled adoption at that tier sits in the mid-teens of a percent.

The curve climbs steeply with duration. By the five-to-fifteen-minute tier roughly four in ten teams shard, and by the fifteen-to-forty-minute tier it is around seven in ten, which is where the practice becomes the norm rather than the exception. Past forty minutes nearly nine in ten teams shard, because at that length the suite is a serious drag on the development loop and the math for parallelism is overwhelming regardless of the overhead.

The signal for leaders is that suite duration is the trigger to watch. A suite quietly crossing the fifteen-minute mark is the point at which sharding stops being optional and starts being expected, and it is a good moment to make sure caching and timing-based balancing are in place first so the sharding the team adds actually delivers.

Runner startup latency is a hidden shard tax

Sharding multiplies the number of runner cold starts a pipeline incurs, because every shard is a separate runner that has to be provisioned, booted, and registered before it runs a single test. On hosted runners with a cold start measured in tens of seconds, that latency is paid in parallel across all shards, but it is still a fixed floor under every shard's wall-clock time, and it is one of the components that keeps realized speedup below the ideal.

The effect is most visible exactly where teams want sharding most: short, wide matrices. If each shard would run for ninety seconds of actual work but spends forty seconds starting up, then more than a quarter of every shard's wall-clock is startup the team cannot parallelize away by adding more shards. The wider the matrix, the more total startup the pipeline pays, even though it overlaps.

A warm pool of pre-provisioned runners removes most of this tax by making each shard start in seconds rather than tens of seconds. That shrinks the fixed per-shard floor, which raises realized speedup and is part of why the runner layer matters as much as the test code in a sharded pipeline. A team can balance shards perfectly and still lose the gain to cold starts if the runners underneath are slow to come up.

Cheaper per-minute runners change the stopping point

The shard count where parallelism stops being worth it is set by the per-minute rate as much as by the overhead. The cost crossover is a comparison between the value of the wall-clock minute saved and the price of the extra runner-minutes spent to save it, and lowering the price of a runner-minute pushes that crossover to a higher shard count. A team on a cheaper rate can justify more parallelism for the same speed target.

Moving sharded jobs to managed runners at a modeled 70 percent saving versus hosted rates lowers the cost of each added shard directly. The same eight-shard matrix that was marginal on hosted pricing becomes comfortably worth it, and the team that wanted ten or twelve shards for a very large suite can have them without the cost line pulling away as sharply. The economics of diminishing returns do not disappear, but the point where they bite moves outward.

Self-healing changes the calculus too. A flaky shard on a plain matrix can force a re-run of the entire matrix, which multiplies the cost of every spurious failure by the shard count. When the runner layer retries a transient shard failure automatically on a fresh environment, a single flaky test in one shard does not drag the other shards back through a full re-run, so the realized cost of a wide matrix stays closer to its intended cost rather than spiking on every mechanical flake.

• The cost crossover is set by the runner rate; a cheaper minute pushes the worth-it shard count higher.
• Managed runners at a modeled 70% saving make wider matrices economical for the same speed target.
• Auto-healing a transient shard failure avoids re-running the whole matrix, keeping a flaky test from multiplying cost by shard count.

Recommendations

Cache dependencies before you add a single shard

Repeated dependency install is the biggest un-parallelizable cost in a sharded run, and it scales with shard count. A warm cache shrinks that fixed cost on every shard at once, which both raises realized speedup and pushes the cost crossover out. Fixing the cache often buys more speed than several added shards would, at a fraction of the spend, so cache first and shard second.

Balance shards by historical timing, not by file count

An even-by-count split routinely lands all the slow tests on one shard, creating a longest pole that caps the whole run while you pay for full parallelism. Split tests using recorded per-test durations so every shard is predicted to finish at the same moment. This is the cheapest single change that moves realized speedup back toward the model.

Pick a shard count from the crossover, not from a round number

Read the speedup and cost-multiple curves together and stop adding shards where the cost line pulls decisively away from the value of the time saved, which for most suites is around eight. More shards past that point buy shrinking time savings at a growing per-shard premium. Treat the shard count as an economic decision per suite, not a fixed default.

Put a warm pool under the matrix to kill cold-start tax

Every shard is a separate cold start, and tens of seconds of startup per shard is a fixed floor that caps realized speedup, especially for short, wide matrices. A warm pool that starts each shard in seconds shrinks that floor, which is why the runner layer matters as much as the test code in a sharded pipeline.

Use cheaper runners and self-healing to move the economics

A lower per-minute rate pushes the worth-it shard count higher for the same speed target, and auto-healing a transient shard failure avoids re-running the entire matrix on every mechanical flake. Together they let a team run the parallelism a large suite actually needs without the cost line spiking, which is the difference between sharding being affordable and sharding being a budget surprise.

Outlook

Expect sharding to keep spreading down-market through 2026 as suite durations creep up and the tooling to split and balance tests gets easier to adopt. The interesting shift is not more teams sharding, which is nearly inevitable, but more teams sharding well: caching first, balancing by timing, and choosing a shard count from the crossover rather than from a round number. The gap between those teams and the ones that simply crank the shard dial will show up directly on their CI invoices.

The economics will keep pointing at the runner layer. As the parallelizable execution per shard shrinks, the fixed per-shard overhead, cold start plus repeated install, becomes the dominant term, and the teams that win are the ones who attack that overhead with warm pools and warm caches rather than with more runners. A cheaper per-minute rate and automatic recovery for flaky shards compound that advantage, because they let a team run the parallelism a large suite genuinely needs without the cost curve punishing them for it.

For most teams the practical takeaway is that test parallelization is not a dial to turn to maximum, it is a system to tune. The suite has a natural ceiling on realized speedup set by its overhead, and the job is to raise that ceiling with caching and balancing while spending only up to the point where the next shard stops earning its minute. Teams that hold that discipline get fast pipelines for a sensible premium, while teams that treat more shards as always better pay super-linear costs for sub-linear speed.

Methodology

This report models the test-parallelization speedup and cost curves from a fixed-overhead-plus-parallelizable-work pipeline shape, using published hosted runner pricing for the cost multiples and Latchkey's analysis of typical suite profiles for the speedup figures. The exact shard count where returns flatten depends on each suite's ratio of parallelizable execution to repeated per-shard overhead, the cache hit rate, and the runner rate, so the crossover values here describe direction and magnitude rather than a precise per-team threshold. Adoption-by-duration figures are modeled from representative suite profiles rather than a primary survey. Figures labeled "modeled" are illustrative estimates derived from public pricing and typical pipeline shapes, not a primary survey; figures attributed to a named source reflect that source. Pricing reflects published rates at time of writing and should be verified against current provider pricing.

Sources

• Stack Overflow Developer Survey
• GitHub Actions documentation
• GitHub Actions - billing & pricing
• JetBrains Developer Ecosystem Survey

More Latchkey reports