CI/CD Pipeline Optimization: From 45 Minutes to 5

A 45-minute CI/CD pipeline isn't a minor inconvenience — it's a productivity tax that compounds across every engineer, every day. This guide documents the concrete techniques that took one client's pipeline from 45 minutes to under 5, without sacrificing test coverage or security.

Step 1: Diagnose Before You Optimize

The worst CI/CD optimization mistake is guessing which stage is slow. Measure first. Most CI platforms (GitHub Actions, GitLab CI, Tekton) provide per-step timing in the pipeline UI. Look for the top three time consumers — they typically account for 70-80% of total pipeline duration. Optimize those before touching anything else.

Common culprits in 45-minute pipelines: sequential test stages that could run in parallel, container builds that don't use layer caching, redundant dependency installations across stages, and slow integration tests that run on every commit when they only need to run on PR merge.

Step 2: Parallelize Everything That Can Run Concurrently

Sequential test stages are the most common and easiest fix. If your pipeline runs unit tests, then linting, then integration tests — in sequence — you're leaving significant time savings on the table. Run all three in parallel. The total pipeline time becomes the duration of the slowest stage, not the sum of all stages.

For test suites, shard by test file or test tag across multiple runners. A 20-minute test suite split across 4 runners becomes 5 minutes. GitHub Actions matrix strategy and GitLab parallel keyword both support this natively. Configure the test runner to report to a merged results file for coverage reporting.

Step 3: Fix Container Build Caching

Docker builds that reinstall all dependencies every run are the most fixable bottleneck in most pipelines. Layer ordering matters critically. Place instructions that change rarely (OS packages, language runtime, package manager lock files) before instructions that change frequently (application code). With correct layer ordering, a build that was reinstalling 200+ npm packages on every commit can skip that step entirely using the layer cache.

For CI environments without persistent Docker layer caches, use BuildKit's --cache-from with a registry cache. Push the build cache to your container registry on each run; pull it on the next. Tools like depot.dev provide remote Docker build caches as a service if you don't want to manage the infrastructure.

Step 4: Cache Dependencies Between Runs

Downloading and installing npm, pip, Maven, or Go modules on every pipeline run is slow and wasteful. Cache the dependency directory using your CI platform's native cache mechanism, keyed on the lock file hash. When the lock file doesn't change (most commits), the cache hit restores dependencies in seconds instead of minutes.

• GitHub Actions: use actions/cache with ~/.npm or ~/.m2 as the path, package-lock.json hash as the key
• GitLab CI: use cache: key: ${CI_COMMIT_REF_SLUG} with paths to your vendor or node_modules directory
• Tekton: use workspace PVCs with ReadWriteMany access mode to share dependency caches across pipeline runs

Step 5: Run the Right Tests at the Right Time

Not every test needs to run on every commit. Implement tiered testing: fast unit tests on every push (target: under 2 minutes), full integration tests on pull requests, and slow end-to-end tests only on merge to main or nightly. This alone can reduce average pipeline duration by 40-60% for most teams.

For monorepos, affected-change detection tells the CI system to run only the pipelines relevant to what actually changed. Nx, Turborepo, and Bazel all support this. Running only the affected services' tests instead of the entire monorepo is often the single biggest time win in large codebases.

Results: 45 Minutes to Under 5

Applied to one client's GitHub Actions pipeline: parallelizing test stages (saved 12 minutes), fixing Docker layer caching (saved 8 minutes), caching npm dependencies (saved 6 minutes), and moving integration tests to PR-only (saved 14 minutes from push pipeline duration). Total: 40 minutes saved. Pipeline dropped from 45 minutes to under 5 on the typical development commit path.

CI/CD Optimization with THNKBIG

THNKBIG's platform engineering team optimizes CI/CD pipelines for engineering organizations running Kubernetes workloads. From pipeline audits and caching strategy to full Tekton or GitLab CI redesigns, we reduce pipeline duration and infrastructure costs. Contact us to schedule a CI/CD optimization assessment.

A 45-minute pipeline isn't just slow — it's a productivity black hole. Developers context-switch. Code reviews pile up. Hotfixes become emergencies. We've cut pipelines from 45 minutes to under 5 without sacrificing quality. Here's how.

Measure First

Before optimizing, understand where time goes. Most CI platforms provide timing breakdowns per stage. Look for:

• Which stages take the longest?
• What's running sequentially that could run in parallel?
• Where are you downloading dependencies you already have?

80% of pipeline time is usually consumed by 20% of the steps. Those steps are your targets.

Parallelization

Tests are embarrassingly parallel. If you have 1,000 tests that take 30 minutes on one machine, spread them across 10 machines and finish in 3 minutes.

How to parallelize:

• Split tests by file, by test, or by timing data from previous runs
• Run linting, type checking, and security scanning simultaneously — they don't depend on each other
• Use matrix builds for multiple platforms or runtime versions in parallel instead of sequentially

Tools that help: Knapsack Pro (Ruby), pytest-split (Python), Jest's --shard flag. Most distribute test files across runners based on historical timing data.

Caching Everything

Downloading npm packages takes time. Downloading the same packages on every build is pure waste.

What to cache:

• Dependency directories — node_modules, .gradle, vendor, __pycache__
• Docker layers — use multi-stage builds with layers ordered from least-to-most-frequently changing. Dependencies go early. Source code goes last
• Package manager caches — BuildKit's cache mounts persist package caches across builds without including them in the final image

Most CI platforms support caching with automatic invalidation when lock files change. Dependency caching alone often cuts 5–10 minutes from builds.

Selective Testing

Not every change needs every test. If you modified the billing module, run billing tests. The authentication tests can wait for the nightly full suite.

Implement a test tier strategy:

1. Fast unit tests (milliseconds) — run on every commit, gate the pipeline early. If they fail, skip the expensive integration suite
2. Affected-area tests — map code changes to relevant test files; run only those tests on feature branches
3. Full integration suite — nightly or pre-merge, not on every push

Infrastructure Matters

Self-hosted runners on faster machines can cut pipeline times dramatically. Cloud-managed runners are convenient but often slow and shared.

Infrastructure improvements that pay off:

• Dedicated build clusters with fast SSD storage and high CPU count
• Enough memory to avoid swap during dependency installation or test runs
• Kubernetes-based CI (Tekton, Argo Workflows) — build steps run as pods, scaling on demand and terminating when idle

Where Pipeline Slowness Really Comes From

Slow pipelines are almost never caused by the CI platform. They're slow because pipelines were designed for convenience early in a project and never optimized as the codebase grew.

Common root causes:

• Sequential stages that have no dependency on each other
• Dependency installation re-running on every build instead of reading from cache
• Test suites that grew without pruning redundant or flaky tests
• Docker builds that invalidate the entire layer cache on minor source code changes

Benchmark your stages individually. Identify the three slowest. Most teams find that test execution (integration and end-to-end tests) and Docker build times account for 60–80% of total duration — those are the targets.

Parallelization and Caching: The Technical Details

Test parallelization: Split your test suite across multiple runners using matrix builds. Tools like Knapsack Pro, pytest-split, and Jest's --shard flag distribute test files based on historical timing data, minimizing total wall-clock time.

Docker layer caching: Copy dependency manifests (package.json, requirements.txt, go.mod) before application source. Install dependencies in a dedicated layer. Only source code changes invalidate subsequent layers.

Kubernetes-native CI: Tekton and Argo Workflows run build steps as Kubernetes pods, scaling runners on demand and terminating when idle. Teams switching from always-on VM-based runners to Kubernetes-native CI reduce CI infrastructure cost by 50–70% on moderate-traffic repositories.

Key Takeaways

• CI/CD pipeline optimization reduces mean time to deploy, cuts infrastructure costs, and improves developer experience — directly impacting team velocity
• The biggest optimization opportunities are usually parallelization, caching, and test suite rationalization — not switching CI platforms
• Kubernetes-native CI (Tekton, Argo Workflows) enables dynamic, autoscaling build infrastructure that reduces idle runner costs by 50–70%

THNKBIG's DevOps consulting practice runs CI/CD optimization engagements that consistently reduce pipeline duration by 40–60% and cut associated infrastructure cost. Talk to us about your pipeline.