In traditional on-premises environments, overprovisioning was a sensible insurance policy: hardware was slow to procure, so teams bought extra capacity to avoid outages. In the era of elastic public cloud, that logic should have faded, yet most organisations still run significantly overprovisioned estates, often by 50% or more, with idle or “zombie” resources quietly burning budget.
Instead of being a solved problem, capacity waste has become an invisible drag on cost, reliability, and sustainability — especially as release cycles accelerate and systems grow more complex. Modern teams now need a systematic way to tune runtimes, rightsize resources, and align DevOps culture with cost and efficiency goals.
Why Overprovisioning Survived the Move to Cloud
Elastic compute promised to replace capital-heavy overprovisioning with on-demand scalability, but organisational habits and technical complexity kept the old patterns alive. Developers and platform teams are under constant pressure to maintain reliability, and the easiest lever has often been to “add more” CPU and memory rather than deeply analyse configuration and workloads.
Runtime and infrastructure layers now expose hundreds of tunable parameters across containers, orchestration platforms, and application runtimes like the JVM or Node.js. Understanding how these interact under real production traffic is difficult and time-consuming, so teams fall back on conservative defaults and copy-pasted “known good” settings that rarely match current reality.
The result is a landscape of static blueprints and oversized allocations, where scaling infrastructure simply amplifies existing inefficiencies rather than resolving them.
DevOps, Silos, and Conflicting Incentives
DevOps culture set out to break down silos, yet subtle divisions still shape how organisations manage reliability and cost. Platform and SRE teams are typically accountable for uptime, efficiency, and platform health, while application teams own runtime and workload configuration.
In practice, this split means:
- Platform and SRE teams can tune infrastructure layers and cluster-level policies but have limited authority to change JVM parameters, container limits, or application-specific configs.
- Application teams optimise for feature delivery and stability, often defaulting to higher resource limits to avoid being blamed for incidents.
These misaligned incentives create a “fracture” where no single team has both the mandate and visibility to optimise the entire stack. Even where collaboration is strong, keeping configurations in sync with rapidly changing workloads is nearly impossible to do manually at scale.
Extending DevOps with Cost and Sustainability Feedback
Most DevOps practices already rely on feedback loops: observe behaviour, orient around data, decide on changes, and act. However, many organisations still lack mature feedback loops for cloud cost and environmental impact, even when they recognise their importance.
FinOps and sustainability initiatives attempt to close this gap by:
- Exposing metrics such as cost per request, carbon impact per transaction, or energy usage per service.
- Making cost and sustainability visible to product and engineering teams, not just finance or operations.
- Embedding these signals alongside performance and reliability in decision-making.
When cost and sustainability become first-class feedback signals, teams can weigh trade-offs more accurately — for example, understanding whether a performance gain justifies additional spend or carbon impact.
The Runtime Optimization Blind Spot
While many tools exist to profile and optimise application code, far fewer help teams systematically tune the full runtime and infrastructure stack. Runtimes like the JVM have historically required specialist knowledge to configure garbage collection, heap sizing, and other low-level parameters, leading to one-off “hero” tuning efforts that are then cloned everywhere, regardless of context.
This creates several long-term issues:
- Configuration drift between what was once optimal and what is currently deployed.
- Inefficiencies baked into templates, Helm charts, or Terraform modules that get reused across services.
- Scaling decisions that multiply waste because they’re built on top of misconfigured runtimes.
Without tools that can continuously analyse observability data and test different configuration combinations across layers, teams stay trapped in a cycle of manual tuning and over-allocation.
How AI-Driven Platforms Approach Full-Stack Optimization
New optimisation platforms are emerging to close this gap by using AI to recommend or apply coordinated changes across workloads, runtimes, and Kubernetes configurations. These solutions typically work in two complementary modes: offline experimentation and continuous, production-aware optimisation.
An offline optimisation module might:
- Take a clear goal (for example, reduce cost while maintaining latency SLOs).
- Use load tests to explore configuration combinations across CPU, memory, JVM settings, and autoscaling thresholds.
- Produce a set of recommended configurations with explanations that teams can validate and roll out.
A continuous insights module then applies similar principles to live environments by:
- Consuming existing observability data from tools like Datadog, Dynatrace, or Prometheus rather than deploying new agents.
- Highlighting services that are overprovisioned, misconfigured, or at risk from reliability or performance issues.
- Providing ready-to-apply configuration suggestions for workloads, runtimes, and Kubernetes resources that align with real traffic patterns.
This approach turns scattered optimisation opportunities into actionable recommendations, reducing guesswork and allowing teams to focus on validating and governing changes rather than hunting for inefficiencies.
From Empowerment to Safe Automation
A practical optimisation journey typically unfolds in two stages. Initially, the focus is on empowering teams with clear, explainable recommendations they can review and apply through existing workflows, such as pull requests. Platform, SRE, or performance engineering teams can:
- Identify unreliable or wasteful services quickly.
- Open PRs with proposed configuration changes generated by the optimisation platform.
- Allow developers to review, test, and merge those changes while retaining ownership of their services.
As confidence grows and patterns stabilise, organisations can selectively automate parts of this loop. Over time, optimisation capabilities can become a native function of the platform itself, continuously:
- Adjusting resource requests and limits.
- Tuning runtime parameters in response to changing workloads.
- Respecting safety constraints and SLOs so that any automated change remains within agreed guardrails.
Technologies such as in-place Kubernetes pod resizing are early building blocks for this vision of real-time, AI-driven optimisation that is both continuous and safe.
The Future of Cloud Efficiency
The pressures on modern engineering teams are only increasing: faster delivery, higher reliability, tighter budgets, and growing sustainability expectations. At the same time, AI-assisted development and rapid release cycles risk widening the gap between how quickly software ships and how efficiently it runs.
Relying on overprovisioning and occasional manual tuning is no longer viable. Organisations will need systematic, data-driven approaches that treat optimisation as an ongoing, automated practice rather than a one-off project. Platforms that can bridge the gap between DevOps, FinOps, and runtime engineering — while respecting existing team boundaries and workflows — are well-positioned to become a core part of cloud operating models.
Whether full-stack, AI-driven optimisation becomes the universal standard remains to be seen, but the trend is clear: casual overprovisioning, zombie resources, and static “superhero” configurations are on borrowed time.
Read more such articles from our Newsletter here.
