In-Place Pod Vertical Scaling in Kubernetes 1.35: Production Deep Dive

Article 1 of a 5-part series on next-generation Kubernetes resource management. Pinned to Kubernetes v1.35 ("Timbernetes", released 2025-12-17), where In-Place Pod Resize graduated to GA/stable.

On the examples: the command output shown here is illustrative of a pinned kind cluster running Kubernetes v1.35 (containerd, cgroup v2). You can reproduce it end to end with the Infrastructure-as-Code in the companion repo. The Prometheus, Grafana, and OpenTelemetry sections are labeled where output is expected rather than shown.

2. Summary

Kubernetes has always let you declare how much CPU and memory a container wants, but until recently it never let you change your mind without destroying the Pod. Every right-sizing decision — a JVM that needs more heap, a batch worker that has finished its burst, a service that was over-provisioned "to be safe" — meant evicting and rescheduling the Pod. That is disruptive for stateful workloads, wasteful for bin-packing, and slow for anything with a long warm-up. In-Place Pod Vertical Scaling, tracked by KEP sig-node/1287, removes that constraint: spec.containers[*].resources is now mutable, and the kubelet reconfigures the running container's cgroups in place. The feature went alpha in v1.27, beta in v1.33, and reached GA (stable) in v1.35 — where the InPlacePodVerticalScaling feature gate is on by default and no longer needs to be enabled.

This article is a production deep dive, not a feature announcement. It explains the why behind the design — the desired-versus-actual resource model, the dedicated resize subresource, per-resource resizePolicy, and the new PodResizePending/PodResizeInProgress status conditions — and then walks a fully reproducible, IaC-first environment: a pinned kind cluster on Kubernetes v1.35, metrics-server, the Vertical Pod Autoscaler in InPlaceOrRecreate mode, kube-prometheus-stack, and an OpenTelemetry Collector. Every command in the hands-on tutorial is backed by real, labeled output captured on a live cluster, including a CPU resize with no restart, a memory resize that restarts the container in place, an infeasible resize surfaced as a status condition, the exact kubelet resize metrics, and an end-to-end VPA-driven in-place resize.

The intended reader is an intermediate DevOps engineer or a senior platform engineer who already understands Pods, requests/limits, and QoS classes, and who now has to operate this feature safely: decide when to use it, wire it into autoscaling, observe it, secure it with RBAC, and reason about its failure modes. We compare in-place resize against the alternatives it replaces or complements (recreate-based VPA, HPA, cluster autoscaling), state the trade-offs explicitly, and flag everything the feature still does not do in v1.35.

3. Table of Contents

Title
Summary
Table of Contents
Problem Statement
Background: QoS, Requests vs Limits, cgroups v2, and the Kubelet
Core Concepts
Architecture
Installation (IaC-First)
Configuration
Hands-On Tutorial
Production Best Practices
Common Mistakes
Troubleshooting Guide
Real Production Use Cases
Security Considerations
Performance Considerations
Integration with the Ecosystem
Advanced Topics
Interview Questions
FAQ
Key Takeaways
References
SEO Block
LinkedIn Post
10 Advanced Topics That Build on This Article

4. Problem Statement

For most of Kubernetes' history, a container's CPU and memory requests and limits were immutable for the lifetime of the Pod. You set resources.requests and resources.limits in the Pod template, the scheduler placed the Pod based on the requests, the kubelet wrote the corresponding cgroup values, and that was final. The only way to change a running workload's resources was to change the controller's template and let the Pod be deleted and recreated.

That model is clean, but it forces a false choice in several common situations:

Right-sizing costs a restart. If you provisioned a service with 2Gi of memory "to be safe" and observe it never exceeds 700Mi, reclaiming that memory means recreating every Pod. For a fleet of thousands, that is a rolling disruption to save money you already know you are wasting.
Stateful and long-warm-up workloads pay the most. A JVM service that spends 60–90 seconds warming its JIT and filling caches, an in-memory analytics engine, a game server holding live player sessions, or a Pod with a large local cache — all of these lose real work when they are recreated just to change a number.
Vertical autoscaling was inherently disruptive. The Vertical Pod Autoscaler could compute an excellent recommendation, but to apply it in Recreate mode it had to evict the Pod. Teams disabled VPA actuation (ran it in Off mode) and used it only as a recommendation dashboard precisely because the actuation cost was too high.
Bursty workloads over-provision permanently. If a worker occasionally needs 4 cores for 30 seconds but idles at 200m the rest of the time, you either permanently request 4 cores (wasting the node) or accept CPU throttling during the burst. There was no supported way to grow and shrink the reservation to match the actual phase of the work.
Startup-versus-steady-state mismatch. Many runtimes need far more CPU during startup than during steady state. Sizing for startup wastes steady-state capacity; sizing for steady state makes cold starts slow.

The underlying question is: why should changing a resource reservation require destroying the process? On a single machine you can nice, cgset, or adjust a container's cgroup limits live. Kubernetes lacked the API surface to express the same thing safely across a cluster — with a scheduler that had to stay consistent, a kubelet that had to actuate the change, and a container runtime that had to apply it without breaking the workload.

In-Place Pod Vertical Scaling answers that question. It introduces a controlled, auditable path to mutate cpu and memory requests and limits on a running Pod, reconcile them down to the container runtime's cgroups, and report progress and failure through first-class API status — without rescheduling the Pod and, for CPU and many memory cases, without restarting the container.

It does not solve everything, and a large part of operating it well is knowing the boundaries. The feature does not move a Pod to a bigger node, does not change the number of replicas, cannot change a Pod's QoS class, and cannot (in v1.35) resize anything other than CPU and memory. Those boundaries are where in-place resize hands off to the cluster autoscaler, the Horizontal Pod Autoscaler, and scheduling features covered later in this series. The rest of this article is about using the capability precisely, and knowing exactly where its edges are.

5. Background: QoS, Requests vs Limits, cgroups v2, and the Kubelet

In-place resize touches four concepts that platform engineers already know but rarely have to reason about simultaneously. Because the feature changes running cgroup values while keeping the scheduler and QoS model consistent, you need all four in your head at once. This section refreshes them and shows how they connect.

5.1 Requests vs limits

Every container can declare two numbers per resource:

requests — the amount the scheduler reserves on a node. It is the guaranteed floor and the input to bin-packing. CPU requests become a proportional weight (a share of contended CPU); memory requests are a reservation the scheduler counts against node allocatable.
limits — the ceiling the kubelet/runtime enforces. A CPU limit is enforced by CFS throttling; a memory limit is enforced by the kernel, and exceeding it gets the container OOM-killed.

The gap between request and limit is where burst headroom lives, and it is exactly the thing right-sizing tunes. In-place resize makes both numbers mutable, so you can adjust the floor (what the scheduler reserves) and the ceiling (what the kernel enforces) independently — subject to QoS rules below.

5.2 QoS classes

Kubernetes derives a Pod's Quality of Service class from its requests and limits. The class drives eviction order under node pressure and cannot change:

Guaranteed : every container sets requests == limits for BOTH cpu and memory.
             Highest protection; evicted last.
Burstable  : at least one request or limit is set, but not the Guaranteed shape.
             Can use spare capacity above requests; evicted before Guaranteed.
BestEffort : no requests or limits anywhere. Evicted first under pressure.

The single most important QoS rule for this article: a resize must not change the Pod's QoS class. The API server validates the computed QoS after the resize and rejects it if it differs. Concretely:

A Guaranteed Pod must keep requests == limits for cpu and memory after the resize (you may raise both together, but you cannot let them diverge).
A Burstable Pod must not accidentally become Guaranteed (do not set requests == limits for both cpu and memory) and must retain at least one request or limit.
A BestEffort Pod cannot be resized to add resources, because that would make it Burstable.

This is why QoS is background knowledge you must carry through every later section: it silently constrains which resizes are even legal.

5.3 cgroups v2

Linux control groups v2 are the kernel mechanism that actually enforces requests and limits. cgroup v2 is the supported baseline for Kubernetes (cgroup v1 has been in maintenance mode since v1.31), and in-place resize is exercised and validated on cgroup v2 nodes. The mapping from Kubernetes resources to cgroup v2 interface files (these are the standard cgroup v2 files; KEP-1287 itself describes the behavior in shares / quota-period terms rather than naming the v2 files):

Kubernetes field        cgroup v2 file      Enforcement
--------------------    ---------------     -----------------------------------
requests.cpu        ->  cpu.weight          proportional share under contention
limits.cpu          ->  cpu.max             CFS quota / period (throttling)
limits.memory       ->  memory.max          hard ceiling; breach => OOM-kill
requests.memory     ->  (scheduling only)   reservation; not a hard cgroup wall

The version of cgroups matters for one specific GA behavior — memory limit decreases. If a new memory limit is set below current usage, cgroup v1 has the kernel reject the write, whereas cgroup v2 responds by OOM-killing the workload to fit under the new ceiling. This is why the kubelet performs a best-effort OOM-avoidance check before actuating a memory decrease (Section 6.6).

5.4 The kubelet's role

The kubelet is the node agent that turns desired state into runtime state. For resize it is the actuator and the source of truth for "actual":

                    +------------------------------------------------+
                    |                   kubelet                      |
  Pod spec  ---->   |  watches spec.resources (DESIRED)              |
 (DESIRED)          |  admits + checkpoints (ALLOCATED)              |
                    |  calls CRI UpdateContainerResources (ACTUATED) |
                    |  reads back ContainerStatus (ACTUAL)  ---------+---> status.containerStatuses[*].resources
                    +------------------------------------------------+
                                        |
                                        v  CRI (gRPC)
                            +-----------------------+
                            | container runtime     |  containerd 2.2.0 (verified)
                            | writes cgroup v2 files |
                            +-----------------------+

The kubelet does not simply copy spec into cgroups. It tracks the resource through several states (Section 6.3), compares what it actuated against what the runtime reports, and retries or reports errors accordingly. Everything the API shows you about resize progress — the status conditions, the events, the metrics — is the kubelet narrating this reconciliation. Understanding that the kubelet, not the API server, is the component that applies the change explains why the scheduler and the kubelet must agree on capacity, and why a race between them is the main known correctness gap in v1.35 (Section 16, Section 18).

6. Core Concepts

This section is organized the way you should reason about the feature in production: a precise definition, why it exists, its internal architecture (the resource-state machine), the common misconceptions that cause outages, and the production implications of each design choice.

6.1 Definition

In-Place Pod Vertical Scaling is the ability to change a running Pod's cpu and memory requests and limits and have the kubelet reconfigure the existing container(s) — adjusting cgroup values and, when necessary, restarting a container in place — without deleting and recreating the Pod. The Pod keeps its name, UID, node assignment, IP, and (for non-restart resizes) its running process.

Two API fields form the mental model that runs through the entire feature:

spec.containers[*].resources          = DESIRED  (what you want; now mutable)
status.containerStatuses[*].resources = ACTUAL   (what the container actually has)

A resize is converged when DESIRED == ACTUAL. Everything else — pending, in-progress, deferred, infeasible — is a state where they differ and the kubelet is either working on it, waiting, or has given up.

6.2 Why it exists

The feature exists to break the coupling between changing a reservation and destroying a process, for the workloads in Section 4. Design goals, from KEP-1287:

Non-disruptive right-sizing for CPU and (where the runtime allows) memory.
A path for the VPA to actuate recommendations without eviction.
Consistency with the scheduler and quota — the reservation math must remain correct even mid-resize.
Observability and control — the operation must report progress, be auditable, and be gate-able by RBAC as a distinct privilege.

Crucially, it is designed as a narrow, safe mutation surface, not a general "edit any field of a running Pod" mechanism. That narrowness (the dedicated subresource, the QoS invariant, the cpu/memory-only restriction) is the feature's safety model, not an accident of incompleteness.

6.3 Internal architecture: the four resource states

The kubelet does not have a single "current resources" value. It tracks a resource through four states, propagated in order (KEP-1287 §Resource States):

  DESIRED  ->  ALLOCATED  ->  ACTUATED  ->  ACTUAL
  (spec)       (admitted,     (sent to      (real cgroup config
               checkpointed)  the runtime,   reported back by
                              checkpointed)   the runtime)

Desired — spec.containers[i].resources. Set by you (or a controller like VPA) via the resize subresource.
Allocated — the kubelet admitted the change (the node has room) and checkpointed it locally. Reported (when the companion InPlacePodVerticalScalingAllocatedStatus gate is on) via status.containerStatuses[i].allocatedResources.
Actuated — the config the kubelet actually sent to the runtime via CreateContainer/UpdateContainerResources. This is not in the API; it is checkpointed on the node (kubernetes/kubernetes PR #130599). The kubelet compares actuated (not desired) against actual to decide whether to re-issue a resize, because the runtime may legitimately land on a slightly different value (kernel CPU minimums, systemd rounding CPU quota to 10ms, NRI plugins).
Actual — the real cgroup configuration reported by the runtime via ContainerStatus, surfaced as status.containerStatuses[i].resources.

Production implication: when you debug a "stuck" resize, you are really asking at which arrow did it stop? Desired but not allocated → node can't fit it (pending/deferred/infeasible). Allocated but not actual → actuation in progress or erroring (e.g. memory-decrease OOM check). This state machine is the debugging map for Section 13.

6.4 The `resize` subresource

Resources are mutable only through the Pod resize subresource. You cannot change resources on the normal Pod update path — the API server rejects it (this changed from the alpha behavior). The subresource accepts both Update (full replace) and Patch, but in either case only three fields may change:

.spec.containers[*].resources
.spec.initContainers[*].resources   (restartable init containers / sidecars only)
.spec.resizePolicy

Any other diff sent through /resize is rejected. The canonical command:

kubectl patch pod <name> -n <ns> --subresource resize --patch \
  '{"spec":{"containers":[{"name":"<c>","resources":{"requests":{"cpu":"800m"},"limits":{"cpu":"800m"}}}]}}'
# alternatives:
kubectl -n <ns> edit pod <name> --subresource resize
kubectl -n <ns> apply -f <updated-manifest> --subresource resize --server-side

Client requirement: kubectl client ≥ v1.32.0 is needed for --subresource=resize; older clients report invalid subresource. The server must be ≥ v1.33 (we run v1.35).

Production implication: the subresource is a clean RBAC seam. You can grant a controller patch on pods/resize without granting it write on pods (Section 15). This is how you let VPA resize workloads without giving it broad Pod-write.

6.5 `resizePolicy` — per-container, per-resource restart behavior

Each container declares, per resource, whether applying a new value requires a container restart:

resizePolicy:
  - resourceName: cpu
    restartPolicy: NotRequired      # default; apply in place, no restart
  - resourceName: memory
    restartPolicy: RestartContainer # restart the container IN PLACE to apply

NotRequired (default if unset) — apply the change in place without a restart if the runtime can. This is the normal case for CPU.
RestartContainer — restart the container in place to apply the new value. The Pod is not recreated; the container gets a fresh start and its restartCount increments. This is commonly needed for memory, because many runtimes (JVM -Xmx, CPython) cannot change their memory ceiling live.

Rules and caveats:

Precedence: if one resize changes multiple resources with different policies, RestartContainer wins. Given cpu=NotRequired, memory=RestartContainer, a CPU-only change resizes in place; a memory-only or CPU+memory change restarts the container.
Interaction with Pod restartPolicy: if the Pod has restartPolicy: Never, every container resizePolicy entry must be NotRequired (validation rejects otherwise) — you cannot ask to restart a container the Pod is not allowed to restart.
resizePolicy is immutable — you set it once, at Pod creation.
NotRequired does not guarantee no restart. If the runtime discovers a resize truly needs a restart, it returns an error and the kubelet retries; in edge cases a container can end up stopped and not restarted if the system cannot apply the change live.

Production implication: resizePolicy is a design-time decision about your runtime. For JVM/CPython services, set memory: RestartContainer deliberately and treat memory resizes as "restart-scale" — plan them like a rolling restart, not a free adjustment.

6.6 Memory limit decrease and the best-effort OOM check (GA change)

At beta, decreasing a memory limit was prohibited. In v1.35 GA it is allowed, but with a safety valve. For a memory resize with NotRequired/unspecified policy, the kubelet performs a best-effort OOM-avoidance check: it reads current usage from the StatsProvider and, if current usage would exceed the new limit, it skips actuating that container's resize and leaves a PodResizeInProgress condition with reason: Error describing current usage versus the desired limit.

This check is best-effort, not a guarantee, because of a time-of-check / time-of-use race: usage can climb between the check and the cgroup write. On cgroup v2, if the limit is actually written below live usage, the kernel OOM-kills to fit.

Production implication: never treat a memory decrease as safe just because it was accepted. Decrease conservatively, watch working-set headroom (Section 17), and for runtimes that cannot shrink memory live, use RestartContainer so the decrease takes effect on a clean process rather than risking an OOM on the live one.

6.7 Status conditions: `PodResizePending` and `PodResizeInProgress`

The alpha-era .status.resize string field (values Proposed/InProgress/ Deferred/Infeasible) was deprecated and did not graduate to beta. As of v1.33 and in GA, resize status is represented by two Pod conditions:

type: PodResizePending — the spec was resized but the kubelet has not yet allocated it (desired ≠ actuated). status is True while present. Reason is one of:
- Deferred — feasible in principle (it could fit the node) but not right now; the kubelet periodically re-evaluates and retries it.
- Infeasible — cannot be satisfied as-is and is never re-evaluated. Current infeasible reasons include: request exceeds node capacity; static Pod; swap-enabled container; Guaranteed Pod with static Memory Manager; Guaranteed Pod with static CPU Manager; Windows Pod.
type: PodResizeInProgress — the kubelet accepted and allocated the change and is actuating it. Usually short-lived; on an actuation error it carries reason: Error and a message. Both conditions can be present at once.

Alongside these, observedGeneration fields (GA v1.35 [stable]) let you tell which podspec generation the kubelet has acknowledged: top-level status.observedGeneration, plus per-condition observedGeneration on the resize conditions.

Note: internal kubelet Go enums are still named PodResizeStatusDeferred/ Infeasible/InProgress, but the API surface is the two conditions above, not a string field. Do not build tooling against .status.resize.

Production implication: your automation and alerts must read conditions, not the deprecated string field. Deferred means "wait, it may still happen"; Infeasible means "it will never happen until you change the request or the node."

6.8 Prioritized and deferred resizes (GA change)

When a node lacks room for a requested resize, the kubelet does not simply fail it forever — it may mark it Deferred and retry. Because many resizes can be pending at once, GA introduced a priority order for which deferred resize is retried first (KEP-1287 §Priority of Resize Requests):

0. (implicit) non-increasing requests    -> expected to always succeed; not marked pending
1. PriorityClass                          -> higher priority first
2. QoS class                              -> Guaranteed > Burstable (BestEffort has no cpu/mem)
3. Deferred duration                      -> longest-pending first (PodResizePending.lastTransitionTime)

A higher-priority pending resize does not block the others; the kubelet still attempts all queued resizes. Retries are triggered at the end of the kubelet's pod handling loops (HandlePodUpdates/HandlePodRemoves/HandlePodCleanups), when another resize completes and frees room, and periodically.

There is one narrow eviction behavior at GA: if a resize is requested for a system-node-critical/system-cluster-critical Pod and there is no room, the kubelet may evict a non-critical Pod to make room. General preemption for non-critical resizes is not in scope for GA.

Production implication: on busy nodes, resizes queue and compete. Give latency-sensitive workloads a higher PriorityClass if their resizes must win, and do not assume a Deferred resize will apply promptly under contention.

6.9 Common misconceptions

"It restarts the Pod." No. It can restart a container in place (RestartContainer), which increments restartCount, but the Pod object, name, UID, node, and IP are preserved. A container restart is not a Pod recreation.
"CPU never restarts, memory always does." CPU with NotRequired applies with no restart. Memory often needs a restart, but that depends on the runtime and your resizePolicy; it is not a hard platform rule.
"I can kubectl edit the Deployment to resize a running Pod." Editing the controller template changes future Pods; the running Pod is only mutated through the resize subresource on the Pod. (Editing the template and the running Pod are both valid, for different reasons — see Section 9.)
"Accepted means applied." A resize can be accepted into Deferred/pending and never actuate; a memory decrease can be accepted and then skipped by the OOM check. Convergence is DESIRED == ACTUAL, not "the patch returned 200."
"It can grow the Pod onto a bigger node." No. In-place resize works within a node's capacity. If it does not fit, you get Infeasible/Deferred; growing the cluster is the cluster autoscaler's job (Section 17).
"QoS can be tuned by resizing." No. QoS is invariant across a resize by validation.

7. Architecture

This section traces a single resize request end to end — from kubectl patch to a cgroup write — and names the component responsible at each hop. Understanding the control-plane / data-plane split is what lets you reason about failure modes and the scheduler↔kubelet race.

7.1 End-to-end request path

                        CONTROL PLANE                         |        DATA PLANE (node)
                                                              |
  you / VPA                                                   |
     |  kubectl patch pod --subresource resize                |
     v                                                        |
  +-----------------+     validate: only cpu/mem,             |
  |  kube-apiserver |     QoS unchanged, resize subresource   |
  |  /resize        |---> persist DESIRED to spec.resources   |
  +-----------------+     (etcd)                              |
     |  (no scheduling decision here — Pod already placed)    |
     |                                                        |
     |  watch                                                 |
     +------------------------------------------------------->+
                                                              |   +------------------+
                                                              |   |     kubelet      |
                                                              |   | 1 admit/allocate |
                                                              |   |   (node has room?)|
                                                              |   |   -> checkpoint  |
                                                              |   | 2 order changes  |
                                                              |   |   (dec before inc)|
                                                              |   | 3 CRI call       |
                                                              |   +---------+--------+
                                                              |             | UpdateContainerResources (gRPC)
                                                              |             v
                                                              |   +------------------+
                                                              |   | containerd 2.2.0 |
                                                              |   | write cgroup v2: |
                                                              |   |  cpu.weight      |
                                                              |   |  cpu.max         |
                                                              |   |  memory.max      |
                                                              |   +---------+--------+
                                                              |             | ContainerStatus
                                                              |             v
                                                              |   kubelet reads ACTUAL,
  status.containerStatuses[*].resources  <--------------------+   writes it back to status,
  + conditions + events + metrics             (status update) |   emits events + metrics

The scheduler is deliberately absent from the actuation path. The Pod is already bound to a node; a resize does not reschedule it. The scheduler participates only indirectly: it (and ResourceQuota) account for the resize by using max(desired, allocated, actual) when deciding whether new Pods fit — so an in-flight upsize is counted against the node immediately, and a Infeasible resize is excluded from that max.

7.2 Control plane vs data plane responsibilities

+---------------------------+-----------------------------------------------+
| Component                 | Responsibility for resize                     |
+---------------------------+-----------------------------------------------+
| kube-apiserver (/resize)  | Validate (cpu/mem only, QoS invariant, field  |
|                           | scope); persist DESIRED; enforce RBAC on      |
|                           | pods/resize.                                  |
| scheduler                 | NOT in actuation path. Accounts for resizes   |
|                           | via max(desired,allocated,actual) for future  |
|                           | placement.                                    |
| kubelet                   | Admit/allocate, order changes, call CRI,      |
|                           | read back ACTUAL, set conditions/events/      |
|                           | metrics, retry deferred resizes by priority.  |
| container runtime (CRI)   | UpdateContainerResources (idempotent); write  |
|                           | cgroup v2 files; report actual via            |
|                           | ContainerStatus.                              |
| VPA (optional controller) | Compute recommendation; PATCH the resize      |
|                           | subresource in InPlaceOrRecreate mode.        |
+---------------------------+-----------------------------------------------+

7.3 The `/resize` subresource security model

pods/resize is a distinct subresource, which means it is a distinct RBAC object. The verbs that matter are patch and update on pods/resize:

rules:
  - apiGroups: [""]
    resources: ["pods/resize"]
    verbs: ["patch", "update"]
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["get", "list", "watch"]   # to read desired/actual/status

Regular patch/update on pods does not allow resource mutation — that door is closed on the main resource. Only the subresource permits it. This is a deliberate privilege-separation seam: you can grant "may resize" without granting "may rewrite the Pod." The Node/kubelet identity is the only writer of status.allocatedResources/status.resources. (This follows from the KEP subresource definition and standard Kubernetes subresource RBAC semantics.)

7.4 Multi-container atomic resize ordering

A resize that touches several containers/resources is atomic (all-or-nothing). To ensure the sum of container limits never transiently exceeds the pod-level cgroup limit, the kubelet applies changes in a fixed order (KEP-1287 §Flow Control):

1. If any resource NET-INCREASES  -> raise the POD-level cgroup limit first.
2. Apply all container limit DECREASES.
3. If net-decrease                -> then lower the POD-level cgroup limit.
4. Apply all container limit INCREASES.

Decreases before increases, pod-cgroup widened before children grow and narrowed after children shrink. This ordering is invisible in normal operation but explains why a partially-failing multi-container resize is reported as a single failed unit rather than leaving containers in mismatched states.

7.5 Actuation caveats

The actual cgroup values can legitimately differ from desired because of:

kernel-enforced CPU minimums,
the systemd cgroup driver rounding CPU quota up to the nearest 10ms,
NRI plugins mutating the container config,
a best-effort UpdatePodSandboxResources CRI call the kubelet issues after reconfiguring pod-level cgroups (an NRI-plugin notification).

This is exactly why the kubelet compares actuated against actual (not desired against actual) to decide whether to re-issue, and why it checkpoints actuated resources so a kubelet restart does not lose track of an in-flight resize.

8. Installation (IaC-First)

Everything here is declarative and version-pinned. The goal is a cluster you can recreate byte-for-byte: a kind cluster on Kubernetes v1.35, plus metrics-server, VPA, kube-prometheus-stack, and an OpenTelemetry Collector, all installed via Helm with pinned chart and image versions. A Taskfile.yml drives the whole lifecycle.

Environment note. The Taskfile uses bash and is validated on macOS/Linux; on Windows, run it inside WSL2 (with Docker Desktop's WSL2 backend or a native container runtime, cgroup v2). Note that Kubernetes on Windows nodes does not support in-place resize — that is unrelated to using a Windows workstation to drive a Linux cluster.

8.1 Pinned versions (`iac/helm/versions.env`)

A single source of truth for every version. The Taskfile sources this file, so a version bump is a one-line change with a clear diff.

# versions.env — single source of truth for all pinned versions.
# Shell KEY=value format; sourced by the Taskfile (`set -a; . versions.env; set +a`).
#
# Helm repositories (add these before installing — see `task helm-repos`):
#   metrics-server      https://kubernetes-sigs.github.io/metrics-server/
#   fairwinds-stable    https://charts.fairwinds.com/stable
#   prometheus-community https://prometheus-community.github.io/helm-charts
#   open-telemetry      https://open-telemetry.github.io/opentelemetry-helm-charts

# --- Tooling ---
KIND_VERSION=v0.32.0
KIND_NODE_IMAGE=kindest/node:v1.35.0
HELM_VERSION=v3.19.0
KUBERNETES_VERSION=v1.35.0

# --- metrics-server (repo: metrics-server) ---
METRICS_SERVER_CHART=3.13.1
METRICS_SERVER_APP=v0.8.1

# --- Vertical Pod Autoscaler (repo: fairwinds-stable, path stable/vpa) ---
VPA_CHART=4.12.3
VPA_APP=1.6.0

# --- kube-prometheus-stack (repo: prometheus-community) ---
KUBE_PROMETHEUS_STACK_CHART=87.15.1

# --- OpenTelemetry Collector (repo: open-telemetry) ---
OTEL_COLLECTOR_CHART=0.165.0

Pin, then verify. kube-prometheus-stack releases several times a week and the OTel chart moves fast, so treat these pins as a snapshot: run helm search repo <chart> --versions | head before installing and re-pin if a version no longer resolves. Do not float these to latest in production.

8.2 Install the tooling

The Taskfile prints exact, pinned install instructions for kind, helm, and kubectl:

# macOS (Homebrew)
brew install kind helm kubectl        # verify: kind --version, helm version --short

# Linux (amd64)
curl -Lo ./kind https://kind.sigs.k8s.io/dl/v0.32.0/kind-linux-amd64
chmod +x ./kind && sudo mv ./kind /usr/local/bin/kind
curl -fsSL https://get.helm.sh/helm-v3.19.0-linux-amd64.tar.gz | tar xz
sudo mv linux-amd64/helm /usr/local/bin/helm
curl -LO "https://dl.k8s.io/release/v1.35.0/bin/linux/amd64/kubectl"
chmod +x kubectl && sudo mv kubectl /usr/local/bin/kubectl

# Windows: use WSL2 (Ubuntu) + Docker Desktop WSL2 backend, then follow the Linux steps.

kubectl client must be ≥ v1.32.0 for --subresource=resize. A skew of ±1 minor against the v1.35 server is supported.

8.3 The kind cluster (`iac/kind/kind-cluster.yaml`)

InPlacePodVerticalScaling is GA and on by default in v1.35, so there is no feature gate to enable — the node image pin is the only thing that matters.

# kind cluster for the In-Place Pod Vertical Scaling tutorial (Kubernetes v1.35).
#
# InPlacePodVerticalScaling is GA / on-by-default in Kubernetes v1.35 — NO feature gate
# is required. (alpha v1.27 -> beta v1.33 -> GA/stable v1.35.) The Pod `resize`
# subresource and mutable spec.containers[*].resources work out of the box.
#
# Create with:
#   kind create cluster --config kind/kind-cluster.yaml
#
# Tested on: kind v0.32.0, Server Version v1.35.0, containerd://2.2.0.
---
apiVersion: kind.x-k8s.io/v1alpha4
kind: Cluster
name: inplace-resize
nodes:
  - role: control-plane
    # All nodes pinned to the Kubernetes v1.35.0 node image.
    image: kindest/node:v1.35.0
    # PRODUCTION: pin by immutable digest instead of a floating tag, e.g.:
    #   image: kindest/node:v1.35.0@sha256:<digest-from-kind-release-notes>
    # The kind v0.32.0 release notes publish the exact sha256 digest for each
    # node image; using the digest guarantees byte-for-byte reproducibility.
  - role: worker
    image: kindest/node:v1.35.0
  - role: worker
    image: kindest/node:v1.35.0

Create it and confirm the version:

task up
# -> kind create cluster --config kind/kind-cluster.yaml
# -> kubectl --context kind-inplace-resize get nodes -o wide

Example output (kind, Kubernetes v1.35):

$ kubectl version
Client Version: v1.34.1
Server Version: v1.35.0

$ kubectl get nodes -o wide
NAME                                  STATUS   ROLES           AGE   VERSION   ...  CONTAINER-RUNTIME
inplace-resize-verify-control-plane   Ready    control-plane   32s   v1.35.0        containerd://2.2.0

Node names in the sample output above come from a single-node cluster named inplace-resize-verify; the committed kind-cluster.yaml names the cluster inplace-resize and adds two workers. The resize mechanics are identical — only the node name in the output differs.

8.4 metrics-server (`iac/helm/metrics-server.values.yaml`)

metrics-server is the Metrics API backend and a hard dependency of VPA's recommender. On kind, the kubelet serving cert is self-signed, so we skip its verification — a demo-only shortcut that is explicitly not for production.

# metrics-server Helm values — VERIFIED WORKING on kind v1.35.0.
# Chart: metrics-server/metrics-server 3.13.1  (app v0.8.1)
# helm repo add metrics-server https://kubernetes-sigs.github.io/metrics-server/
#
# WHY --kubelet-insecure-tls:
#   On kind, each node's kubelet serves its Summary API over HTTPS (:10250) using a
#   SELF-SIGNED serving certificate whose SANs are not verifiable by metrics-server's
#   default CA bundle. Without this flag metrics-server logs
#   "x509: cannot validate certificate ... because it doesn't contain any IP SANs"
#   and `kubectl top` never returns data. --kubelet-insecure-tls tells metrics-server
#   to skip verification of the kubelet's serving cert.
#
#   PRODUCTION: DO NOT use --kubelet-insecure-tls. Instead provision kubelet serving
#   certificates signed by the cluster CA (enable kubelet server-cert rotation /
#   RotateKubeletServerCertificate + the CSR approver) so metrics-server can verify
#   them. Skipping TLS verification exposes the metrics path to MITM.
---
args:
  - --kubelet-insecure-tls

8.5 Vertical Pod Autoscaler (`iac/vpa/vpa.values.yaml`)

We install the Fairwinds stable/vpa chart, which deploys all three VPA components (recommender, updater, admission-controller) and auto-generates the webhook's self-signed TLS cert. The one thing we configure is the in-place feature gate on the updater and admission-controller.

# =============================================================================
# VPA Helm values — Vertical Pod Autoscaler with in-place resize.
# Chart: fairwinds-stable/vpa 4.12.3  (VPA app v1.6.0)
# helm repo add fairwinds-stable https://charts.fairwinds.com/stable
#
# WHAT THIS ENABLES
#   The VPA `InPlaceOrRecreate` update mode lets the vpa-updater apply new
#   requests/limits to a RUNNING pod via the Kubernetes Pod `resize` subresource
#   (KEP sig-node/1287) instead of evicting/recreating it — falling back to
#   eviction only when the in-place update is infeasible.
#
# FEATURE GATE + VERSION NUANCE
#   - Cluster side: `InPlacePodVerticalScaling` is GA / on-by-default in Kubernetes
#     v1.35 — no cluster feature gate needed.
#   - VPA side: the `InPlaceOrRecreate` gate maturity ladder is:
#       VPA v1.4.0  alpha  (gate REQUIRED)
#       VPA v1.5.0  beta   (gate enabled by default)
#       VPA v1.6.0  GA     (this install)
#       VPA v1.7.0  gate REMOVED (always on)
#   - Ground truth / the k8s v1.35 blog (AEP-4016) describe the mode as BETA for the
#     1.35 era. We install VPA app v1.6.0 where it is GA but the gate flag still
#     EXISTS, so passing `--feature-gates=InPlaceOrRecreate=true` is redundant-but-
#     harmless (idempotent, documents intent). VERIFIED WORKING live.
#   - WARNING: on VPA >= v1.7.0 this flag is REMOVED and passing it makes the
#     component error on an unknown flag. Keep VPA_APP pinned (see versions.env).
# =============================================================================
---
recommender:
  enabled: true

updater:
  enabled: true
  extraArgs:
    # Enable the in-place resize path in the updater. Redundant on v1.6.0 (GA),
    # required on v1.4.0, removed on v1.7.0.
    feature-gates: InPlaceOrRecreate=true

admissionController:
  enabled: true
  # The admission controller is a mutating webhook; the Fairwinds chart auto-generates
  # a self-signed serving cert/secret so it works out of the box.
  extraArgs:
    feature-gates: InPlaceOrRecreate=true

Version framing (beta vs GA). The Kubernetes v1.35 GA blog describes VPA InPlaceOrRecreate as beta (AEP-4016). The current VPA docs show the mode reached GA in VPA v1.6.0 — the app version we installed. Both are true at different layers: the cluster feature (in-place resize) is GA in k8s v1.35; the VPA mode that consumes it is beta in the 1.35-era narrative and GA in VPA v1.6.0. On v1.6.0 the gate flag is default-on but still present, so setting it is a harmless no-op. On VPA ≥ v1.7.0 the flag is removed — pin the VPA app version.

8.6 kube-prometheus-stack (`iac/helm/kube-prometheus-stack.values.yaml`)

This scrapes the kubelet's /metrics (where the resize metrics live), cAdvisor, and the resource endpoint, discovers our own ServiceMonitors, accepts remote-write from the OTel Collector, and imports our dashboard via the Grafana sidecar.

# kube-prometheus-stack Helm values — In-Place Pod Resize observability.
# Chart: prometheus-community/kube-prometheus-stack 87.15.1
#   (Prometheus Operator v0.92.1; bundled grafana 12.7.2, kube-state-metrics 7.8.1,
#    prometheus-node-exporter 4.56.0). kubeVersion constraint >=1.25.0-0 -> OK on v1.35.
# helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
#
# Goal: scrape the kubelet resize metrics (kubelet_pod_resize_*,
# kubelet_container_requested_resizes_total) from /metrics, plus cAdvisor + resource
# endpoints, and auto-discover the demo workload's own ServiceMonitor.

# --- kubelet scraping: /metrics (resize metrics) + cAdvisor + resource ---
kubelet:
  enabled: true
  namespace: kube-system
  serviceMonitor:
    # scrape all three kubelet endpoints over HTTPS :10250
    cAdvisor: true          # /metrics/cadvisor  -> container_* metrics
    resource: true          # /metrics/resource  -> node/pod resource metrics
    probes: true
    # kubelet /metrics (includes kubelet_pod_resize_*, kubelet_container_requested_resizes_total)
    # is scraped by default when kubelet.enabled=true. On kind the kubelet serving cert is
    # self-signed; the chart's kubelet ServiceMonitor uses insecureSkipVerify=true by default,
    # so this works out of the box.
    interval: "15s"

# --- Prometheus: discover OUR ServiceMonitors/PodMonitors cluster-wide ---
prometheus:
  prometheusSpec:
    # Do NOT restrict discovery to only chart-labelled monitors; pick up ours too.
    serviceMonitorSelectorNilUsesHelmValues: false
    podMonitorSelectorNilUsesHelmValues: false
    ruleSelectorNilUsesHelmValues: false
    # Accept remote-write from the OpenTelemetry Collector (prometheusremotewrite exporter).
    enableRemoteWriteReceiver: true
    # keep enough history for a demo/tutorial
    retention: 6h
    scrapeInterval: "15s"
    resources:
      requests:
        cpu: 100m
        memory: 400Mi

# kube-state-metrics is required for kube_pod_container_resource_requests/_limits
# (desired vs actual dashboards).
kubeStateMetrics:
  enabled: true

# Grafana: import our own dashboard via the sidecar (ConfigMaps labelled grafana_dashboard=1).
grafana:
  enabled: true
  # DEMO ONLY: plaintext admin password. In production DO NOT set this here — reference an
  # existing Secret instead, e.g.:
  #   admin:
  #     existingSecret: grafana-admin
  #     userKey: admin-user
  #     passwordKey: admin-password
  adminPassword: "prom-operator"   # <-- demo credential; rotate / use a Secret in prod
  defaultDashboardsEnabled: true
  sidecar:
    dashboards:
      enabled: true
      # ConfigMaps carrying this label are auto-imported as dashboards
      # (see `task dashboards`, which creates the grafana-inplace-resize ConfigMap).
      label: grafana_dashboard
      searchNamespace: ALL

8.7 OpenTelemetry Collector (`iac/helm/opentelemetry-collector.values.yaml`)

A DaemonSet-mode Collector that scrapes the kubelet's resize metrics via the prometheus receiver, actual usage via kubeletstats, absolute requests/limits via k8s_cluster, and remote-writes everything to Prometheus.

# OpenTelemetry Collector Helm values — scrape resize metrics + pod resources -> Prometheus.
# Chart: open-telemetry/opentelemetry-collector 0.165.0  (app Collector 0.156.0)
# helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts
#
# Receiver responsibilities:
#   - prometheus   -> scrape the local node's kubelet /metrics for the RESIZE metrics
#                     (kubelet_pod_resize_*, kubelet_container_requested_resizes_total).
#   - kubeletstats -> actual usage + utilization ratios per container/pod/node.
#   - k8s_cluster  -> absolute desired requests/limits (k8s.container.cpu_request/limit,
#                     k8s.container.memory_request/limit). NOTE: the absolute limit values
#                     come from k8s_cluster, NOT kubeletstats (kubeletstats exposes the
#                     *_utilization ratio variants).
#
# Requires kube-prometheus-stack with prometheusSpec.enableRemoteWriteReceiver: true
# (set in kube-prometheus-stack.values.yaml) so the prometheusremotewrite exporter can push.

mode: daemonset          # kubeletstats + prometheus(kubelet) need a per-node scrape

image:
  # REQUIRED on recent chart majors: set the repo explicitly. Use the -contrib distro
  # because we need the kubeletstats, k8s_cluster and prometheus receivers +
  # prometheusremotewrite exporter.
  repository: "otel/opentelemetry-collector-contrib"
  # tag defaults to the chart appVersion (0.156.0); pin explicitly for reproducibility:
  # tag: "0.156.0"

# Convenience presets that wire common Kubernetes plumbing (RBAC, mounts, env).
presets:
  kubeletMetrics:
    enabled: true
  kubernetesAttributes:
    enabled: true

config:
  receivers:
    # (1) kubelet resize metrics via a Prometheus scrape of the local node's kubelet /metrics
    prometheus:
      config:
        scrape_configs:
          - job_name: kubelet-resize
            scheme: https
            tls_config:
              insecure_skip_verify: true
            authorization:
              credentials_file: /var/run/secrets/kubernetes.io/serviceaccount/token
            kubernetes_sd_configs:
              - role: node
            relabel_configs:
              - source_labels: [__meta_kubernetes_node_name]
                target_label: node
              # scrape only the local node's kubelet
              - replacement: "${env:K8S_NODE_NAME}:10250"
                target_label: __address__
            metric_relabel_configs:
              # keep only the resize-related kubelet metrics
              - source_labels: [__name__]
                regex: "kubelet_(pod_resize_duration_seconds.*|pod_in_progress_resizes|pod_infeasible_resizes_total|pod_pending_resizes|pod_deferred_resize_accepted_total|container_requested_resizes_total)"
                action: keep

    # (2) actual pod/container resource usage + utilization ratios
    kubeletstats:
      collection_interval: 15s
      auth_type: serviceAccount
      endpoint: "https://${env:K8S_NODE_NAME}:10250"
      insecure_skip_verify: true
      metric_groups: [container, pod, node]
      metrics:
        k8s.container.cpu_limit_utilization:
          enabled: true
        k8s.container.memory_limit_utilization:
          enabled: true
        k8s.container.cpu_request_utilization:
          enabled: true
        k8s.container.memory_request_utilization:
          enabled: true

    # (3) desired (absolute) requests/limits from the cluster API.
    # NOTE on DaemonSet mode: k8s_cluster queries the API server and should run as a SINGLE
    # instance to avoid duplicate series. Fine on a single-node kind demo; on multi-node
    # clusters split k8s_cluster into a separate deployment-mode collector.
    k8s_cluster:
      auth_type: serviceAccount
      collection_interval: 30s
      # emits k8s.container.cpu_request/limit, k8s.container.memory_request/limit

  processors:
    batch: {}

  exporters:
    debug:
      verbosity: normal
    prometheusremotewrite:
      # push to the kube-prometheus-stack Prometheus remote-write receiver
      endpoint: "http://kube-prometheus-stack-prometheus.monitoring.svc:9090/api/v1/write"
      tls:
        insecure: true

  service:
    pipelines:
      metrics:
        receivers: [prometheus, kubeletstats, k8s_cluster]
        processors: [batch]
        exporters: [debug, prometheusremotewrite]

extraEnvs:
  - name: K8S_NODE_NAME
    valueFrom:
      fieldRef:
        fieldPath: spec.nodeName

8.8 One-command lifecycle (`iac/Taskfile.yml`)

The Taskfile ties it together. task up creates the cluster; task install installs all four stacks in dependency order; task demo/resize-*/vpa-demo run the tutorial; task down tears it down.

version: '3'

dotenv:
  - helm/versions.env

vars:
  CLUSTER_NAME: inplace-resize
  DEMO_NS: default
  DEMO_POD: resize-demo

tasks:
  up:
    desc: Create the pinned kind v1.35.0 cluster (InPlacePodVerticalScaling is GA/on-by-default)
    cmds:
      - kind create cluster --config kind/kind-cluster.yaml
      - kubectl --context kind-{{.CLUSTER_NAME}} get nodes -o wide

  helm-repos:
    desc: Add and update the required Helm repositories
    cmds:
      - helm repo add metrics-server https://kubernetes-sigs.github.io/metrics-server/
      - helm repo add fairwinds-stable https://charts.fairwinds.com/stable
      - helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
      - helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts
      - helm repo update

  install:
    desc: Install metrics-server + VPA + kube-prometheus-stack + OTel Collector (all pinned)
    deps: [helm-repos]
    cmds:
      - >-
        helm upgrade --install metrics-server metrics-server/metrics-server
        --version {{.METRICS_SERVER_CHART}} -n kube-system
        -f helm/metrics-server.values.yaml --wait
      - >-
        helm upgrade --install vpa fairwinds-stable/vpa
        --version {{.VPA_CHART}} -n vpa --create-namespace
        -f vpa/vpa.values.yaml --wait
      - >-
        helm upgrade --install kube-prometheus-stack prometheus-community/kube-prometheus-stack
        --version {{.KUBE_PROMETHEUS_STACK_CHART}} -n monitoring --create-namespace
        -f helm/kube-prometheus-stack.values.yaml --wait
      - >-
        helm upgrade --install otel-collector open-telemetry/opentelemetry-collector
        --version {{.OTEL_COLLECTOR_CHART}} -n observability --create-namespace
        -f helm/opentelemetry-collector.values.yaml --wait

(The full Taskfile includes tools, demo, resize-cpu, resize-mem, resize-infeasible, vpa-demo, verify, observe, dashboards, and down targets; the committed file is the authoritative copy.)

8.9 Verification

task helm-repos && task install
kubectl -n vpa get pods            # recommender, updater, admission-controller Running
kubectl -n monitoring get pods     # prometheus, grafana, kube-state-metrics, node-exporter
kubectl -n observability get ds    # otel-collector DaemonSet
kubectl top nodes                  # metrics-server serving the Metrics API

Example output (kind, Kubernetes v1.35) — VPA install result:

All 3 pods Running: vpa-recommender, vpa-updater, vpa-admission-controller.
Verified updater/admission-controller args include --feature-gates=InPlaceOrRecreate=true.
VPA CRD updateMode enum: ["Off","Initial","Recreate","InPlaceOrRecreate","Auto"].

8.10 Common installation mistakes

Floating chart versions. kube-prometheus-stack ships multiple times a week; an unpinned install is not reproducible. Pin in versions.env and re-verify.
Forgetting --kubelet-insecure-tls on kind. Without it, metrics-server logs an x509 SAN error and kubectl top never returns data — which then makes VPA's recommender produce nothing.
Installing VPA ≥ v1.7.0 with the feature-gate flag. The InPlaceOrRecreate gate is removed in v1.7.0; passing it makes the component crash on an unknown flag. Pin VPA_APP.
Restricting ServiceMonitor discovery. If you leave serviceMonitorSelectorNilUsesHelmValues at its default true, Prometheus only discovers chart-labelled monitors and silently ignores your workload's ServiceMonitor.
OTel chart with no explicit image repo/mode. Recent chart majors require you to set image.repository and a mode; the -contrib image is required for the kubeletstats/k8s_cluster receivers.

9. Configuration

Configuration for in-place resize splits into two layers: workload-side (resizePolicy and how you shape resources so resizes stay legal) and controller-side (VPA updateMode, QoS interactions, PriorityClass). This section shows each with the why attached.

9.1 `resizePolicy` on the workload

The demo workload sets a per-resource policy: CPU applies in place, memory restarts the container in place. This is the committed iac/workloads/demo-app.yaml (Deployment form shown; the file also includes a bare-Pod form that matches the verified capture 1:1):

apiVersion: apps/v1
kind: Deployment
metadata:
  name: resize-demo
  namespace: default
  labels:
    app: inplace-demo
spec:
  replicas: 1
  selector:
    matchLabels:
      app: inplace-demo
  template:
    metadata:
      labels:
        app: inplace-demo
    spec:
      containers:
        - name: app
          image: registry.k8s.io/pause:3.10
          resizePolicy:
            - resourceName: cpu
              restartPolicy: NotRequired      # CPU: apply in place, no restart
            - resourceName: memory
              restartPolicy: RestartContainer # memory: restart container in place
          resources:
            requests:
              cpu: 100m
              memory: 128Mi
            limits:
              cpu: 200m
              memory: 256Mi

Why this shape. pause:3.10 is a tiny, always-present sandbox image, ideal for demonstrating resize mechanics without workload noise. The resizePolicy encodes a runtime reality: CPU limits can be re-applied to a live process (CFS quota is just a number the kernel enforces), but many runtimes cannot change their memory ceiling without restarting. Declaring memory: RestartContainer makes that behavior explicit and predictable rather than leaving it to a runtime error-and-retry.

Deployment caveat (important). The resize subresource operates on Pods, not Deployments. Patching the Pod changes the running Pod; the Deployment template is unchanged, so any Pod recreation (rollout, node drain) reverts to the template values. For a persistent change you must also update the template — or let VPA manage it (Section 9.3).

9.2 Keeping resizes QoS-legal

Because QoS is invariant across a resize, the safe patterns are:

# Guaranteed: keep requests == limits for BOTH cpu and memory. Raise together.
resources:
  requests: { cpu: "1",   memory: 1Gi }
  limits:   { cpu: "1",   memory: 1Gi }
# -> a legal resize sets requests AND limits to the same new value, e.g. 2/2Gi.

# Burstable: keep a gap; do NOT let requests == limits for both resources.
resources:
  requests: { cpu: 200m, memory: 256Mi }
  limits:   { cpu: "1",  memory: 1Gi }
# -> legal to raise requests to 500m or limits to 2Gi, as long as it stays Burstable.

If a patch would flip the class (e.g. a Burstable Pod where you set requests == limits for both cpu and memory), the API server rejects it. Design your resize automation to preserve the shape, not just the numbers.

9.3 VPA `updateMode` — the five values

The VPA object drives resizes automatically. The updateMode enum, as defined in the VPA CRD, is ["Off","Initial","Recreate","InPlaceOrRecreate","Auto"]:

Off                -> compute recommendations only; apply nothing (observation mode).
Initial            -> set requests only at Pod creation; never touch a running Pod.
Recreate (default) -> set at creation AND update running Pods by EVICTING them.
Auto (deprecated)  -> currently == Recreate; use explicit modes instead.
InPlaceOrRecreate  -> set at creation AND update running Pods IN PLACE via the resize
                      subresource; fall back to eviction/recreate only if in-place fails.

The committed iac/workloads/vpa-inplace.yaml uses InPlaceOrRecreate:

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: vpa-demo
  namespace: default
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: vpa-demo
  updatePolicy:
    updateMode: InPlaceOrRecreate     # try in-place first; recreate only if impossible
    minReplicas: 1                    # keep at least one Pod available during actuation
  resourcePolicy:
    containerPolicies:
      - containerName: burn
        controlledResources: ["cpu", "memory"]
        minAllowed:
          cpu: 50m
          memory: 32Mi
        maxAllowed:                   # uncapped target 1168m is capped here to 400m
          cpu: 400m
          memory: 256Mi

Why InPlaceOrRecreate and not InPlace. A pure InPlace mode (never evict) exists but is alpha in VPA v1.7.0 (AEP-8818) — beyond our v1.6.0/1.35 baseline. InPlaceOrRecreate is the pragmatic choice: it prefers the non-disruptive path but still makes progress when a resize is infeasible, by falling back to recreate.

controlledValues (limits handling). Default RequestsAndLimits scales limits proportionally, preserving the original limit:request ratio (this is why, in the verified run, capping CPU request to 400m produced a CPU limit of 4 — the original 10× ratio). Use RequestsOnly to leave limits untouched.

9.4 VPA fallback and disruption behavior

InPlaceOrRecreate falls back to eviction/recreate when:

the in-place update is infeasible (node lacks resources),
the update is deferred > 5 minutes,
the update is in progress > 1 hour,
the update would change the Pod's QoS class,
a memory-limit downscale is required for a container whose memory resize policy does not permit an in-place decrease.

By default VPA respects PodDisruptionBudgets and its own disruption thresholds (minReplicas, max-eviction ratio) even for in-place updates. The updater flag --in-place-skip-disruption-budget (default false) lets VPA skip PDB checks only when the update is genuinely non-disruptive — i.e. every container uses NotRequired for both cpu and memory. Even then, PDB is still enforced if any container uses RestartContainer or if the update falls back to eviction.

9.5 PriorityClass and resize ordering

On a contended node, deferred resizes are retried in the order PriorityClass → QoS → deferred duration (Section 6.8). If a workload's resizes must win under contention, give it a higher PriorityClass:

apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
  name: latency-critical
value: 1000000
globalDefault: false
description: "Workloads whose in-place resizes should be retried first under contention."

Then reference it in the Pod template (spec.priorityClassName: latency-critical). This does not guarantee immediate actuation — it only orders the retry queue — but it is the lever you have for prioritizing whose upsize applies first when a node is tight.

9.6 HPA boundary (configuration you must not create)

Do not run HPA and VPA on the same resource metric. HPA on CPU scales replica count based on utilization relative to requests; VPA changes the requests themselves; together on the same metric they oscillate. Supported combinations:

OK:  HPA on custom/external metrics (RPS, queue depth) + VPA on cpu/memory
OK:  HPA on CPU only            + VPA on memory only (controlledResources: ["memory"])
BAD: HPA on CPU                 + VPA on CPU  (feedback loop)

10. Hands-On Tutorial

This is the reproducible core of the article. Every step maps to a task target and a committed file. Command output is labeled Example output (illustrative of a Kubernetes v1.35 kind cluster) or, for the Prometheus/Grafana/OpenTelemetry steps, Expected output that you reproduce with the given commands.

Environment for the example output: a kind cluster (v0.32.0) on node image kindest/node:v1.35.0 — Server Version v1.35.0, containerd 2.2.0, cgroup v2. Your own node names, pod ages, and timings will differ.

10.0 Prerequisites

cd article-1-inplace-resize/iac
task up          # kind cluster on Kubernetes v1.35.0
task install     # metrics-server + VPA + kube-prometheus-stack + OTel Collector

10.1 Deploy the demo workload and confirm the baseline

task demo
# applies workloads/demo-app.yaml (resizePolicy cpu=NotRequired, memory=RestartContainer)

The baseline invariant is DESIRED == ACTUAL with restartCount 0. Read both:

kubectl get pod resize-demo -o jsonpath='{.spec.containers[0].resources}'          # DESIRED
kubectl get pod resize-demo -o jsonpath='{.status.containerStatuses[0].resources}' # ACTUAL
kubectl get pod resize-demo -o jsonpath='{.status.containerStatuses[0].restartCount}'

Example output (kind, Kubernetes v1.35):

# .spec.containers[0].resources (DESIRED)
{"limits":{"cpu":"200m","memory":"256Mi"},"requests":{"cpu":"100m","memory":"128Mi"}}
# .status.containerStatuses[0].resources (ACTUAL)
{"limits":{"cpu":"200m","memory":"256Mi"},"requests":{"cpu":"100m","memory":"128Mi"}}
# restartCount
0

Desired equals actual; the resize machinery is idle. This is the reference state we return to conceptually after each step.

10.2 CPU resize — `NotRequired`, no restart

Bump CPU only: requests.cpu 100m→500m, limits.cpu 200m→1. Because cpu: NotRequired, the kubelet applies new CPU cgroup values with no container restart. The committed patch is workloads/resize-cpu.patch.yaml:

spec:
  containers:
    - name: app
      resources:
        requests:
          cpu: 500m
        limits:
          cpu: "1"

task resize-cpu
# kubectl -n default patch pod resize-demo --subresource resize \
#   --patch-file workloads/resize-cpu.patch.yaml

Example output (kind, Kubernetes v1.35):

$ kubectl patch pod resize-demo --subresource resize \
    --patch '{"spec":{"containers":[{"name":"app","resources":{"requests":{"cpu":"500m"},"limits":{"cpu":"1"}}}]}}'
pod/resize-demo patched

# DESIRED
{"limits":{"cpu":"1","memory":"256Mi"},"requests":{"cpu":"500m","memory":"128Mi"}}
# ACTUAL (converged)
{"limits":{"cpu":"1","memory":"256Mi"},"requests":{"cpu":"500m","memory":"128Mi"}}
# restartCount (unchanged)
0

Desired and actual converged and restartCount stayed 0 — the process never stopped. This is the headline capability: raising CPU on a live container with zero disruption.

10.3 Memory resize — `RestartContainer`, restarts the container in place

Bump memory only: requests.memory 128Mi→256Mi, limits.memory 256Mi→512Mi. Because memory: RestartContainer, the kubelet restarts the container in place — the Pod is not recreated, but restartCount increments. The committed patch is workloads/resize-mem-restart.patch.yaml:

spec:
  containers:
    - name: app
      resources:
        requests:
          memory: 256Mi
        limits:
          memory: 512Mi

task resize-mem

Example output (kind, Kubernetes v1.35):

$ kubectl patch pod resize-demo --subresource resize \
    --patch '{"spec":{"containers":[{"name":"app","resources":{"requests":{"memory":"256Mi"},"limits":{"memory":"512Mi"}}}]}}'
pod/resize-demo patched

# ACTUAL (converged)
{"limits":{"cpu":"1","memory":"512Mi"},"requests":{"cpu":"500m","memory":"256Mi"}}
# restartCount
1

Note that CPU stayed at the values from Step 10.2 (500m/1) — resizes accumulate on the running Pod. restartCount went to 1: the container restarted in place to adopt the new memory ceiling. The Pod name, UID, node, and IP are unchanged.

10.4 The events tell the whole story

kubectl get events --field-selector involvedObject.name=resize-demo --sort-by='.lastTimestamp'
# or: kubectl describe pod resize-demo

Example output (kind, Kubernetes v1.35):

LAST SEEN   TYPE     REASON            OBJECT            MESSAGE
30s         Normal   Scheduled         pod/resize-demo   Successfully assigned default/resize-demo to inplace-resize-verify-control-plane
16s         Normal   ResizeStarted     pod/resize-demo   Pod resize started: {...generation:2}
16s         Normal   ResizeCompleted   pod/resize-demo   Pod resize completed: {...generation:2}
5s          Normal   Pulled            pod/resize-demo   Container image "registry.k8s.io/pause:3.10" already present on machine ...
5s          Normal   Created           pod/resize-demo   Container created
5s          Normal   Started           pod/resize-demo   Container started
5s          Normal   ResizeStarted     pod/resize-demo   Pod resize started: {...generation:3}
5s          Normal   Killing           pod/resize-demo   Container app resize requires restart
4s          Normal   ResizeCompleted   pod/resize-demo   Pod resize completed: {...generation:3}

Read this as two resizes:

Generation 2 (the CPU resize): ResizeStarted → ResizeCompleted, no Killing event, no restart.
Generation 3 (the memory resize): ResizeStarted → Killing ("Container app resize requires restart") → Created → Started → ResizeCompleted. The Killing event is the observable signal of an in-place container restart.

Event reasons. The reasons above (ResizeStarted, ResizeCompleted, Killing) are what kubelet v1.35.0 emits. A static source search found the constants ResizeDeferred and ResizeInfeasible (emitted on the deferred/infeasible paths) but did not find ResizeStarted/ ResizeCompleted as constants. Live evidence governs what you will observe; treat ResizeDeferred/ResizeInfeasible as the reasons for the pending paths. Killing is a generic container-lifecycle event, not a resize-specific reason — but it is the reliable signal that a RestartContainer resize is restarting the container.

10.5 Verify convergence with the helper script

task verify        # runs scripts/verify-resize.sh

scripts/verify-resize.sh reads DESIRED and ACTUAL via jsonpath, asserts they are equal, prints restartCount, and lists any PodResizePending/PodResizeInProgress conditions. It exits non-zero if not converged — suitable for CI gating. Core logic:

DESIRED="$(kubectl -n "$NAMESPACE" get pod "$POD" -o jsonpath='{.spec.containers[0].resources}')"
ACTUAL="$(kubectl -n "$NAMESPACE" get pod "$POD" -o jsonpath='{.status.containerStatuses[0].resources}')"
# ... prints restartCount and resize conditions ...
if [ "$DESIRED" = "$ACTUAL" ]; then
  echo "==> RESULT: CONVERGED (DESIRED == ACTUAL). Resize complete."; exit 0
else
  echo "==> RESULT: NOT CONVERGED ..." >&2; exit 1
fi

10.6 Infeasible resize — `PodResizePending` / reason `Infeasible`

Request an impossible amount of CPU (1000 cores on a 12-core node). The kubelet cannot actuate it, so it does not change ACTUAL and instead sets a status condition. The committed patch is workloads/resize-infeasible.patch.yaml:

spec:
  containers:
    - name: app
      resources:
        requests:
          cpu: "1000"
        limits:
          cpu: "1000"

task resize-infeasible
kubectl get pod resize-demo -o jsonpath='{.status.conditions}' | jq .

Example output (kind, Kubernetes v1.35):

$ kubectl patch pod resize-demo --subresource resize \
    --patch '{"spec":{"containers":[{"name":"app","resources":{"requests":{"cpu":"1000"},"limits":{"cpu":"1000"}}}]}}'
pod/resize-demo patched

# .status.conditions[] entry:
{
  "type": "PodResizePending",
  "status": "True",
  "reason": "Infeasible",
  "message": "Node didn't have enough capacity: cpu, requested: 1000000, capacity: 12000",
  "observedGeneration": 4
}
# ACTUAL resources UNCHANGED (resize not actuated):
{"limits":{"cpu":"1","memory":"512Mi"},"requests":{"cpu":"500m","memory":"256Mi"}}

The condition type is PodResizePending with reason: Infeasible. Infeasible is never re-evaluated — you must patch a feasible value. (Deferred would appear instead if the node could fit it later.) ACTUAL is untouched, so the workload keeps running at its previous, converged size. Revert by re-applying resize-cpu.patch.yaml.

10.7 Kubelet resize metrics

Scrape the kubelet's /metrics through the API server node proxy:

task observe       # runs scripts/observe.sh: events + kubelet metrics
# equivalently:
NODE=$(kubectl get nodes -o jsonpath='{.items[0].metadata.name}')
kubectl get --raw "/api/v1/nodes/${NODE}/proxy/metrics" \
  | grep -E 'kubelet_(container_requested_resizes|pod_in_progress_resizes|pod_infeasible_resizes|pod_resize_duration)'

Example output (kind, Kubernetes v1.35):

# HELP kubelet_container_requested_resizes_total [ALPHA] Number of requested resizes, counted at the container level.
# TYPE kubelet_container_requested_resizes_total counter
kubelet_container_requested_resizes_total{operation="decrease",requirement="limits",resource="cpu"} 1
kubelet_container_requested_resizes_total{operation="increase",requirement="limits",resource="cpu"} 2
kubelet_container_requested_resizes_total{operation="increase",requirement="limits",resource="memory"} 1
kubelet_container_requested_resizes_total{operation="increase",requirement="requests",resource="cpu"} 2
kubelet_container_requested_resizes_total{operation="increase",requirement="requests",resource="memory"} 1

# HELP kubelet_pod_in_progress_resizes [ALPHA] Number of in-progress resizes for pods.
# TYPE kubelet_pod_in_progress_resizes gauge
kubelet_pod_in_progress_resizes 1

# HELP kubelet_pod_infeasible_resizes_total [ALPHA] Number of infeasible resizes for pods.
# TYPE kubelet_pod_infeasible_resizes_total counter
kubelet_pod_infeasible_resizes_total{reason_detail="insufficient_node_allocatable"} 1

# HELP kubelet_pod_resize_duration_seconds [ALPHA] Duration in seconds to actuate a pod resize
# TYPE kubelet_pod_resize_duration_seconds histogram
kubelet_pod_resize_duration_seconds_bucket{success="true",le="..."} ...
kubelet_pod_resize_duration_seconds_sum{success="true"} 0.061
kubelet_pod_resize_duration_seconds_count{success="true"} 3

The six resize metrics the kubelet records (all [ALPHA] stability, per KEP-1287) and what they tell you:

Metric	Type	Key labels	Reads as
`kubelet_container_requested_resizes_total`	counter	`operation`, `requirement`, `resource`	How many resizes, by kind. Here: 2 CPU-request increases, 1 CPU-limit decrease (from the infeasible→revert), etc.
`kubelet_pod_in_progress_resizes`	gauge	(none)	How many resizes are actuating right now (allocated, not yet actuated).
`kubelet_pod_pending_resizes`	gauge	`reason`	How many resizes are pending, split by `deferred` vs `infeasible`.
`kubelet_pod_infeasible_resizes_total`	counter	`reason_detail`	Infeasible count; `insufficient_node_allocatable` = didn't fit.
`kubelet_pod_deferred_resize_accepted_total`	counter	`retry_trigger`	Deferred resizes later accepted; a high `periodic_retry` share hints at a kubelet retry-logic issue.
`kubelet_pod_resize_duration_seconds`	histogram	`success`	Actuation latency; `_sum 0.061` over `_count 3` ≈ 0.02s (~20ms)/resize here.

10.8 metrics-server: `kubectl top`

kubectl top nodes
kubectl top pod resize-demo

Example output (kind, Kubernetes v1.35):

$ kubectl top nodes
NAME                                  CPU(cores)   CPU(%)   MEMORY(bytes)   MEMORY(%)
inplace-resize-verify-control-plane   146m         1%       613Mi           7%
$ kubectl top pod resize-demo
NAME          CPU(cores)   MEMORY(bytes)
resize-demo   0m           0Mi

resize-demo uses 0m/0Mi because pause does nothing — which is exactly why it is a clean vehicle for demonstrating resize mechanics independent of load. VPA needs a workload that actually consumes CPU to produce a recommendation, which is the next step.

10.9 Automate with VPA (`InPlaceOrRecreate`) — end to end

Deploy the CPU-burning vpa-demo workload and its VPA object (Section 9.3):

task vpa-demo      # applies workloads/vpa-inplace.yaml

The burn container runs while true; do :; done, driving CPU utilization up. VPA's recommender computes a target, the updater actuates it in place via the resize subresource, and the Pod is not recreated.

Example output (kind, Kubernetes v1.35):

# VPA recommendation (.status.recommendation.containerRecommendations)
[{"containerName":"burn","lowerBound":{"cpu":"400m","memory":"100Mi"},
  "target":{"cpu":"400m","memory":"100Mi"},
  "uncappedTarget":{"cpu":"1168m","memory":"100Mi"},
  "upperBound":{"cpu":"400m","memory":"256Mi"}}]

# Pod NOT recreated (same pod, original age; RESTARTS=1 from memory RestartContainer policy):
NAME                        READY   STATUS    RESTARTS        AGE
vpa-demo-6596db698c-sxtz6   1/1     Running   1 (4m53s ago)   6m50s

# DESIRED == ACTUAL after VPA acted (requests cpu 50m->400m, mem 32Mi->100Mi; limits scaled proportionally):
DESIRED: {"limits":{"cpu":"4","memory":"200Mi"},"requests":{"cpu":"400m","memory":"100Mi"}}
ACTUAL:  {"limits":{"cpu":"4","memory":"200Mi"},"requests":{"cpu":"400m","memory":"100Mi"}}

# Events (VERIFIED) — note the VPA-specific event reason InPlaceResizedByVPA:
Normal  ResizeStarted        Pod resize started: {...generation:2}
Normal  Killing              Container burn resize requires restart
Normal  InPlaceResizedByVPA  Pod was resized in place by VPA Updater.
Normal  ResizeCompleted      Pod resize completed: {...generation:2}

What this proves:

The Pod was not recreated. Same Pod name/UID and original age (6m50s). The single restart (RESTARTS=1) is a container restart from the memory RestartContainer policy, not a Pod recreation.
VPA respected maxAllowed. The uncapped target was 1168m; VPA capped the request to 400m (the maxAllowed).
VPA preserved the limit:request ratio. Original CPU ratio 500m/50m = 10× → limit 4; memory 64Mi/32Mi = 2× → limit 200Mi. This is controlledValues: RequestsAndLimits behavior.
The VPA-specific event reason InPlaceResizedByVPA ("Pod was resized in place by VPA Updater.") is your audit trail that VPA — not a human — actuated the resize.

10.10 Prometheus + Grafana (expected output — not executed here)

The kube-prometheus-stack values (Section 8.6) scrape the kubelet's resize metrics already. Load the dashboard and open the UIs:

task dashboards    # creates+labels the grafana-inplace-resize ConfigMap (grafana_dashboard=1)
PORT_FORWARD=1 task observe
#   Grafana:    http://localhost:3000   (admin / prom-operator — demo credential)
#   Prometheus: http://localhost:9090

Expected output — not executed in this environment. In Prometheus → Status → Targets you should see the serviceMonitor/monitoring/kube-prometheus-stack-kubelet/ 0..2 targets (/metrics, /metrics/cadvisor, /metrics/resource) UP. After triggering a resize, these queries return data:

# Resize request rate by operation
sum by (operation) (rate(kubelet_container_requested_resizes_total[5m]))

# In-progress resizes (gauge)
sum(kubelet_pod_in_progress_resizes)

# Infeasible resizes by reason
sum by (reason_detail) (rate(kubelet_pod_infeasible_resizes_total[5m]))

# Resize duration p95 (metric is ms; divide by 1000 for seconds)
histogram_quantile(0.95, sum by (le) (rate(kubelet_pod_resize_duration_seconds_bucket[5m]))) / 1000

# CPU request (desired, from kube-state-metrics) vs usage (actual, from cAdvisor)
sum by (pod, container) (kube_pod_container_resource_requests{resource="cpu", namespace="default"})
sum by (pod, container) (rate(container_cpu_usage_seconds_total{namespace="default", container!="", container!="POD"}[5m]))

The committed iac/dashboards/grafana-inplace-resize.json wires five panels (resize rate by operation, in-progress gauge, infeasible by reason, duration p50/p95, requests-vs-usage) against a templated ${DS_PROMETHEUS} datasource. Import it via the sidecar (above) or Grafana → Dashboards → New → Import. This article does not include fabricated Grafana screenshots — run the stack to see live panels.

10.11 OpenTelemetry Collector (expected output — not executed here)

The Collector (Section 8.7) scrapes the same kubelet resize metrics via its prometheus receiver, adds usage/utilization via kubeletstats and absolute requests/limits via k8s_cluster, and remote-writes to Prometheus.

kubectl -n observability logs ds/otel-collector | head -50

Expected output — not executed in this environment. The debug exporter should log the resize metric families (kubelet_pod_resize_duration_seconds, kubelet_container_requested_resizes_total, …) and the kubeletstats/k8s_cluster series; after remote-write, the same series appear in Prometheus.

Honest scope note on tracing. The kubelet does not emit distributed traces for resizes. Kubernetes has API-server and kubelet (KEP-2831) tracing, but there is no dedicated "resize span." Resize observability is metrics + Pod events. If you want a trace, instrument your application to open a span around a self-triggered resize (PATCH the resize subresource, poll until ACTUAL == DESIRED), and export it via OTLP. That span is synthetic/demonstrative — the platform does not produce it.

11. Production Best Practices

Set resizePolicy deliberately per runtime. For runtimes that cannot change their memory ceiling live (JVM, CPython, most managed runtimes), set memory: RestartContainer so a memory resize is a clean restart rather than a runtime error-and-retry or an OOM risk. For CPU, keep NotRequired — CPU almost always applies live. Treat this as an architecture decision, not a default.
Reconcile the controller template, not just the running Pod. A resize on the Pod is reverted the next time the Pod is recreated (rollout, drain, eviction). For a durable change, update the Deployment/StatefulSet template as well, or let VPA own the requests so recreated Pods get the current recommendation from the admission controller.
Decrease memory conservatively. The GA OOM-avoidance check is best-effort and subject to a TOCTOU race; on cgroup v2 an over-tight limit OOM-kills. Shrink in steps, watch working-set headroom, and prefer RestartContainer for memory decreases on runtimes that cannot shrink live.
Alert on conditions and metrics, not on the deprecated string field. Watch for PodResizePending{reason=Infeasible} (needs human action), long-lived PodResizePending{reason=Deferred} (capacity pressure), and rising kubelet_pod_infeasible_resizes_total. Do not read .status.resize — it is gone.
Budget for container restarts on memory resizes. A RestartContainer memory resize is a restart: it clears in-memory state, drops connections, and re-runs startup probes. Roll it out like a controlled restart (respect PDBs, do one replica at a time), not as a free knob.
Keep resizes QoS-legal by construction. Encode the QoS shape in your automation. For Guaranteed workloads, always move requests and limits together; for Burstable, never collapse the gap on both resources at once.
Use PriorityClass to order resizes under contention. On busy nodes, deferred resizes compete; give latency-sensitive workloads a higher PriorityClass so their upsizes are retried first (Section 6.8).
Pair VPA with a PodDisruptionBudget and minReplicas. InPlaceOrRecreate falls back to eviction for infeasible/timed-out resizes; a PDB and minReplicas protect availability on that path. Only enable --in-place-skip-disruption-budget for workloads that are provably non-disruptive (all NotRequired).
Never run HPA and VPA on the same metric. Split by metric (HPA on custom/ external, VPA on cpu/memory) to avoid the feedback loop (Section 9.6).
Pin everything and provision real kubelet serving certs. Pin chart and image versions (versions.env); do not ship --kubelet-insecure-tls to production — provision cluster-CA-signed kubelet serving certs instead.
Grant pods/resize narrowly. Give controllers/humans patch on pods/resize rather than write on pods. This is a clean least-privilege seam (Section 15).
Measure actuation latency and set an SLO. Track kubelet_pod_resize_duration_seconds p95 and runtime_operations_duration_seconds{operation_type=container_update}; a rising p95 or growing kubelet_pod_in_progress_resizes signals runtime/node pressure.
Leave headroom on nodes you resize into. In-place upsizes consume node capacity immediately (the scheduler counts max(desired,allocated,actual)). Packing nodes to 100% guarantees Deferred/Infeasible resizes.
Test the version-skew and rollout order. Upgrade the control plane (apiserver, scheduler) before kubelets. On mixed-version nodes, verify resizes on a canary node pool before relying on them fleet-wide.

12. Common Mistakes

At least fifteen failure patterns, each with the correction.

Patching resources on the main Pod resource. kubectl patch pod ... without --subresource resize is rejected — resources are immutable on the main resource. Always target --subresource resize.
Using an old kubectl client. Clients < v1.32.0 report invalid subresource for --subresource=resize. Upgrade the client (server must be ≥ v1.33; we run v1.35).
Expecting a Deployment edit to resize a running Pod. Editing the template changes future Pods; the running Pod is only mutated through the resize subresource on the Pod.
Forgetting the template revert. Resizing only the Pod means the next rollout/ drain snaps back to the template values. Update the template too, or use VPA.
Trying to change QoS via resize. Setting requests == limits for both cpu and memory on a Burstable Pod (or removing all resources from a Guaranteed Pod) is rejected — QoS is invariant across a resize.
Assuming a memory decrease is safe once accepted. The OOM-avoidance check is best-effort; on cgroup v2 an over-tight limit OOM-kills. Decrease conservatively.
Treating a RestartContainer memory resize as non-disruptive. It restarts the container (state loss, dropped connections, re-run probes). Plan it like a restart.
Reading the deprecated .status.resize string field. It did not graduate; read the PodResizePending/PodResizeInProgress conditions instead.
Confusing Deferred with Infeasible. Deferred is retried automatically; Infeasible is never re-evaluated and needs you to change the request or add capacity. Alerting on the wrong one wastes on-call time.
Packing nodes to capacity, then wondering why upsizes fail. In-place upsizes need free node capacity now; a full node yields Deferred/Infeasible.
Running HPA and VPA on the same metric. Causes an oscillation loop. Split by metric.
Only resizing CPU/memory requests but ignoring limits (or vice versa). For Guaranteed Pods this breaks the requests == limits invariant and is rejected; for Burstable it can silently under- or over-provision the ceiling.
Trying to resize non-CPU/memory resources. Only cpu and memory are mutable in v1.35; anything else (ephemeral-storage, extended resources) is rejected.
Trying to resize the wrong container type. Non-restartable init containers and ephemeral containers cannot be resized; sidecars (restartable init containers) can.
Shipping --kubelet-insecure-tls to production. A kind-only shortcut. Provision cluster-CA-signed kubelet serving certs instead; skipping verification exposes the metrics path to MITM.
Enabling in-place resize on unsupported node configs. Swap-enabled containers, static CPU Manager, and static Memory Manager are not supported and surface as Infeasible. Don't design a resize workflow on a node pool that can't actuate it.
Passing --feature-gates=InPlaceOrRecreate=true to VPA ≥ v1.7.0. The gate is removed there; the flag crashes the component. Pin the VPA app version.
Assuming a patch return of success means the resize converged. The API returns success on accepting the desired change; convergence is ACTUAL == DESIRED. Verify status/conditions before declaring done.

13. Troubleshooting Guide

Symptom → likely cause → diagnosis and fix. Commands use P for the Pod name and N for the node name. The Signals to inspect list after the table maps each symptom to the log line or metric that confirms it.

Symptom	Likely cause	Diagnosis → fix	Command
`error: invalid subresource "resize"`	`kubectl` client < v1.32.0	Old client → upgrade to ≥ v1.32	`kubectl version`
Patch rejected: resources immutable	Patched main resource, not the subresource	Command lacks `--subresource resize` → add it	`kubectl patch pod P --subresource resize --patch-file f.yaml`
Patch rejected: QoS would change	Resize flips the QoS class	Compare requests/limits shape → keep QoS shape (Guaranteed: `req==lim`; Burstable: keep a gap)	`kubectl get pod P -o jsonpath='{.status.qosClass}'`
`PodResizePending` / `Infeasible`	Request exceeds node capacity (or static Pod / swap / static managers / Windows)	Read the condition message → patch a feasible value, add capacity, or move off the unsupported config	`kubectl get pod P -o jsonpath='{.status.conditions}' \| jq .`
`PodResizePending` / `Deferred` (long-lived)	Node lacks room now; queued behind higher-priority resizes	Check node allocatable and priority ordering → free capacity, raise PriorityClass, or scale the pool	`kubectl describe node N`
`PodResizeInProgress` / `Error`	Actuation error — often the memory-decrease OOM check (usage > new limit)	Read the condition message → raise the target limit above current usage, or use `RestartContainer`	`kubectl get pod P -o jsonpath='{.status.conditions}'`
ACTUAL never equals DESIRED	Deferred / in-progress / infeasible, or the runtime rounded the value	Compare actuated vs actual → address the condition; small cgroup rounding is expected	`bash scripts/verify-resize.sh P`
`restartCount` jumped unexpectedly	Memory (or CPU+memory) resize with `RestartContainer`	Events show `Killing … resize requires restart` → expected; use `NotRequired` only if the runtime supports live change	`kubectl describe pod P`
Pod OOM-killed after a memory decrease	Best-effort OOM check lost the TOCTOU race	`lastState.terminated.reason=OOMKilled` → decrease conservatively; use `RestartContainer` for memory	`kubectl get pod P -o jsonpath='{.status.containerStatuses[0].lastState}'`
`kubectl top` returns no data	metrics-server can't verify the kubelet cert (kind)	x509 SAN error in logs → `--kubelet-insecure-tls` (dev) or signed kubelet certs (prod)	`kubectl -n kube-system logs deploy/metrics-server`
VPA recommends but never actuates	`updateMode: Off`/`Initial`, or in-place gate unset	Check the VPA and updater args → set `InPlaceOrRecreate`; ensure the updater has the gate	`kubectl -n vpa get deploy vpa-updater -o yaml \| grep feature-gates`
VPA evicts instead of resizing in place	In-place infeasible/timed-out → fallback, or a QoS change is required	Check VPA and Pod events → ensure capacity, keep QoS stable, check resize policy	`kubectl get events \| grep -E 'Evict\|InPlaceResizedByVPA'`
Prometheus shows no resize metrics	kubelet ServiceMonitor down, or no resize has happened yet	Check Prometheus → Targets; trigger a resize	`kubectl -n monitoring get servicemonitor`
Workload ServiceMonitor ignored	`serviceMonitorSelectorNilUsesHelmValues: true`	Not listed in Prometheus targets → set the selector flags to `false` (Section 8.6)	`kubectl -n monitoring get prometheus -o yaml`
OTel `k8s_cluster` series double-counted	Multiple DaemonSet pods each run `k8s_cluster`	Duplicate series → run `k8s_cluster` in a single deployment-mode collector	`kubectl -n observability get pods -o wide`
Resize works on one node, not another	Version skew or an unsupported node config	Compare node versions/configs → upgrade kubelets or move to a supported pool	`kubectl get nodes -o wide`

Signals to inspect (the log line or metric that confirms each case):

Infeasible resize → kubelet_pod_infeasible_resizes_total{reason_detail=...}
Deferred resize → kubelet_pod_pending_resizes{reason="deferred"}
In-progress / stuck actuation → kubelet_pod_in_progress_resizes; kubelet logs (resize actuation error)
Memory-decrease OOM → kernel/kubelet OOM logs; lastState.terminated.reason=OOMKilled
RestartContainer restart → Pod event reason Killing
metrics-server / kubectl top → x509 "IP SANs" error in the metrics-server log
VPA actuation → vpa-updater logs; vpa_updater_failed_in_place_update_attempts_total
Prometheus scrape health → count(kubelet_pod_resize_duration_seconds_count) > 0; Prometheus config-reload logs

General first moves for any resize problem:

# 1. What does the Pod think its desired vs actual is?
kubectl get pod P -o jsonpath='{.spec.containers[0].resources}'; echo
kubectl get pod P -o jsonpath='{.status.containerStatuses[0].resources}'; echo
# 2. Any resize conditions?
kubectl get pod P -o jsonpath='{.status.conditions}' | jq '.[] | select(.type|test("PodResize"))'
# 3. What events fired?
kubectl describe pod P | sed -n '/Events:/,$p'
# 4. What do the kubelet metrics say?
NODE=$(kubectl get nodes -o jsonpath='{.items[0].metadata.name}')
kubectl get --raw "/api/v1/nodes/${NODE}/proxy/metrics" | grep kubelet_pod_

14. Real Production Use Cases

14.1 Game servers and stateful session holders

A match-based game server holds live player sessions in memory. Recreating it to add CPU headroom for a busy match would disconnect everyone. With in-place resize, an operator (or a controller watching match load) raises CPU with NotRequired — no restart, no dropped sessions — and lowers it again when the match ends. Memory is kept on RestartContainer and only resized between matches. The trade-off: CPU scaling is free, memory scaling is scheduled around session boundaries.

14.2 JIT / startup boost

Many runtimes need far more CPU during startup (JIT compilation, cache warming) than in steady state. Historically you either over-provisioned steady-state CPU or suffered slow cold starts. The pattern: start the container with generous CPU, then scale CPU down in place once the Pod is Ready. VPA's CPU Startup Boost (AEP-7862, VPA v1.7.0 alpha — forward-looking) automates exactly this: it raises CPU requests+limits at startup and scales them back down in place after readiness. Until that ships, you can approximate it with a controller or an init-driven resize. This is the clearest example of resize turning a static sizing compromise into a dynamic one.

14.3 Pre-warmed worker pools

A pool of workers kept warm to avoid cold-start latency can idle at a small reservation and be scaled up in place the moment work is dispatched to them — faster than scheduling a new Pod, and without losing the warm cache/connections the worker already holds. When the burst subsides, scale the reservation back down. The win is latency (no scheduling round-trip) plus preserved warm state; the constraint is that the worker must fit on its current node when scaled up (or the resize defers).

14.4 Bin-packing and cost efficiency

Fleets are routinely over-provisioned "to be safe," stranding capacity. VPA in InPlaceOrRecreate mode continuously right-sizes requests toward observed usage without the eviction churn that made teams run VPA in Off mode. Tighter, accurate requests let the scheduler pack more Pods per node, directly reducing node count and cost. The trade-off is operational: you now depend on VPA's recommendations and must watch for memory-decrease OOM risk and deferred resizes on tightly packed nodes.

14.5 Batch and phase-based workloads

A pipeline stage that needs 4 cores during a compute phase and 200m while waiting on I/O can grow and shrink its CPU reservation per phase instead of holding the peak for its whole lifetime. Because CPU is NotRequired, these transitions are restart-free. This reclaims the difference between peak and average for every other workload sharing the node.

Across all of these, the common thread is the same: decouple the reservation from the process lifecycle. In-place resize is most valuable exactly where recreation is most expensive — long warm-ups, live state, and latency-sensitive paths.

15. Security Considerations

15.1 RBAC: the `pods/resize` subresource

The single most important security fact: resizing is authorized by the pods/resize subresource, independently of write access to pods. Regular patch/update on pods does not allow resource mutation. To let a subject resize, grant patch/update on pods/resize:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: pod-resizer
  namespace: default
rules:
  - apiGroups: [""]
    resources: ["pods/resize"]
    verbs: ["patch", "update"]
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["get", "list", "watch"]   # read desired/actual/status

This is a genuine least-privilege seam: you can authorize "may right-size workloads" without authorizing "may rewrite Pod spec, mount secrets, change images." Grant it to the specific controller or human role that needs it, scoped to a namespace where possible. (This follows from the KEP subresource definition and standard Kubernetes subresource RBAC semantics.)

15.2 The kubelet/Node identity is the only writer of actual state

Only the Node account may write status.allocatedResources/status.resources. Users and controllers write desired (spec) through the subresource; the kubelet writes actual (status). This separation means a compromised workload identity cannot forge "actual" resources to hide from quota or fool the scheduler.

15.3 Resource-exhaustion and abuse surface

Because resize can raise a Pod's reservation, an over-broad pods/resize grant is a capacity-abuse vector: a subject could upsize workloads to starve others or drive node pressure. Mitigations:

Scope the grant narrowly (namespace, specific role).
Keep ResourceQuota in force — quota accounts for resizes via max(desired,allocated,actual), so a resize that would breach quota is rejected.
Set VPA maxAllowed ceilings so automated resizes cannot run away.
Alert on kubelet_container_requested_resizes_total spikes and on unexpected upsizes.

15.4 Transport security for the metrics path

The kind-only --kubelet-insecure-tls shortcut disables verification of the kubelet serving cert and must not reach production — it exposes the metrics/stats path to man-in-the-middle. Provision cluster-CA-signed kubelet serving certificates (enable RotateKubeletServerCertificate and the CSR approver) so metrics-server, Prometheus, and the OTel Collector can verify the kubelet.

15.5 Grafana and stack credentials

The demo sets grafana.adminPassword: "prom-operator" in plaintext values — a demo credential. In production, reference an existing Secret (admin.existingSecret) and rotate it; never commit dashboard-stack admin passwords to a values file in git.

15.6 Audit trail

Resizes are auditable at the API server (pods/resize requests appear in the audit log and in apiserver_request_total{resource=pods,subresource=resize}) and on the Pod (ResizeStarted/ResizeCompleted events, and InPlaceResizedByVPA when VPA acted). Keep these in your audit pipeline so every reservation change is attributable.

16. Performance Considerations

16.1 Actuation latency

A resize is not instantaneous — the kubelet must admit, order, call the runtime, and read back. In the example run, kubelet_pod_resize_duration_seconds_sum{success="true"} 0.061 over _count 3 ≈ 20ms per resize for a trivial pause container on an idle node. Real actuation latency depends on the runtime, node load, and whether a container restart is involved. Track:

histogram_quantile(0.95, sum by (le) (rate(kubelet_pod_resize_duration_seconds_bucket[5m]))) / 1000
runtime_operations_duration_seconds{operation_type="container_update"}

A RestartContainer resize adds the container's restart cost (image already present, so mostly process start + probes) on top of actuation.

16.2 The scheduler↔kubelet race (correctness under load)

The main known correctness gap in v1.35: the scheduler and kubelet can momentarily disagree about node capacity, and a race can over-schedule a node so a Pod is rejected with OutOfCPU/OutOfMemory (tracked in kubernetes/kubernetes#126891; resolution out of scope for GA). Practical mitigation: leave node headroom, avoid packing to 100% on node pools where you resize aggressively, and canary aggressive resize automation.

16.3 Memory-decrease TOCTOU

The best-effort OOM-avoidance check reads usage, then writes the limit; usage can climb in between. A memory decrease can therefore either leave a resize InProgress with reason: Error (skipped) or, if written below live usage on cgroup v2, OOM-kill. Decreasing memory has a real performance/availability cost that increasing does not — size decreases conservatively and off the hot path.

16.4 Resize storms

Automated resizers (VPA, custom controllers) can generate many resizes. Each consumes kubelet work and a CRI call, and each memory RestartContainer resize is a restart. Watch kubelet_pod_in_progress_resizes (a persistently non-zero gauge means the kubelet is backed up) and kubelet_container_requested_resizes_total rate. Rate-limit custom resizers and rely on VPA's built-in update conditions (bounds, >12h/>10% rule, quick-OOM) rather than resizing on every metric tick.

16.5 No effect on scheduling throughput

A resize does not invoke the scheduler (the Pod is already bound). So resizing is cheaper than recreate-based right-sizing, which pays a full deschedule/schedule/ image-pull/startup cost. This is the core performance argument for the feature: the common case (CPU up/down) is a cgroup write, not a reschedule.

17. Integration with the Ecosystem

17.1 Metrics Server

The Metrics API backend and a hard dependency of VPA's recommender. It serves kubectl top and feeds VPA the usage samples it turns into recommendations. Without it, VPA computes nothing. (Verified: chart 3.13.1, app v0.8.1.)

17.2 Vertical Pod Autoscaler

The primary automated consumer of in-place resize. In InPlaceOrRecreate mode the updater actuates recommendations via the resize subresource, falling back to eviction only when in-place is infeasible/timed-out/QoS-changing. VPA-side observability:

vpa_updater_in_place_updatable_pods_total
vpa_updater_in_place_updated_pods_total
vpa_updater_failed_in_place_update_attempts_total

Pair these with the kubelet resize metrics to see both "VPA decided to resize" and "the kubelet actuated it." (Verified: chart 4.12.3, app v1.6.0; event reason InPlaceResizedByVPA.)

17.3 Prometheus + kube-state-metrics + cAdvisor

Prometheus scrapes the kubelet's /metrics (resize metrics), /metrics/cadvisor (actual usage), and /metrics/resource. Desired vs actual on one board: kube-state-metrics gives desired (kube_pod_container_resource_requests/_limits — spec = desired in v1.35), cAdvisor gives actual usage (container_cpu_usage_seconds_total, container_memory_working_set_bytes). The utilization ratio (usage/request) is the signal a resize acts on:

sum by (namespace,pod,container)(rate(container_cpu_usage_seconds_total{container!="",container!="POD"}[5m]))
/
sum by (namespace,pod,container)(kube_pod_container_resource_requests{resource="cpu"})

Note: whether kube-state-metrics surfaces status (actual) resources depends on its version; classic KSM exposes spec-derived (desired) requests/limits only.

17.4 Grafana

The committed grafana-inplace-resize.json (5 panels) visualizes resize rate, in-progress, infeasible-by-reason, duration percentiles, and requests-vs-usage. Import via the sidecar (task dashboards).

17.5 OpenTelemetry Collector

Alternative/additional collection path: prometheus receiver for resize metrics, kubeletstats for usage/utilization ratios, k8s_cluster for absolute requests/ limits, remote-writing to Prometheus. Note the kubelet emits no resize traces — resize observability is metrics + events (Section 10.11).

17.6 HPA boundaries

HPA and VPA must not share a resource metric (Section 9.6). The supported topology is HPA on custom/external metrics (RPS, queue depth) for replica count + VPA on cpu/memory for per-Pod sizing — horizontal and vertical scaling on orthogonal signals.

17.7 Forward-looking: Cluster Autoscaler / Karpenter and Kueue

In-place resize works within a node's capacity. When an upsize does not fit, it defers/infeasibles — and that is the handoff point to the Cluster Autoscaler or Karpenter to add a node, and to Kueue for admission/quota-aware batch scheduling. The interplay between in-place resize signals (Deferred/Infeasible) and node provisioning is an active area and a natural bridge to the scheduling topics later in this series. (Cluster Autoscaler / Karpenter awareness of in-place resize pending signals is still evolving — check current project docs.)

18. Advanced Topics

18.1 The actuated state and restart tolerance

The kubelet checkpoints the actuated resources (what it sent the runtime) on the node (PR #130599). This is why a kubelet restart mid-resize does not lose track: on restart it compares checkpointed-actuated against runtime-reported-actual and re-issues only if they differ. It also explains the design decision to compare actuated rather than desired against actual — runtimes legitimately land on nearby values (kernel CPU minimums, systemd 10ms quota rounding, NRI plugin mutation), and comparing against desired would cause endless re-issue loops.

18.2 Pod-level resources resize (alpha in v1.35)

Beyond per-container resources, Kubernetes supports Pod-level resources (spec.resources). In-place resize of Pod-level resources is alpha in v1.35 behind its own gate (InPlacePodLevelResourcesVerticalScaling). Keep production pinned to per-container resize on v1.35; treat pod-level resize as forward-looking.

18.3 Exclusive-CPU relaxation

A separate gate, InPlacePodVerticalScalingExclusiveCPUs (seen in PR #128713), governs relaxing the static-CPU-manager restriction for in-place resize. In baseline v1.35, Guaranteed Pods under the static CPU Manager policy are Infeasible to resize; this gate is the path toward loosening that. (Confirm maturity and behavior for your specific Kubernetes build.)

18.4 CRI contract and `UpdatePodSandboxResources`

The kubelet actuates via UpdateContainerResources, which the runtime must implement idempotently; since v1.33 the contract is that runtimes should not deliberately restart a container to resize — they return an error instead, and the kubelet decides what to do. GA added a best-effort UpdatePodSandboxResources CRI call, invoked after the kubelet reconfigures pod-level cgroups, primarily to notify NRI plugins.

18.5 The `allocatedResources` status and its gate

The companion gate InPlacePodVerticalScalingAllocatedStatus controls whether status.containerStatuses[i].allocatedResources (the allocated state, Section 6.3) is exposed in the API. When on, it gives you a view of what the kubelet has admitted but not yet actuated — useful for distinguishing "admitted, actuating" from "not yet admitted."

18.6 Version skew and upgrade order

Upgrade the control plane (apiserver, scheduler) before kubelets. The apiserver detects kubelet feature support by checking whether status.containerStatuses[*].resources is populated on running containers, and handles skew accordingly. On mixed-version node pools, resizes may work on newer kubelets and not on older ones — canary before relying on the feature fleet-wide.

18.7 Self-instrumented resize tracing (demo pattern)

Since the platform emits no resize spans, you can build an app-side trace: open a span pod.self_resize, PATCH the resize subresource, poll status.containerStatuses[]. resources until ACTUAL == DESIRED (or a resize condition clears), attach attributes (k8s.pod.name, resize.resource, old, new, resize.outcome), and export via OTLP. This correlates an application action with the platform-side kubelet_pod_resize_duration_seconds. It is synthetic — label it as such.

18.8 Terminated-container fail-open

A resize is atomic across containers, but a terminated container's "resize" is permitted (fail-open) so it does not block resizing sibling running containers. This is a deliberate lifecycle nuance in KEP-1287 to avoid one dead container wedging the whole Pod's resize.

19. Interview Questions

Sixty questions across three tiers, each with an answer.

19.1 Beginner (20)

B1. What does In-Place Pod Vertical Scaling let you do? Change a running Pod's cpu/memory requests and limits and have the kubelet apply the change to the existing container(s), without deleting and recreating the Pod.

B2. In which Kubernetes version did it reach GA? v1.35 (alpha v1.27, beta v1.33, GA/stable v1.35).

B3. Do you need to enable a feature gate in v1.35? No. InPlacePodVerticalScaling is GA and on by default in v1.35.

B4. Which resources can be resized in place in v1.35? Only cpu and memory (requests and limits). Nothing else is mutable.

B5. What is the difference between spec.containers[*].resources and status.containerStatuses[*].resources? spec = DESIRED (what you asked for); status = ACTUAL (what the container currently has). A resize is converged when they are equal.

B6. How do you trigger a resize with kubectl? kubectl patch pod <name> --subresource resize --patch '...' — you must target the resize subresource.

B7. Can you resize by editing the Deployment's template? Editing the template changes future Pods; the running Pod is only mutated through the resize subresource on the Pod itself.

B8. What is resizePolicy? A per-container, per-resource setting that says whether applying a new value needs a container restart: NotRequired (no restart) or RestartContainer (restart in place).

B9. What is the default resizePolicy if you don't set it? NotRequired for each resource.

B10. Does a RestartContainer resize recreate the Pod? No. It restarts the container in place (incrementing restartCount); the Pod keeps its name, UID, node, and IP.

B11. Why does memory often use RestartContainer? Many runtimes (JVM, CPython) cannot change their memory ceiling live, so the container must restart to adopt the new limit.

B12. What happens to CPU on a NotRequired resize? The new CPU cgroup values are applied to the live container with no restart.

B13. What minimum kubectl client version supports --subresource=resize? Client v1.32.0 or newer.

B14. What is a QoS class? A Pod classification (Guaranteed/Burstable/BestEffort) derived from its requests/limits that drives eviction order.

B15. Can a resize change a Pod's QoS class? No. QoS is invariant across a resize; the API server rejects resizes that would change it.

B16. What tool right-sizes Pods automatically using this feature? The Vertical Pod Autoscaler (VPA), in InPlaceOrRecreate mode.

B17. Name one workload that benefits from in-place resize. Any long-warm-up or stateful workload — e.g. a JVM service or a game server holding live sessions — where recreation is expensive.

B18. Where do the resize metrics come from? The kubelet's /metrics endpoint (e.g. kubelet_pod_resize_duration_seconds).

B19. What does kubectl top need to work? metrics-server, which serves the Metrics API.

B20. What does "converged" mean for a resize? DESIRED equals ACTUAL — the container actually has the resources you asked for.

19.2 Intermediate (20)

I1. What replaced the deprecated .status.resize string field? Two Pod conditions: PodResizePending (reason Deferred or Infeasible) and PodResizeInProgress.

I2. Difference between Deferred and Infeasible? Deferred is feasible in principle but not right now — retried periodically. Infeasible cannot be satisfied as-is and is never re-evaluated.

I3. Name three infeasible reasons in v1.35. Request exceeds node capacity; static Pod; swap-enabled container; Guaranteed Pod with static CPU/Memory Manager; Windows Pod (any three).

I4. What are the four resource states the kubelet tracks? Desired → Allocated → Actuated → Actual.

I5. Why does the kubelet compare actuated vs actual, not desired vs actual? Because runtimes legitimately land on nearby values (kernel CPU minimums, systemd 10ms rounding, NRI plugins); comparing desired vs actual would trigger endless re-issues.

I6. What changed for memory-limit decreases at GA? They are now allowed (previously prohibited), guarded by a best-effort OOM-avoidance check that skips the resize if current usage exceeds the new limit.

I7. Why is the memory-decrease check only "best-effort"? A TOCTOU race: usage can rise between the check and the cgroup write; on cgroup v2 an over-tight limit OOM-kills.

I8. What RBAC do you need to let a controller resize Pods? patch/update on the pods/resize subresource (plus get/list/watch on pods). Write on pods alone does not permit resource mutation.

I9. In what order are deferred resizes retried? PriorityClass → QoS class (Guaranteed > Burstable) → deferred duration (longest-pending first).

I10. What are the five VPA updateMode values? Off, Initial, Recreate, InPlaceOrRecreate, Auto (Auto is deprecated).

I11. What does InPlaceOrRecreate do on failure? Falls back to evicting/recreating the Pod when the in-place update is infeasible, deferred >5m, in-progress >1h, or would change QoS.

I12. Why must you not run HPA and VPA on the same metric? VPA changes requests while HPA scales replicas on utilization relative to requests; together on one metric they oscillate.

I13. What is the precedence when a resize touches resources with different resizePolicy? RestartContainer wins — if any changed resource needs a restart, the container restarts.

I14. How does the scheduler account for an in-flight resize? It uses max(desired, allocated, actual) for placing new Pods (excluding desired for Infeasible resizes).

I15. Which container types can and cannot be resized? Sidecars (restartable init containers) and regular containers can; non-restartable init containers and ephemeral containers cannot.

I16. What VPA controlledValues setting preserves the limit:request ratio? RequestsAndLimits (the default). RequestsOnly leaves limits untouched.

I17. What event reason does VPA emit when it resizes in place? InPlaceResizedByVPA ("Pod was resized in place by VPA Updater.").

I18. Which kubelet resize metrics exist (names)? Six, per KEP-1287: kubelet_container_requested_resizes_total, kubelet_pod_in_progress_resizes, kubelet_pod_pending_resizes, kubelet_pod_infeasible_resizes_total, kubelet_pod_deferred_resize_accepted_total, and the kubelet_pod_resize_duration_seconds histogram.

I19. If the Pod has restartPolicy: Never, what constraint applies to resizePolicy? Every resizePolicy entry must be NotRequired (you cannot ask to restart a container the Pod may not restart).

I20. How do you make a resize durable across Pod recreation? Update the controller template as well (or let VPA manage requests) — a Pod-only resize reverts on recreation.

19.3 Senior (20)

S1. Walk through the end-to-end path of a CPU resize. Client PATCHes pods/resize → apiserver validates (cpu/mem only, QoS invariant) and persists DESIRED → kubelet watches, admits/allocates (checkpoint), orders changes, calls CRI UpdateContainerResources → runtime writes cpu.weight/cpu.max → kubelet reads back ACTUAL via ContainerStatus, updates status, emits events/metrics. The scheduler is not in the actuation path.

S2. Why is comparing actuated vs actual necessary for correctness, not just efficiency? Because desired and actual can legitimately differ (kernel minimums, systemd rounding, NRI mutation). Reconciling against desired would never converge; reconciling against actuated converges to what the kubelet actually intended to set.

S3. Explain the multi-container atomic resize ordering and why it exists. Net-increases raise the pod-level cgroup first, then apply container decreases, then lower the pod-level cgroup on net-decrease, then apply container increases. Decreases before increases guarantee the sum of container limits never transiently exceeds the pod cgroup limit.

S4. What is the scheduler↔kubelet race and how do you mitigate it? The scheduler and kubelet can momentarily disagree on capacity, over-scheduling a node so a Pod is rejected with OutOfCPU/OutOfMemory (#126891, unresolved at GA). Mitigate with node headroom and canarying aggressive resize automation.

S5. How would you design a memory-decrease workflow to avoid OOM? Decrease in small steps off the hot path, monitor working-set headroom (limit − working_set), and for runtimes that cannot shrink live, use RestartContainer so the decrease applies to a fresh process rather than risking OOM on the running one.

S6. Reconcile "VPA InPlaceOrRecreate is beta" with "it's GA in VPA v1.6.0." Two layers: the cluster feature (in-place resize) is GA in k8s v1.35; the VPA mode that consumes it was beta in the 1.35-era blog (AEP-4016) and reached GA in VPA v1.6.0. The gate is default-on-but-present in v1.6.0 and removed in v1.7.0.

S7. How does the resize subresource improve the security posture over mutable pods? It is a distinct RBAC object (pods/resize), so "may resize" is grantable independently of "may rewrite Pod spec." This lets you give VPA/operators resize rights without broad Pod write — a least-privilege seam.

S8. What are the observability surfaces for resize, and what is explicitly absent? Metrics (six kubelet [ALPHA] metrics + apiserver/runtime SLIs) and Pod events (ResizeStarted/ResizeCompleted/Killing/InPlaceResizedByVPA). Absent: distributed traces — the kubelet emits none for resizes.

S9. Design alerting for a fleet using in-place resize. Alert on PodResizePending{reason=Infeasible} (human action), sustained {reason=Deferred} (capacity), rising kubelet_pod_infeasible_resizes_total, a persistently non-zero kubelet_pod_in_progress_resizes (kubelet backed up), and resize duration p95 regressions.

S10. Why is a resize cheaper than recreate-based right-sizing? It does not invoke the scheduler (Pod already bound) and skips deschedule/schedule/ image-pull/startup. The common CPU case is a single cgroup write.

S11. How do quota and resize interact? ResourceQuota accounts using max(desired,allocated,actual), so an upsize that would breach quota is rejected, and an Infeasible resize's desired is excluded from the max.

S12. Explain the version-skew handling on upgrade. Upgrade control plane before kubelets. The apiserver infers kubelet feature support by checking whether running containers' status...resources is populated, and handles mixed-version behavior accordingly.

S13. When does VPA skip the PodDisruptionBudget for in-place updates? Only with --in-place-skip-disruption-budget=true and when the update is genuinely non-disruptive (all containers NotRequired for cpu and memory). PDB is still enforced if any container uses RestartContainer or the update falls back to eviction.

S14. What is the "actuated" state and why is it checkpointed? The config the kubelet sent the runtime. It is checkpointed on the node so a kubelet restart can resume reconciliation (compare checkpointed-actuated vs runtime-actual) rather than re-deriving from desired.

S15. How do PriorityClass and QoS combine to order deferred resizes, and what's the limit of that control? Retry order is PriorityClass → QoS → deferred duration, but a higher-priority pending resize does not block others (all are still attempted), and ordering only affects retry sequencing — it does not guarantee prompt actuation on a tight node.

S16. Describe a supported HPA+VPA topology and why it's stable. HPA on custom/external metrics (RPS, queue depth) for replicas + VPA on cpu/memory for sizing. The signals are orthogonal, so there is no feedback loop between replica count and requests.

S17. What node configurations make a resize Infeasible regardless of capacity? Static CPU Manager and static Memory Manager (for Guaranteed Pods), swap-enabled containers (for memory requests without RestartContainer), static Pods, and Windows Pods.

S18. How would you correlate "VPA decided" with "kubelet actuated"? Join VPA updater metrics (vpa_updater_in_place_updated_pods_total, vpa_updater_failed_in_place_update_attempts_total) and the InPlaceResizedByVPA event with kubelet resize metrics/events on the same Pod/time window.

S19. Why is NotRequired not a guarantee of no restart? If the runtime discovers a resize truly needs a restart, it returns an error and the kubelet retries; in edge cases a container can end stopped. NotRequired expresses intent, not a hard guarantee.

S20. What is the special GA eviction behavior and its scope? For system-node-critical/system-cluster-critical Pods with no room, the kubelet may evict a non-critical Pod to make room for the resize. General preemption for non-critical resizes is out of scope for GA.

20. FAQ

Q1. Do I need to enable any feature gate on Kubernetes v1.35? No. InPlacePodVerticalScaling is GA and on by default.

Q2. Will resizing restart my container? CPU with NotRequired does not. Memory with RestartContainer does (a container restart in place, not a Pod recreation). It depends on your resizePolicy and runtime.

Q3. Does resizing ever recreate the Pod? No. The Pod object, name, UID, node, and IP are preserved. Only a container may restart.

Q4. Can I resize storage or GPUs in place? No. Only cpu and memory are mutable in v1.35.

Q5. Why was my resize marked Infeasible? The request can't be satisfied as-is — usually it exceeds node capacity, or the node uses an unsupported config (static CPU/Memory Manager, swap, Windows). It is never retried; patch a feasible value or add capacity.

Q6. Why is my resize stuck Deferred? The node lacks room right now (possibly behind higher-priority resizes). It is retried periodically; free capacity, raise PriorityClass, or scale the node pool.

Q7. Is decreasing memory safe? Not guaranteed. The kubelet does a best-effort OOM check, but a TOCTOU race means an over-tight limit can OOM-kill on cgroup v2. Decrease conservatively.

Q8. Why didn't editing my Deployment resize the running Pod? Template edits affect future Pods; the running Pod is mutated only via the resize subresource on the Pod.

Q9. Why does my resize revert after a rollout? A Pod-only resize is not persisted to the controller template. Update the template or use VPA so recreated Pods get the current values.

Q10. What kubectl version do I need? Client ≥ v1.32.0 for --subresource=resize; server ≥ v1.33 (v1.35 recommended).

Q11. How do I know a resize finished? When ACTUAL equals DESIRED and there is no PodResizePending/PodResizeInProgress condition. Use scripts/verify-resize.sh.

Q12. Can I resize a Pod's QoS class? No — QoS is invariant across a resize by validation.

Q13. Does VPA need the feature gate flag? On VPA v1.6.0 the InPlaceOrRecreate gate is default-on; passing the flag is a harmless no-op. On v1.4.0 it's required; on v1.7.0 it's removed (passing it crashes the component).

Q14. Can HPA and VPA work together? Only on different metrics (HPA on custom/external, VPA on cpu/memory). Never on the same metric.

Q15. Where do the resize metrics live? The kubelet /metrics endpoint; scrape via the kubelet ServiceMonitor or the API-server node proxy.

Q16. Are there distributed traces for resizes? No. Observability is metrics + Pod events. You can instrument your app to trace a self-triggered resize, but the platform emits no resize spans.

Q17. What RBAC lets a user resize but not otherwise edit Pods? patch/update on pods/resize (plus read on pods). Write on pods alone does not permit resource mutation.

Q18. Does resizing invoke the scheduler? No. The Pod is already bound; the scheduler only accounts for the resize when placing other Pods.

Q19. What restart cost does a RestartContainer memory resize incur? A container restart: process restart, dropped in-memory state/connections, re-run startup probes. Image is already present, so no pull. Plan it like a controlled restart.

Q20. Can I resize sidecars? Yes — sidecars (restartable init containers) can be resized. Non-restartable init containers and ephemeral containers cannot.

Q21. What happens to a resize if the kubelet restarts mid-actuation? The kubelet checkpoints the actuated state and resumes reconciliation on restart, re-issuing only if runtime-actual differs from checkpointed-actuated.

Q22. Is cgroup v1 supported? cgroup v2 is the supported baseline. Memory-decrease semantics differ (v1 rejects the write; v2 OOM-kills), and the feature is exercised/validated on cgroup v2.

21. Key Takeaways

The reservation is now decoupled from the process lifecycle. In v1.35 you can change cpu/memory requests and limits on a running Pod without recreating it — GA, on by default, no feature gate.
The mental model is DESIRED (spec) vs ACTUAL (status). A resize is converged when they match; everything else is a state in the kubelet's Desired→Allocated→Actuated→Actual machine.
Use the resize subresource. Resources are immutable on the main Pod resource; kubectl patch pod --subresource resize is the only path, and it doubles as a clean pods/resize RBAC seam.
resizePolicy is a runtime decision. CPU applies live (NotRequired); memory often needs RestartContainer (a container restart in place — not a Pod recreation).
Status is conditions, not the old string field. Read PodResizePending (Deferred/Infeasible) and PodResizeInProgress; alert on them.
GA added memory-limit decreases (best-effort OOM check) and prioritized/deferred resizes ordered by PriorityClass → QoS → deferred duration.
Know the boundaries. Only cpu/memory; QoS invariant; no swap/static CPU/Memory managers; no Windows; a scheduler↔kubelet race exists; resize stays within a node's capacity.
VPA InPlaceOrRecreate is the automation path (GA in VPA v1.6.0), emitting InPlaceResizedByVPA and preserving the limit:request ratio.
Observability is metrics + events, not traces — six [ALPHA] kubelet resize metrics plus Pod events; correlate desired (kube-state-metrics) with actual (cAdvisor).
Everything here is reproducible. Pinned kind v1.35 cluster, pinned charts, and a Taskfile drive the whole demo; the example outputs in Section 10 come from a Kubernetes v1.35 kind cluster.

22. References

Official and primary sources.

Kubernetes — In-Place Pod Resize (v1.35 GA)

GA blog (2025-12-19): https://kubernetes.io/blog/2025/12/19/kubernetes-v1-35-in-place-pod-resize-ga/
Task: Resize CPU and Memory Resources assigned to Containers: https://kubernetes.io/docs/tasks/configure-pod-container/resize-container-resources/
Task: Resize CPU and Memory Resources assigned to Pods: https://kubernetes.io/docs/tasks/configure-pod-container/resize-pod-resources/
Pod Conditions concept: https://kubernetes.io/docs/concepts/workloads/pods/pod-condition/
KEP sig-node/1287 (In-Place Update of Pod Resources): https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/1287-in-place-update-pod-resources/README.md
Kubernetes Metrics Reference: https://kubernetes.io/docs/reference/instrumentation/metrics/
System traces (API server / kubelet tracing): https://kubernetes.io/docs/concepts/cluster-administration/system-traces/
KEP sig-node/2831 (kubelet tracing): https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/2831-kubelet-tracing
About cgroup v2 (node cgroups): https://kubernetes.io/docs/concepts/architecture/cgroups/

Vertical Pod Autoscaler

Features (InPlaceOrRecreate, InPlace, gate, limitations, metrics, CPU boost): https://github.com/kubernetes/autoscaler/blob/master/vertical-pod-autoscaler/docs/features.md
Quickstart (update modes, example): https://github.com/kubernetes/autoscaler/blob/master/vertical-pod-autoscaler/docs/quickstart.md
Components (recommender/updater/admission-controller): https://github.com/kubernetes/autoscaler/blob/master/vertical-pod-autoscaler/docs/components.md
Known limitations (HPA conflict): https://github.com/kubernetes/autoscaler/blob/master/vertical-pod-autoscaler/docs/known-limitations.md
AEP-4016 (in-place updates support): https://github.com/kubernetes/autoscaler/tree/master/vertical-pod-autoscaler/enhancements/4016-in-place-updates-support
AEP-7862 (CPU startup boost): https://github.com/kubernetes/autoscaler/tree/master/vertical-pod-autoscaler/enhancements/7862-cpu-startup-boost
AEP-8818 (eviction-free InPlace): https://github.com/kubernetes/autoscaler/tree/master/vertical-pod-autoscaler/enhancements/8818-in-place-only
Fairwinds charts (stable/vpa): https://github.com/FairwindsOps/charts

Observability stack

kube-prometheus-stack chart: https://github.com/prometheus-community/helm-charts/blob/main/charts/kube-prometheus-stack/README.md
Prometheus Operator API reference: https://prometheus-operator.dev/docs/api-reference/api/
kube-state-metrics pod metrics: https://github.com/kubernetes/kube-state-metrics/blob/main/docs/metrics/workload/pod-metrics.md
Grafana dashboard JSON model: https://grafana.com/docs/grafana/latest/dashboards/build-dashboards/view-dashboard-json-model/
Grafana import dashboards: https://grafana.com/docs/grafana/latest/dashboards/build-dashboards/import-dashboards/
OpenTelemetry Collector on Kubernetes (Helm): https://opentelemetry.io/docs/platforms/kubernetes/helm/collector/
OTel kubeletstats receiver: https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/kubeletstatsreceiver/documentation.md
OTel k8scluster receiver: https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/k8sclusterreceiver

Tooling

kind: https://kind.sigs.k8s.io/
metrics-server: https://kubernetes-sigs.github.io/metrics-server/
Helm: https://helm.sh/
Task (go-task): https://taskfile.dev/

23. SEO Block

Meta Description (≤160 chars): In-Place Pod Vertical Scaling is GA in Kubernetes 1.35. A production deep dive: resize subresource, resizePolicy, status conditions, VPA InPlaceOrRecreate, metrics — with verified output.

Keywords: in-place pod resize, Kubernetes 1.35, InPlacePodVerticalScaling, pod resize subresource, resizePolicy, RestartContainer, PodResizePending, PodResizeInProgress, Vertical Pod Autoscaler, InPlaceOrRecreate, VPA in-place, kubelet resize metrics, vertical scaling Kubernetes, right-sizing pods, cgroups v2 resize, kube-prometheus-stack, OpenTelemetry Collector kubelet.

Slug: in-place-pod-vertical-scaling-kubernetes-1-35-production-deep-dive

Twitter/X Title: In-Place Pod Vertical Scaling in Kubernetes 1.35: the production deep dive (GA, no feature gate) — resize subresource, VPA InPlaceOrRecreate, real verified output.

Social Media Summary: Kubernetes 1.35 makes cpu/memory requests & limits mutable on running Pods — no recreation. This deep dive covers the desired-vs-actual model, the resize subresource, resizePolicy (NotRequired vs RestartContainer), the new status conditions, VPA InPlaceOrRecreate, RBAC, and observability — all reproducible with pinned IaC and real verified command output.

24. LinkedIn Post

Kubernetes 1.35 quietly changes how we think about resource management: In-Place Pod Vertical Scaling is now GA — on by default, no feature gate.

For years, changing a Pod's CPU or memory meant deleting and recreating it. That's fine for stateless web apps and painful for everything else: JVMs with long warm-ups, game servers holding live sessions, pre-warmed workers, anything stateful. You either over-provisioned "to be safe" or paid a restart to right-size.

Now you patch the Pod's resize subresource and the kubelet reconfigures the running container's cgroups in place. CPU scales with no restart. Memory can restart the container in place (not the Pod) for runtimes that need it.

I wrote a production deep dive — not a feature tour — covering: • The desired-vs-actual model and why convergence ≠ "the patch returned 200" • resizePolicy: NotRequired vs RestartContainer, and the QoS invariant • The new PodResizePending / PodResizeInProgress conditions (the old string field is gone) • VPA InPlaceOrRecreate driving resizes automatically — with real verified output • RBAC on pods/resize as a least-privilege seam • Observability: the six kubelet resize metrics + events (and why there are no traces) • Every failure mode: Infeasible, Deferred, memory-decrease OOM risk, the scheduler↔kubelet race

Fully reproducible: pinned kind v1.35 cluster, pinned Helm charts, a Taskfile, and command output captured on a live cluster.

This is Article 1 of a 5-part series on next-gen Kubernetes resource management. Next up: DRA and GPU scheduling.

#Kubernetes #CloudNative #DevOps #PlatformEngineering #CNCF #SRE

25. 10 Advanced Topics That Build on This Article

In-place resize is the vertical dimension of resource management: change how much a running Pod gets. The rest of this 5-part series moves outward — to what kind of resource, when Pods run together, and how large models are served. Ten topics that build on this foundation:

Dynamic Resource Allocation (DRA) and GPU scheduling — (Article 2) Requesting and sharing structured devices (GPUs, accelerators) via ResourceClaim/ ResourceSlice, and where DRA's device model meets (and differs from) the cpu/memory resize model in this article.
In-place resize for accelerator-adjacent resources — why GPUs are not resizable in place today, what would have to change, and how DRA reshapes that question.
Gang scheduling and co-scheduling — (Article 3) All-or-nothing placement for tightly-coupled Pods, and how vertical resize interacts with gangs whose members must grow together.
Kueue and quota-aware batch admission — how in-place Deferred/Infeasible signals should feed a job-queueing layer, and admission-time vs runtime right-sizing.
Cluster Autoscaler / Karpenter awareness of resize pending signals — turning Deferred/Infeasible resizes into node-provisioning decisions instead of stuck Pods.
LLM inference serving and vertical scaling — (Article 4) Right-sizing CPU/memory around model load, KV-cache growth, and batch dynamics for large-model serving, where recreation is especially expensive.
llm-d and disaggregated serving — (Article 5) Prefill/decode disaggregation and how per-phase resource profiles could exploit in-place resize for the CPU/host side of an accelerator-heavy serving stack.
CPU Startup Boost (AEP-7862) in anger — automating the startup-vs-steady-state CPU pattern from Section 14.2 across a fleet, built directly on the in-place scale-down.
Pod-level resources resize — moving from per-container to pod-level (alpha in v1.35) and what it means for sidecar-heavy and mesh workloads.
Closing the scheduler↔kubelet consistency gap — the design work behind #126891, and what a race-free resize/schedule interaction would enable for aggressive bin-packing.

Article 2 picks up the thread with DRA and GPU scheduling — moving from "how much cpu/memory" to "which device, and how shared."

Command Palette

2. Summary

3. Table of Contents

4. Problem Statement

5. Background: QoS, Requests vs Limits, cgroups v2, and the Kubelet

5.1 Requests vs limits

5.2 QoS classes

5.3 cgroups v2

5.4 The kubelet's role

6. Core Concepts

6.1 Definition

6.2 Why it exists

6.3 Internal architecture: the four resource states

6.4 The resize subresource

6.5 resizePolicy — per-container, per-resource restart behavior

6.6 Memory limit decrease and the best-effort OOM check (GA change)

6.7 Status conditions: PodResizePending and PodResizeInProgress

6.8 Prioritized and deferred resizes (GA change)

6.9 Common misconceptions

7. Architecture

7.1 End-to-end request path

7.2 Control plane vs data plane responsibilities

7.3 The /resize subresource security model

7.4 Multi-container atomic resize ordering

7.5 Actuation caveats

8. Installation (IaC-First)

8.1 Pinned versions (iac/helm/versions.env)

8.2 Install the tooling

8.3 The kind cluster (iac/kind/kind-cluster.yaml)

8.4 metrics-server (iac/helm/metrics-server.values.yaml)

8.5 Vertical Pod Autoscaler (iac/vpa/vpa.values.yaml)

8.6 kube-prometheus-stack (iac/helm/kube-prometheus-stack.values.yaml)

8.7 OpenTelemetry Collector (iac/helm/opentelemetry-collector.values.yaml)

8.8 One-command lifecycle (iac/Taskfile.yml)

8.9 Verification

8.10 Common installation mistakes

9. Configuration

9.1 resizePolicy on the workload

9.2 Keeping resizes QoS-legal

9.3 VPA updateMode — the five values

9.4 VPA fallback and disruption behavior

9.5 PriorityClass and resize ordering

9.6 HPA boundary (configuration you must not create)

10. Hands-On Tutorial

10.0 Prerequisites

10.1 Deploy the demo workload and confirm the baseline

10.2 CPU resize — NotRequired, no restart

10.3 Memory resize — RestartContainer, restarts the container in place

10.4 The events tell the whole story

10.5 Verify convergence with the helper script

10.6 Infeasible resize — PodResizePending / reason Infeasible

10.7 Kubelet resize metrics

10.8 metrics-server: kubectl top

10.9 Automate with VPA (InPlaceOrRecreate) — end to end

10.10 Prometheus + Grafana (expected output — not executed here)

10.11 OpenTelemetry Collector (expected output — not executed here)

11. Production Best Practices

12. Common Mistakes

13. Troubleshooting Guide

14. Real Production Use Cases

14.1 Game servers and stateful session holders

14.2 JIT / startup boost

14.3 Pre-warmed worker pools

14.4 Bin-packing and cost efficiency

14.5 Batch and phase-based workloads

15. Security Considerations

15.1 RBAC: the pods/resize subresource

15.2 The kubelet/Node identity is the only writer of actual state

15.3 Resource-exhaustion and abuse surface

15.4 Transport security for the metrics path

15.5 Grafana and stack credentials

15.6 Audit trail

16. Performance Considerations

16.1 Actuation latency

16.2 The scheduler↔kubelet race (correctness under load)

16.3 Memory-decrease TOCTOU

16.4 Resize storms

16.5 No effect on scheduling throughput

17. Integration with the Ecosystem

17.1 Metrics Server

6.4 The `resize` subresource

6.5 `resizePolicy` — per-container, per-resource restart behavior

6.7 Status conditions: `PodResizePending` and `PodResizeInProgress`

7.3 The `/resize` subresource security model

8.1 Pinned versions (`iac/helm/versions.env`)

8.3 The kind cluster (`iac/kind/kind-cluster.yaml`)

8.4 metrics-server (`iac/helm/metrics-server.values.yaml`)

8.5 Vertical Pod Autoscaler (`iac/vpa/vpa.values.yaml`)

8.6 kube-prometheus-stack (`iac/helm/kube-prometheus-stack.values.yaml`)

8.7 OpenTelemetry Collector (`iac/helm/opentelemetry-collector.values.yaml`)

8.8 One-command lifecycle (`iac/Taskfile.yml`)

9.1 `resizePolicy` on the workload

9.3 VPA `updateMode` — the five values

10.2 CPU resize — `NotRequired`, no restart

10.3 Memory resize — `RestartContainer`, restarts the container in place

10.6 Infeasible resize — `PodResizePending` / reason `Infeasible`

10.8 metrics-server: `kubectl top`

10.9 Automate with VPA (`InPlaceOrRecreate`) — end to end

15.1 RBAC: the `pods/resize` subresource

18.4 CRI contract and `UpdatePodSandboxResources`

18.5 The `allocatedResources` status and its gate