POC resource baseline (requests / limits)
Starting values for a single-node POC: what to put in each service's four resource fields, why, and which to change first. This is a demo and acceptance baseline, not production sizing.
1. Three different things
| Field | What it controls | What goes wrong |
|---|---|---|
| requests | Scheduling: the node reserves this capacity; it also affects eviction order | Too high → new Pods stay Pending; too low → the node is oversubscribed and these Pods are evicted first under pressure |
| limits | Runtime ceiling: exceeding the CPU limit throttles, exceeding the memory limit OOMKills | Too low → slow or killed; too high → one service can eat the whole machine |
| Actual usage | Real consumption; used only to choose the two values above | Does not participate in scheduling |
Scheduling looks at requests only. "The node still has 1.5 GiB" and "a new Pod can start" are
different questions: kubectl top node shows actual usage, while Allocated resources in
kubectl describe node is the scheduler's view.
When a LimitRange exists (packingtable creates one), containers without explicit resources receive its defaults; those defaults are usually larger than the service needs, and are the usual reason requests fill a single POC node and new Pods cannot schedule.
2. What the reference environment measured
One 4 vCPU / 8 GiB single node with every service running and Istio injection on, sampled every 15
seconds for four minutes with kubectl top pods --containers:
- Most application containers peaked at 17-70 MiB. Three exceptions:
theia-agentabout 192-226 MiB,tethys-shippingabout 221-244 MiB, andmcp-perceptionabout 108-144 MiB. - Each
istio-proxyactually used 32-41 MiB, but its default request is 128 MiB. - As a result, memory requests sat at 96-99% of the node while actual usage was only about
80%. Pods stuck
Pendingand rolling updates that never finish come from requests, not from memory actually running out.
3. Recommended values
Requests leave a little headroom above the peak; limits are a ceiling against one service running away, not a target.
| Product line | Service | requests CPU | requests memory | limits CPU | limits memory |
|---|---|---|---|---|---|
| rhea | rhea-email |
50m | 64Mi | 500m | 512Mi |
rhea-payment |
50m | 64Mi | 500m | 512Mi | |
rhea-push |
50m | 64Mi | 500m | 512Mi | |
rhea-gateway |
50m | 64Mi | 500m | 512Mi | |
rhea-web |
50m | 96Mi | 500m | 512Mi | |
| themis | themis-auth |
50m | 64Mi | 500m | 512Mi |
themis-gateway |
50m | 96Mi | 500m | 512Mi | |
themis-web |
50m | 96Mi | 500m | 512Mi | |
| theia | theia-agent |
100m | 256Mi | 1000m | 1Gi |
theia-gateway |
50m | 96Mi | 500m | 512Mi | |
theia-web |
50m | 96Mi | 500m | 512Mi | |
mcp-* |
50m | 128Mi | 500m | 1Gi | |
| tethys | tethys-shipping |
100m | 256Mi | 1000m | 1Gi |
tethys-gateway |
50m | 64Mi | 500m | 512Mi | |
tethys-web |
50m | 96Mi | 500m | 512Mi | |
| phoebe | phoebe-gateway |
50m | 64Mi | 500m | 512Mi |
phoebe-web |
50m | 96Mi | 500m | 512Mi |
The product line is the family label tethys lists by (fewbox.com/family). Filter the list by it,
change one line at a time, and move on only once the Pods are healthy.
mcp-* is one row for every MCP service: the five bundled ones and the three optional ones are all
Node services of similar size, so they share one set of values rather than being tuned
individually. mcp-perception peaked at about 144Mi, slightly above its request - that is
deliberate Burstable behaviour, with the 1Gi limit as the ceiling. The optional data-source
services (mcp-gsc, mcp-domain, mcp-trends) use the same values when installed, adding
50m / 128Mi each to the totals.
Totals (the 21 services above, mcp-* counted as 5, one Pod each): application requests about
1150m / 2.2 GiB; adding the sidecars (25m / 128Mi × 21) brings it to about
1675m / 4.8 GiB. Compressing sidecar memory to 64Mi as described in the next section lowers
that to about 3.5 GiB. If a service runs more than one replica, multiply by the Pod count.
The reference node has about 7.5 GiB allocatable. At 3.5 GiB, requests occupy roughly 47%; the rest is what makes rolling updates and bursts possible. That is why a single node can refuse a new Pod while actual usage sits at 80%.
If you install theia-jobs-tick, it starts a Pod every five minutes and takes another request at the
LimitRange defaults; include it in the headroom calculation, or it silently fails to schedule on a
full node and the heartbeat stops.
4. How to apply it
- Change the four fields in the service details in tethys; change one group, confirm the Pods are healthy, then move to the next.
- Order: change the nineteen 50m services first, and
theia-agent/tethys-shippinglast. The earlier rolling updates need the spare capacity, so leave the heavy services until the end. - A single-replica rolling update can deadlock: a Deployment defaults to
maxSurge: 25%(rounds up to 1 for one replica) andmaxUnavailable: 25%(rounds down to 0), so the new Pod must start before the old one may stop. If free capacity is below the new Pod's request (roughly 190-320Mi depending on the service), the new Pod staysPendingforever. Three ways out:- free capacity first (lower earlier requests, or briefly stop an unimportant service);
- set
strategytomaxSurge: 0+maxUnavailable: 1and restart in place; - delete the old Pod manually, freeing capacity so the already-
Pendingnew Pod can schedule.
- Lower requests make the Pod Burstable: if actual usage stays above requests, it is evicted first under node pressure. On a single-tenant POC that is acceptable; in production, raise requests toward the 30-day p95.
5. istio-proxy is the other half
Every business Pod carries a native sidecar (the second container in 2/2), with a default request
of 100m / 128Mi. It can be lowered at install time (see Install Istio):
istioctl install --set profile=default --set hub=docker.io/istio \
--set values.global.proxy.resources.requests.cpu=25m \
--set values.global.proxy.resources.requests.memory=64Mi
Each sidecar actually used 32-41 MiB, so 64Mi still leaves roughly double the headroom. Across 21
Pods this saves another ~1.3 GiB.
- On an existing cluster, re-run
istioctl installand then roll the business Pods — the sidecar is part of the Pod template, so changing the control plane does not touch running Pods. - A single Pod can override it with the annotation
sidecar.istio.io/proxyMemory: 64Mi; workloads created by tethys do not always make that convenient, so the global value is simpler for a POC.
6. This table is not production sizing
- The data comes from idle and demo traffic, not a stress test; production values come from actual QPS and the 30-day p95.
- 4C/8G is enough for demos and single-person acceptance; 8C/16G is the recommended POC size (see Cluster preparation).
- After tuning, check
kubectl describe node <NODE_NAME> | grep -A6 "Allocated resources"and keep total requests below 60% of allocatable, leaving room for rolling updates and bursts. - The optional data-source services are installed by importing their Bundle; do not hand-apply per-service manifests. For registration and credentials see MCP tool services.