Quantifying Value in System Design: Boundaries, Frameworks, and a Practical Checklist

1. Draw the Line First: Engineering Weights Are Not Ethical Pricing

This article deals with engineering analogies. Assigning priorities to services and setting risk weights is about making consistent decisions when resources are limited. These numbers reflect business impact and cost—they do not represent the value of human life and must never be used to trade off lives across groups of people.

In AI ethics contexts, such as self-driving hazard avoidance or the cost of misdiagnosis in medical systems, numerical methods are only for organizing risk analysis. Boundaries involving personal safety must be hard constraints, determined by ethical review and human decision-making—not solved automatically by weighted formulas.

2. Turning Business Impact into Comparable Numbers

Before allocating resources, teams need to standardize three things:

1. Business impact: estimated direct revenue loss, compliance risk, and customer churn from a single outage, expressed in monetary terms or levels (P0 through P3).
2. Cost: compute, storage, and network expenses a service consumes, normalized to a monthly or annual basis.
3. Assumption source: the estimation basis for each number and its last update date.

A simple metric compares the priority of different services under the same budget: business impact per unit of cost, i.e., Impact / Cost. Use it only for ranking and tilting resources—never to declare one service “more important.” Revise these numbers quarterly based on monitoring data and incident postmortems. Any number that is estimated but not measured should be flagged in documentation.

3. Kubernetes Scheduling: QoS, PriorityClass, and PDB

These three mechanisms govern different things and are easily confused:

• QoS classes (Guaranteed, Burstable, BestEffort) are determined by a Pod’s requests and limits. When requests equal limits, the Pod gets Guaranteed class and is less likely to be killed under OOM conditions.
• PriorityClass value determines scheduling order and, under resource pressure, which Pods can preempt others or be preempted. It does not change QoS class.
• PodDisruptionBudget (PDB) limits the number of simultaneously unavailable replicas during voluntary evictions (node drains, rolling maintenance) and scheduler preemption. The scheduler tries to pick victims that don’t violate the PDB, but may violate it when no other option exists. Kubelet evictions driven by node pressure do not respect PDBs.

The following configuration assigns a high priority to a payment service and a low priority to offline jobs:

apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
  name: business-critical
value: 100000
globalDefault: false
preemptionPolicy: PreemptLowerPriority
description: "Core transaction path; may preempt low-priority batch workloads."
---
apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
  name: batch-offline
value: 1000
globalDefault: false
preemptionPolicy: Never
description: "Offline reporting and data cleaning; never preempts others, may be evicted under pressure."
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: payment-api
spec:
  replicas: 3
  selector:
    matchLabels:
      app: payment-api
  template:
    metadata:
      labels:
        app: payment-api
    spec:
      priorityClassName: business-critical
      containers:
        - name: api
          image: example/payment-api:1.4.2
          resources:
            requests:
              cpu: "1"
              memory: 1Gi
            limits:
              cpu: "1"
              memory: 1Gi
---
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: payment-api-pdb
spec:
  minAvailable: 2
  selector:
    matchLabels:
      app: payment-api

With 3 replicas and minAvailable set to 2, at most 1 replica can be unavailable at any time due to voluntary eviction.

Three things to note:

1. When a Pod is created, its priority value is written into spec.priority. Later changes to a PriorityClass’s value do not affect the priority of already-running Pods.
2. With preemptionPolicy: Never, batch-offline will not preempt other services, but it can still be preempted by higher-priority workloads or evicted under node pressure.
3. Don’t rely on priority alone to protect critical services. They also need sufficient replicas, distribution across nodes or availability zones, and reasonable requests.

4. Storage Tiering: Decide by Access Frequency and Billing Structure

Tiering should be based on access frequency, read latency requirements, and billing structure—not on a dataset’s “value.” Monthly cost breaks down as:

Monthly cost = storage price × data volume + read price × monthly reads

When moving data from a hot tier to an infrequent-access tier, the storage savings must exceed the added retrieval cost. The condition is:

(hot-tier storage price − cool-tier storage price) × data volume > (cool-tier read price − hot-tier read price) × monthly reads

Before migrating, do three things: measure read counts and retrieval volume over the last 30 days; confirm the cool tier’s minimum storage duration and retrieval latency meet business requirements; and set up lifecycle rules, then reconcile actual costs per tier against the monthly bill. Refer to your cloud provider’s current price list for exact pricing.

5. Redundancy Calculation: A Reproducible Cost Model

Adding replicas increases storage and compute costs, and adds sync latency and write-throughput loss, but reduces the probability of a full outage. Use this model to compare annual total cost for different replica counts N:

Total cost(N) = c × N + s × (N − 1) + C × P_N

• c: annual cost per replica.
• s: estimated annual sync and throughput loss per additional replica.
• C: annual loss from a single full outage.
• P_N: annual probability that all N replicas are unavailable simultaneously.

P_N must distinguish independent failures from common-cause failures. Let q be the per-replica independent failure probability and q_c the common-cause failure probability (e.g., a power outage in one availability zone, or a defect in a shared software version). Then:

P_N = q_c + (1 − q_c) × q^N

The common-cause term is independent of N. If all replicas sit in the same failure domain, adding replicas barely reduces this part of the risk. Multi-AZ deployment reduces q_c.

Example assumptions: c = 20,000 (currency units), s = 5,000, C = 5,000,000, q = 0.02, q_c = 0.001. Results:

Replicas N	P_N	Annual Total Cost
1	0.02098	124,900
2	0.0013996	52,000
3	0.001008	75,000

Under these assumptions, 2 replicas minimize cost. If outage loss drops to 1,000,000, the single-replica annual cost is about 41,000, lower than roughly 46,400 for 2 replicas—adding replicas is then not worthwhile. Conclusions hold only for the assumptions used.

When applying the model: estimate q and q_c from the past year’s incident records rather than copying the example values; sync loss s is not necessarily linear at higher replica counts; and the numbers above only demonstrate the calculation method—they cannot be applied directly to other systems.

6. AI Systems: Multi-Objective Rewards with Human Fallback

Recommendation systems and agents usually optimize more than one objective. Beyond task reward, side effects such as user over-engagement and filter bubbles should enter the reward as penalty terms:

HARD_LIMIT_HARM = 0.8
WEIGHTS = {"task": 1.0, "engagement": 0.3, "harm": 5.0}
class HardConstraintViolation(Exception):
    pass
def total_reward(task_score, excess_engagement, harm_score):
    if harm_score >= HARD_LIMIT_HARM:
        # Hard constraints bypass weighting and go straight to safe fallback or human review
        raise HardConstraintViolation("Harm score exceeds hard threshold")
    return (WEIGHTS["task"] * task_score
            - WEIGHTS["engagement"] * excess_engagement
            - WEIGHTS["harm"] * harm_score)

Follow this process for weight calibration:

1. Weights are set jointly by product, security, and ethics owners, with review records kept.
2. Evaluate offline on labeled datasets, checking whether the model games penalties by inflating one score (reward hacking).
3. Use A/B experiments to observe business metrics while monitoring safety and experience guardrail metrics.
4. Review weights and thresholds monthly; any weight change requires a new review.
5. Keep human review for high-impact outputs (medical advice, financial decisions, safety-related scenarios).

Weighting only applies to objectives that can be traded off against each other. Content involving personal safety, illegal activity, or irreversible harm must use hard constraints—the model must never trade these for other gains.

7. Rollout Checklist and Rollback Plan

Checklist:

• Every service has a documented business impact level (P0–P3) with supporting rationale.
• Critical services set requests equal to limits to achieve Guaranteed QoS.
• Every PriorityClass value has an approval record; custom values stay below one billion.
• For services with PDBs, minAvailable is less than the replica count, leaving at least one interruptible slot.
• Batch jobs use low priority and set preemptionPolicy: Never where appropriate.
• Monitor: Pending Pod duration, preemption event count, node-pressure eviction count, and disruptions allowed by PDBs.
• Storage access frequency is tracked per tier, bills are reconciled monthly, and lifecycle rules have owners.
• Replicas are distributed across availability zones; redundancy model inputs are recalibrated quarterly with real incident data.
• AI reward weights and thresholds have change records, and high-risk outputs have a human review path.

Rollback plan:

1. Keep the previous Deployment and PDB manifests; export them before any change.
2. If a priority change causes business issues, revert priorityClassName to the old value and perform a rolling update. Note that running Pods do not pick up PriorityClass changes—Pods must be recreated for the change to take effect.
3. Before deleting a PriorityClass, verify no workload still references it:
   kubectl get pods -A -o custom-columns=NAME:.metadata.name,PC:.spec.priorityClassName
4. If a PDB blocks node drains, consider temporarily relaxing minAvailable, and record the reason and recovery time.
5. After redundancy or storage tiering changes, keep the old configuration for at least one full billing cycle to compare costs and failure rates.