Discussion: SLA Strategies, Eviction Policies, and Resource Management in Kubernetes Deployments

[andy108369]

This discussion stems from observations and suggestions by Samuel Terburg regarding Kubernetes best practices for SLA management, eviction policies, and resource allocation. Below are the key points for consideration and discussion:

SLA Definition on Deployments

• User-Defined SLA Preferences:
  Allow users to define the SLA they need for their deployments.
  • Low SLA (Short Downtime Tolerable): Use Recreate strategy, where the old pod is taken down before a new one is spun up.
  • High SLA (High Availability Required): Use RollingUpdate strategy with replicas set to 2 or more.

Eviction Policy Optimization

• Priority-Based Evictions:
  • Configure EvictionPolicy to evict lower-priority pods when higher-priority pods need scheduling or when a node experiences eviction pressure.
  • After eviction, once temporary disruptions (e.g., extra pods during rolling updates) are resolved, evicted lower-priority pods should fit back onto the node.

Resource Reservations

• Resource Quotas for Purchased Resources:
  • Use ResourceQuota to reserve user-purchased resources.
  • Ensure that a certain portion of resources is always available to meet the user's needs.

Recommendations for Providers

1. Cluster Management:

   • Providers should never let clusters fill up completely.
   • Keep a mandatory margin of free resources.

2. Monitoring and Capacity Management:

   • Enforce capacity monitoring and management as a mandatory guideline.

3. Proactive Focus:

   • The goal should be to prevent node fill-ups and capacity issues, not just to remediate them reactively.

Rationale

• Kubernetes offers robust features for SLA management, scheduling, and resource optimization.
• Using shell scripts or cronjobs to address such issues is not aligned with cloud-native principles.
• These features and best practices should be fully utilized to handle scheduling and resource management more effectively.

---

Let's discuss how these practices can be implemented and refined to better meet user needs while adhering to Kubernetes' cloud-native philosophy.