Deploying Bare-Metal Kubernetes: The High-Performance Guide
Running Kubernetes inside public cloud instances is convenient, but the performance tax is high. Virtualized network interfaces, overlay networks (VXLAN), and network-attached block storage introduce latency and CPU cycles.
To extract absolute performance, large-scale systems are shifting back to bare metal. This guide explains how to build a high-performance bare-metal Kubernetes cluster from scratch.
1. Network Foundation: BGP and Cilium
In a virtualized cloud, the cloud controller manager handles load balancer provisioning. On bare metal, you must manage your own load balancers and node routing.
We recommend Cilium as the Container Network Interface (CNI), configured in eBPF-native routing mode alongside BGP (Border Gateway Protocol).
# Cilium ConfigMap snippet for native routing
routing-mode: native
ipv4-native-routing-cidr: 10.244.0.0/16
auto-direct-node-routes: "true"
bpf-lb-mode: dsr # Direct Server Return- Native Routing: cilium routes pod traffic directly over the physical network interface without encapsulation overhead.
- eBPF Load Balancing: Replaces the legacy, slow IPVS/iptables system, routing network traffic with O(1) complexity.
- Direct Server Return (DSR): Allows load-balanced traffic to return directly from pods to clients, skipping the ingress node hop.
2. Storage System: NVMe-Backed Rook Ceph
For stateful applications, virtual cloud storage is often slow. On bare metal, you can build a distributed storage pool using local NVMe drives with Rook Ceph.
Ceph links the unused local NVMe drives on all nodes into a unified, replicated, and highly available block storage system. Data is written directly to physical drives at PCIe speeds, yielding write speeds of 250,000+ IOPS per volume.
3. High Availability Control Plane
To prevent single points of failure, set up a multi-master control plane using three master nodes. Use kubeadm to initialize the control plane, backed by an external or stacked etcd cluster stored on dedicated local NVMe drives to prevent disk sync delays.
By bypassing hypervisors, your containers gain direct access to raw memory bandwidth and CPU execution ports. The result is faster API processing, reduced scheduling jitter, and lower operation costs.