Traffic Management & Service Mesh

HAProxy, Ingress & Istio

Load balancing, Kubernetes Ingress Controllers, and Istio service mesh — from basics to production traffic management.

Chapters

30+

Configs

100%

Free

01🌐

Traffic Management Overview

Why Load Balancing & Service Mesh Matter

Every production application needs to handle traffic intelligently — distribute requests across servers, route based on URL paths, encrypt with TLS, and manage microservice communication. This guide covers three essential tools: HAProxy (traditional load balancer), Kubernetes Ingress (K8s-native routing), and Istio (service mesh for microservices).

⚖️

HAProxy

High-performance TCP/HTTP load balancer. Routes traffic to backend servers. Used standalone or in front of Kubernetes. Handles millions of connections.

🚪

K8s Ingress

Kubernetes-native way to expose HTTP services. Routes based on hostname and path. Uses Ingress Controllers like Nginx or Traefik.

🕸️

Istio

Service mesh for microservices. Handles traffic routing, security (mTLS), observability, and retries between services inside Kubernetes.

Feature	HAProxy	K8s Ingress	Istio
Layer	L4/L7	L7	L4/L7
Where	Standalone/K8s	Inside K8s only	Inside K8s only
Best For	High-performance LB	Simple HTTP routing	Microservice traffic management
TLS	Yes	Yes	mTLS (automatic)
Observability	Stats page	Basic	Full (Kiali, Jaeger, Prometheus)

02⚖️

HAProxy Basics

High-Performance Load Balancer

HAProxy (High Availability Proxy) is a free, open-source load balancer. It sits in front of your servers and distributes incoming requests. If one server dies, HAProxy stops sending traffic to it. Used by GitHub, Reddit, Twitter, and most large-scale web applications.

How HAProxy Works

Client sends request to HAProxy IP. HAProxy picks a backend server using a load balancing algorithm (round-robin, least connections, etc.). Backend server processes the request and sends the response back through HAProxy. The client never talks to the backend directly.

Installation

TERMINAL# Ubuntu/Debian sudo apt update && sudo apt install haproxy -y # CentOS/RHEL sudo yum install haproxy -y # Verify haproxy -v # HAProxy version 2.x # Config file location /etc/haproxy/haproxy.cfg # Start/restart sudo systemctl start haproxy sudo systemctl enable haproxy

Load Balancing Algorithms

Algorithm	How It Works	When to Use
roundrobin	Cycles through servers one by one	Default, equal servers
leastconn	Sends to server with fewest connections	Long-running requests (WebSocket, DB)
source	Same client IP always goes to same server	Session persistence without cookies
uri	Same URL always goes to same server	Caching servers

03⚙️

HAProxy Configuration

Complete Config Explained

HAPROXY# /etc/haproxy/haproxy.cfg global maxconn 50000 log /dev/log local0 stats socket /run/haproxy/admin.sock mode 660 defaults mode http timeout connect 5s timeout client 30s timeout server 30s option httplog option forwardfor # Pass client IP to backend # Stats dashboard (http://haproxy-ip:8404/stats) frontend stats bind *:8404 stats enable stats uri /stats stats refresh 10s # Main HTTP frontend frontend web_frontend bind *:80 bind *:443 ssl crt /etc/ssl/certs/myapp.pem # Route based on hostname acl is_api hdr(host) -i api.myapp.com acl is_web hdr(host) -i www.myapp.com use_backend api_servers if is_api default_backend web_servers # Backend: web application servers backend web_servers balance roundrobin option httpchk GET /health server web1 10.0.1.10:3000 check server web2 10.0.1.11:3000 check server web3 10.0.1.12:3000 check # Backend: API servers backend api_servers balance leastconn option httpchk GET /api/health server api1 10.0.2.10:8080 check server api2 10.0.2.11:8080 check

Health Checks

The "check" keyword tells HAProxy to ping each server periodically. If a server fails the health check, HAProxy stops sending traffic to it. When the server recovers, traffic resumes automatically. This is how zero-downtime is achieved.

04🚪

Kubernetes Ingress

HTTP Routing in K8s

Ingress is the Kubernetes-native way to expose HTTP and HTTPS services to the outside world. Instead of creating a LoadBalancer Service for every app, you create ONE Ingress that routes traffic based on hostname and URL path.

Ingress Controller Installation

TERMINAL# Install Nginx Ingress Controller kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.9.0/deploy/static/provider/cloud/deploy.yaml # Verify kubectl get pods -n ingress-nginx # NAME READY STATUS # ingress-nginx-controller-xxx 1/1 Running # It creates a LoadBalancer Service automatically kubectl get svc -n ingress-nginx

Basic Ingress Resource

YAML# ingress.yaml — Route traffic by hostname apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: myapp-ingress annotations: nginx.ingress.kubernetes.io/rewrite-target: / spec: ingressClassName: nginx rules: - host: myapp.com http: paths: - path: / pathType: Prefix backend: service: name: frontend-service port: number: 80 - path: /api pathType: Prefix backend: service: name: api-service port: number: 8080 - host: admin.myapp.com http: paths: - path: / pathType: Prefix backend: service: name: admin-service port: number: 3000

05🔒

Ingress Advanced

TLS, Rate Limiting, Auth

TLS/HTTPS with Ingress

YAML# Create TLS secret from certificate kubectl create secret tls myapp-tls \ --cert=fullchain.pem \ --key=privkey.pem # Ingress with TLS apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: myapp-tls-ingress annotations: nginx.ingress.kubernetes.io/ssl-redirect: "true" spec: ingressClassName: nginx tls: - hosts: - myapp.com secretName: myapp-tls rules: - host: myapp.com http: paths: - path: / pathType: Prefix backend: service: name: frontend-service port: number: 80

Useful Annotations

Annotation	What It Does	Example Value
ssl-redirect	Force HTTPS	"true"
proxy-body-size	Max upload size	"50m"
rate-limit	Requests per second	"10"
whitelist-source-range	IP whitelist	"10.0.0.0/8"
proxy-connect-timeout	Backend timeout	"60"

06🕸️

Istio Service Mesh

Introduction

Istio is a service mesh — it manages ALL communication between your microservices inside Kubernetes. It injects a sidecar proxy (Envoy) into every pod that intercepts all network traffic. This gives you: automatic mTLS encryption, traffic routing, retries, circuit breaking, and full observability — WITHOUT changing your application code.

Installation

TERMINAL# Download Istio curl -L https://istio.io/downloadIstio | sh - cd istio-* export PATH=$PWD/bin:$PATH # Install Istio into Kubernetes istioctl install --set profile=demo -y # Enable automatic sidecar injection for a namespace kubectl label namespace default istio-injection=enabled # Verify kubectl get pods -n istio-system # istiod, istio-ingressgateway, istio-egressgateway

Sidecar Proxy

Every pod gets an Envoy sidecar automatically. Your app container talks to the sidecar, the sidecar talks to other sidecars. All traffic is encrypted (mTLS), measured (metrics), and controlled (routing rules). Zero code changes needed.

07🚦

Istio Traffic Management

VirtualService & DestinationRule

Canary Deployment (90/10 Split)

YAML# Send 90% traffic to v1, 10% to v2 apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: myapp spec: hosts: - myapp http: - route: - destination: host: myapp subset: v1 weight: 90 - destination: host: myapp subset: v2 weight: 10 --- apiVersion: networking.istio.io/v1alpha3 kind: DestinationRule metadata: name: myapp spec: host: myapp subsets: - name: v1 labels: version: v1 - name: v2 labels: version: v2

Circuit Breaker

YAML# If backend fails, stop sending traffic (prevent cascade failure) apiVersion: networking.istio.io/v1alpha3 kind: DestinationRule metadata: name: myapp spec: host: myapp trafficPolicy: connectionPool: tcp: maxConnections: 100 http: h2UpgradePolicy: DEFAULT http1MaxPendingRequests: 100 outlierDetection: consecutive5xxErrors: 3 interval: 30s baseEjectionTime: 30s

08📊

Istio Observability

Kiali, Jaeger, Prometheus

Istio gives you full observability out of the box — see which service calls which, how fast they respond, where errors happen, and trace requests across 10 microservices.

📊

Kiali

Service mesh dashboard. Visualizes service-to-service communication. Shows traffic flow, health, and error rates in a graph.

🔍

Jaeger

Distributed tracing. Follow a single request as it travels through multiple microservices. Find which service is slow.

📈

Prometheus + Grafana

Istio exports metrics automatically. Request rate, latency, error rate per service — without any code changes.

TERMINAL# Access Kiali dashboard istioctl dashboard kiali # Access Jaeger tracing istioctl dashboard jaeger # Access Grafana dashboards istioctl dashboard grafana

09🏆

Best Practices

Production Traffic Management

✓Use Ingress for simple HTTP routing (hostname/path based)

✓Use Istio when you have 5+ microservices needing traffic control

✓HAProxy for non-K8s environments or as external load balancer in front of K8s

✓Always enable TLS — use cert-manager for automatic certificate renewal

✓Canary deployments: start with 5% traffic to new version, increase gradually

✓Set circuit breakers to prevent cascade failures across microservices

✓Monitor with Kiali (traffic graph) + Jaeger (tracing) + Grafana (metrics)

✓Use rate limiting on Ingress to prevent abuse and DDoS

✓Health checks on HAProxy backends — never send traffic to dead servers

✓Keep Istio version updated — security patches are critical for service mesh

10💼

Interview Questions

Traffic Management Q&A

What is a load balancer?

Distributes incoming traffic across multiple servers. Prevents any single server from being overwhelmed. If one server dies, traffic goes to healthy ones. Types: L4 (TCP) and L7 (HTTP). HAProxy, Nginx, AWS ALB/NLB.

HAProxy vs Nginx?

HAProxy: purpose-built for load balancing, better performance at scale, TCP and HTTP. Nginx: web server + reverse proxy + load balancer, more versatile. Both excellent. HAProxy for pure LB, Nginx for web serving + LB.

What is K8s Ingress?

L7 load balancer native to Kubernetes. Routes HTTP traffic based on hostname and URL path to Services. Needs an Ingress Controller (Nginx, Traefik, HAProxy). Single entry point for multiple services.

What is a Service Mesh?

Infrastructure layer that manages service-to-service communication. Handles encryption (mTLS), retries, circuit breaking, observability. Istio and Linkerd are popular. Implemented via sidecar proxies in each pod.

Ingress vs LoadBalancer Service?

LoadBalancer: one external IP per Service (expensive, simple). Ingress: one external IP for many Services, routes by hostname/path (cost-effective, flexible). Use Ingress for HTTP, LoadBalancer for TCP/UDP.

What is mTLS?

Mutual TLS — both client and server verify each other certificates. Istio does this automatically between all pods. All inter-service traffic is encrypted without code changes.

Canary deployment?

Release new version to small percentage of traffic (5-10%). Monitor for errors. Gradually increase to 100%. If issues found, route all traffic back to old version. Istio VirtualService controls the traffic split.

Circuit breaker pattern?

If a service starts failing (5xx errors), stop sending traffic to it temporarily. Prevents cascade failure across microservices. Istio DestinationRule with outlierDetection implements this automatically.