Introduction
Understanding the networking concepts in Azure Kubernetes Service (AKS) is crucial for deploying, managing, and securing your containerized applications. AKS provides a robust and flexible networking model that integrates with Azure's networking infrastructure to deliver powerful capabilities.
This document explores the fundamental networking concepts in AKS, including the Kubernetes networking model, different networking implementations, IP addressing schemes, security considerations, and load balancing strategies.
The Kubernetes Networking Model
In Kubernetes, all Pods can communicate with all other Pods on any node by default, without explicit NAT. This is a core principle of the Kubernetes networking model. Each Pod is assigned a unique IP address within the cluster. Each Node is also routable to Pods on that node. To achieve this:
- Kubernetes defines a networking interface for plugins.
- The network must provide the ability for all containers on all nodes to communicate.
- All containers on a node can reach all other containers on all other nodes.
AKS implements this model using various underlying network technologies provided by Azure.
AKS Networking Models
AKS offers two primary networking models, each with different characteristics and use cases:
Kubenet
Kubenet is the default and simpler networking model in AKS. It uses Azure Virtual Network (VNet) subnets for node communication and Kubernetes networking for Pod communication.
- Node Communication: Nodes are deployed into an Azure VNet subnet.
- Pod Communication: Pods are assigned IP addresses from a separate virtual network (overlay network). Traffic between Pods across nodes is routed via Network Address Translation (NAT) on the source node.
- Simplicity: Easier to set up and manage for basic deployments.
- Limitations: Does not assign a direct Azure VNet IP address to each Pod, which can limit certain scenarios like direct Pod-to-Pod communication from outside the cluster without an Ingress controller.
Azure CNI
Azure CNI offers a more advanced networking model that integrates directly with Azure VNets. In this model, each Pod is assigned an IP address directly from the Azure VNet subnet where the AKS nodes are deployed.
- Node and Pod Communication: Both nodes and Pods receive IP addresses from the same Azure VNet subnet.
- Direct VNet Integration: Pods are directly routable within the Azure VNet, allowing for seamless integration with other Azure services and on-premises networks.
- Benefits: Enables scenarios requiring direct VNet IP addresses for Pods, such as connecting to databases or other services that expect direct IP connectivity.
- Requirements: Requires a larger IP address space for the VNet subnet to accommodate all nodes and Pods.
Network Plugins
AKS uses Kubernetes network plugins to provide Pod-to-Pod networking. The most common plugins used in AKS are:
- Kubenet: Uses the `kubenet` network plugin (or no plugin in some contexts) which is the default for the Kubenet networking model.
- Azure CNI: Uses the `azure-cni` plugin for the Azure CNI networking model.
- Calico: An optional, more feature-rich network plugin that can be installed on AKS clusters, offering advanced network policy enforcement and network segmentation.
The choice of network plugin is often tied to the selected networking model (Kubenet vs. Azure CNI).
IP Addressing
Effective IP address management is critical for AKS deployments. AKS supports different IP addressing configurations:
Pod IP Addressing
- Kubenet: Pods get IPs from an overlay network, NATted by the node.
- Azure CNI: Pods get IPs directly from the Azure VNet subnet.
Service IP Addressing
Kubernetes Services are assigned virtual IP addresses within the cluster that act as an abstraction layer for accessing Pods. These Service IPs are routable within the cluster.
For external access to Services, AKS can integrate with Azure Load Balancer or Azure Application Gateway.
Ingress IP Addressing
Ingress controllers manage external access to Services within the cluster, typically via HTTP and HTTPS. Ingress resources can be configured with specific IP addresses (static or dynamic) or hostnames.
AKS supports using Azure Load Balancer or Azure Application Gateway as the underlying Ingress controller, providing external IP addresses for your applications.
Network Security
Securing network traffic is paramount. AKS offers several mechanisms for network security:
Network Policies
Kubernetes Network Policies are a standard Kubernetes resource that allows you to specify how Pods are allowed to communicate with each other and with other network endpoints. AKS supports Network Policies via plugins like Calico or the native Azure Network Policy provider.
Network Policies enable you to implement fine-grained network segmentation within your cluster, enforcing least-privilege access.
Firewall Rules
Azure Firewall can be integrated with AKS to provide centralized network security. You can define firewall rules to control ingress and egress traffic to and from your AKS cluster, including Pod traffic when using Azure CNI.
AKS Network Security Groups (NSGs)
When using Azure CNI, AKS nodes are associated with Network Security Groups (NSGs). These NSGs allow you to filter network traffic to and from AKS resources in an Azure VNet. AKS manages NSGs for node inbound/outbound traffic, but you can create custom NSGs for further control.
Ingress Controllers
An Ingress controller is a piece of software that acts as a reverse proxy and load balancer for HTTP and HTTPS traffic to your Services. AKS offers:
- Nginx Ingress Controller: A popular, community-driven Ingress controller that can be deployed on AKS.
- Azure Application Gateway Ingress Controller (AGIC): Integrates AKS with Azure Application Gateway, providing advanced L7 load balancing, SSL termination, and web application firewall (WAF) capabilities.
Ingress resources are used to define routing rules that the Ingress controller uses to direct traffic.
Load Balancing
AKS provides several options for load balancing your applications:
- Kubernetes Service of type LoadBalancer: This automatically provisions an Azure Load Balancer for your Service, providing a public IP address for external access.
- Ingress Controllers: As mentioned above, Ingress controllers (like Nginx or AGIC) provide L7 load balancing and routing capabilities.
- Azure Load Balancer: Can be used directly for L4 load balancing.
- Azure Application Gateway: Offers advanced L7 load balancing with features like SSL offloading, URL-based routing, and WAF.
DNS
Kubernetes provides internal DNS resolution for Pods and Services within the cluster using CoreDNS (or Kube-DNS in older versions). For external DNS, you would typically use Azure DNS or another external DNS provider.
When using Azure CNI, Pods can resolve external DNS records directly.
Troubleshooting
Common networking issues in AKS can arise from misconfigurations in IP addressing, Network Policies, NSGs, or Ingress controllers.
Tools like kubectl logs, kubectl describe, kubectl exec, and Azure Network Watcher can be invaluable for diagnosing and resolving networking problems.
Understanding the flow of traffic from external clients to Pods, and between Pods, is key to effective troubleshooting.