Documentation | AWS Multi-ENI Controller for Kubernetes

Installation

Prerequisites

Kubernetes cluster running on AWS (e.g., EKS)
kubectl configured to access your cluster
Helm 3.0+ (for Helm installation)
IAM permissions for EC2 ENI operations

Install with Helm (Recommended)

# Install the latest version
helm install aws-multi-eni oci://ghcr.io/johnlam90/charts/aws-multi-eni-controller --version 1.3.5 \
  --namespace eni-controller-system --create-namespace

# With custom values
helm install aws-multi-eni oci://ghcr.io/johnlam90/charts/aws-multi-eni-controller --version 1.3.5 \
  --namespace eni-controller-system --create-namespace \
  --set awsRegion=us-east-1 \
  --set nodeSelector.ng=multi-eni

Configuration

Basic NodeENI Resource

apiVersion: networking.k8s.aws/v1alpha1
kind: NodeENI
metadata:
  name: multus-eni-config
spec:
  nodeSelector:
    ng: multi-eni
  subnetID: subnet-0f59b4f14737be9ad  # Use your subnet ID
  securityGroupIDs:
  - sg-05da196f3314d4af8  # Use your security group ID
  deviceIndex: 2
  mtu: 9001  # Optional: Set MTU for jumbo frames
  deleteOnTermination: true
  description: "Multus ENI for secondary network interfaces"

Using Subnet Names Instead of IDs

apiVersion: networking.k8s.aws/v1alpha1
kind: NodeENI
metadata:
  name: multus-eni-subnet-name
spec:
  nodeSelector:
    ng: multi-eni
  subnetName: my-subnet-name  # Subnet with this Name tag will be used
  securityGroupIDs:
  - sg-05da196f3314d4af8
  deviceIndex: 2

Controller Concurrency Configuration

# In Helm values.yaml
controller:
  maxConcurrentReconciles: 10  # Default: 5
  maxConcurrentENICleanup: 5   # Default: 3

Usage Examples

Label Your Nodes

kubectl label node your-node-name ng=multi-eni

Verify ENI Creation

kubectl get nodeeni multus-eni-config -o yaml

Check Controller Logs

kubectl logs -n eni-controller-system deployment/eni-controller

Check ENI Manager Logs

kubectl logs -n eni-controller-system daemonset/eni-manager

Integration with Multus CNI

The AWS Multi-ENI Controller was primarily built to simplify the use of Multus CNI with AWS EC2 instances. Multus CNI enables attaching multiple network interfaces to Kubernetes pods, but it requires those interfaces to exist on the node first.

Example NetworkAttachmentDefinition for Multus

apiVersion: k8s.cni.cncf.io/v1
kind: NetworkAttachmentDefinition
metadata:
  name: secondary-network
spec:
  config: '{
    "cniVersion": "0.3.1",
    "type": "ipvlan",
    "master": "eth2",
    "mode": "l2",
    "ipam": {
      "type": "host-local",
      "subnet": "192.168.1.0/24",
      "rangeStart": "192.168.1.200",
      "rangeEnd": "192.168.1.250"
    }
  }'

Example DPDK NetworkAttachmentDefinition

apiVersion: k8s.cni.cncf.io/v1
kind: NetworkAttachmentDefinition
metadata:
  name: dpdk-network
spec:
  config: '{
    "cniVersion": "0.3.1",
    "type": "host-device",
    "device": "0000:00:06.0",
    "vlan": 1000,
    "ipam": {
      "type": "host-local",
      "subnet": "192.168.1.0/24",
      "rangeStart": "192.168.1.200",
      "rangeEnd": "192.168.1.250"
    }
  }'

Troubleshooting

Common Issues

ENI not being created:
- Check controller logs: kubectl logs -n eni-controller-system deployment/eni-controller
- Verify node labels: kubectl get nodes --show-labels | grep multi-eni
- Check AWS permissions for ENI creation
ENI not being deleted:
- Check finalizers on NodeENI: kubectl get nodeeni -o yaml
- Verify AWS permissions for ENI deletion
Interface not coming up:
- Check ENI Manager logs: kubectl logs -n eni-controller-system daemonset/eni-manager
- Verify ENI Manager is running on the node
MTU not being applied:
- Ensure MTU is set in the NodeENI resource
- Check ENI Manager logs for MTU configuration issues

Using as a Library

The AWS Multi-ENI Controller can also be used as a Go library for programmatic ENI management:

go get github.com/johnlam90/aws-multi-eni-controller

Basic Example

package main

import (
    "context"
    "log"
    "time"

    "github.com/go-logr/logr"
    "github.com/go-logr/zapr"
    "github.com/johnlam90/aws-multi-eni-controller/pkg/lib"
    "go.uber.org/zap"
)

func main() {
    // Create a logger
    zapLog, _ := zap.NewDevelopment()
    logger := zapr.NewLogger(zapLog)

    // Create a context with timeout
    ctx, cancel := context.WithTimeout(context.Background(), 5*time.Minute)
    defer cancel()

    // Create an ENI manager
    eniManager, err := lib.NewENIManager(ctx, "us-east-1", logger)
    if err != nil {
        log.Fatalf("Failed to create ENI manager: %v", err)
    }

    // Create an ENI
    options := lib.ENIOptions{
        SubnetID:           "subnet-12345678",
        SecurityGroupIDs:   []string{"sg-12345678"},
        Description:        "Example ENI",
        DeviceIndex:        1,
        DeleteOnTermination: true,
    }

    eniID, err := eniManager.CreateENI(ctx, options)
    if err != nil {
        log.Fatalf("Failed to create ENI: %v", err)
    }

    log.Printf("Created ENI: %s", eniID)
}

Documentation Contents