AWS Multi-ENI Controller Architecture

Detailed architecture diagrams and component explanations

System Architecture

The AWS Multi-ENI Controller follows the Kubernetes operator pattern with two main components: the NodeENI Controller and the ENI Manager DaemonSet. This diagram illustrates the high-level architecture and component interactions.

graph TB subgraph K8s[Kubernetes Cluster] A[NodeENI CRD] B[NodeENI Resources] C[NodeENI Controller] D[ENI Manager DaemonSet] E[Kubernetes Nodes] end subgraph AWS[AWS Cloud] F[EC2 API] G[Elastic Network Interfaces] end A -->|Defines| B B -->|Watched by| C C -->|API Calls| F C -->|Creates/Manages| G D -->|Configures interfaces on| E G -->|Attached to| E

Component Descriptions:

  • NodeENI CRD: The Custom Resource Definition that defines the NodeENI resource type in Kubernetes
  • NodeENI Resources: Instances of the NodeENI custom resource that specify which nodes should get ENIs and how they should be configured
  • NodeENI Controller: The main controller that watches NodeENI resources and creates/manages ENIs in AWS
  • ENI Manager DaemonSet: Runs on nodes with matching labels and configures the network interfaces
  • Kubernetes Nodes: The worker nodes in the Kubernetes cluster that receive the ENIs
  • EC2 API: The AWS EC2 API that the controller uses to create, attach, and manage ENIs
  • Elastic Network Interfaces: The AWS ENIs that are created and attached to the nodes

ENI Lifecycle

This diagram illustrates the complete lifecycle of an ENI managed by the AWS Multi-ENI Controller, from creation to cleanup.

graph TD A[Start] --> B[NodeENI Created] B --> C{Node Matches Selector?} C -->|Yes| D[Create ENI] C -->|No| E[Wait for Matching Node] E --> C D --> F[Attach ENI to Node] F --> G[Update NodeENI Status] G --> H[Monitor Node and NodeENI] H --> I{Change Detected?} I -->|No| H I -->|Yes| J{Node Still Matches?} J -->|Yes| G J -->|No| K[Detach ENI] L[NodeENI Deleted] --> M[Finalizer Triggered] M --> K K --> N[Delete ENI] N --> O[Update Status or Remove Finalizer] O --> P[End]

Lifecycle Stages:

  1. Creation: When a NodeENI resource is created and a node matches its selector, the controller creates an ENI in the specified subnet
  2. Attachment: The controller attaches the ENI to the node at the specified device index
  3. Configuration: The ENI Manager DaemonSet configures the network interface on the node
  4. Monitoring: The controller continuously monitors the NodeENI resource and matching nodes
  5. Detachment: When a node no longer matches or the NodeENI is deleted, the controller detaches the ENI
  6. Cleanup: The controller deletes the ENI and updates the NodeENI status or removes the finalizer