Introduction

CNI, as in Container Networking Interface for kubernetes, dedicated to provide network solution for Kubernetes containers. There are tons of CNI plugin for kubernetes networking on the market, some of them are opensource projects.(e.g. flannel, calico, cilium) Besides, the CNI officially provides some sample cni demo for end users.

How does kubelet interact with CNI

Implemented by Dockershim

In preview version of Kubernetes(less or equal 1.23), if the container runtime is specified to docker, CNI plugin will be called in dockershim#cni.go inside kubelet.

rt := &libcni.RuntimeConf{
		ContainerID: podSandboxID.ID,
		NetNS:       podNetnsPath,
		IfName:      network.DefaultInterfaceName,
		CacheDir:    plugin.cacheDir,
		Args: [][2]string{
			{"IgnoreUnknown", "1"},
			{"K8S_POD_NAMESPACE", podNs},
			{"K8S_POD_NAME", podName},
			{"K8S_POD_INFRA_CONTAINER_ID", podSandboxID.ID},
		},
	}

dockershim pass 4 args to the CNI plugin: IgnoreUnknown, K8S_POD_NAMESPACE, K8S_POD_NAME and K8S_POD_INFRA_CONTAINER_ID which contain pod name, pod namespace and pod container id.

In the CNI plugin, the args received from kubelet will be placed in skel.CmdArgs.Args

// CmdArgs captures all the arguments passed in to the plugin
// via both env vars and stdin
type CmdArgs struct {
	ContainerID string
	Netns       string
	IfName      string
	Args        string // IgnoreUnknown=1;K8S_POD_NAMESPACE=default;K8S_POD_NAME=pod-1;K8S_POD_INFRA_CONTAINER_ID=xxx
	Path        string
	StdinData   []byte
}

Normally, we will implement a k8s client inside our CNI plugin, so that we can easily get information about the Pods(e.g. pod label, annotation, CRD).

But when a CNI plugin only needs name and namespace of the Pods, which is able to attain from kubelet, it won’t be necessary to introduce an extra k8s client.

Implemented by Containerd

After kubernetes 1.23, dockershim was removed from upstream: Completely remove in-tree dockershim from kubelet #97252

After the removal, CNI plugin will not be directly used by kubelet. It is used by container runtime instead.(e.g. containerd, CRI-O)

Let’s take containerd for example:

• kubelet call containerd by CRI(container runtime interface) to create pod sandbox
• containerd setup network for pod before running it. refer to: setupPodNetwork

The following CNI args will be passed to CNI plugin:

// toCNILabels adds pod metadata into CNI labels.
func toCNILabels(id string, config *runtime.PodSandboxConfig) map[string]string {
	return map[string]string{
		"K8S_POD_NAMESPACE":          config.GetMetadata().GetNamespace(),
		"K8S_POD_NAME":               config.GetMetadata().GetName(),
		"K8S_POD_INFRA_CONTAINER_ID": id,
		"K8S_POD_UID":                config.GetMetadata().GetUid(),
		"IgnoreUnknown":              "1",
	}
}

A simple CNI example

refer to: sriov-cni

• When a container is created, kubelet connects to CNI, CNI executes cmdAdd() to set up container network environment.

// cmdAdd sets up container network environment in creation
func cmdAdd(args *skel.CmdArgs) error {
	
}

In cmdAdd(), the main process:

• allocate IP address and MAC address for container(call IPAM plugin).

• add NIC device into container network namespace.

• config NIC, route and DNS for container.

• When a container is deleted, kubelet connects to CNI, CNI executes cmdDel() to clean up container network environment.

// cmdDel cleans up container network environment in deletion
func cmdDel(args *skel.CmdArgs) error {

}

In cmdDel(), the main process:

• release IP address and MAC address for container(call IPAM plugin).
• clean up NIC configuration.

• ← Previous Post
• Next Post →