Best Practices for Securing Kubelet in Kubernetes

Best Practices for Securing Kubelet in Kubernetes

NOTE: Whenever you setup a kubernetes environment using kubeadm, the worker node does not come up with the kubelet installed. Hence you should install it.

The Kubelet, a critical component of Kubernetes, runs on every node and is responsible for managing pods and communicating with the control plane. By default, the Kubelet exposes two ports:

10250: Kubelet API (authenticated access)

10255: Read-only port (unauthenticated access, deprecated in recent Kubernetes versions)

If the Kubelet service is not properly secured, it can expose sensitive information or even allow unauthorized access to critical resources.

Without securing the Kubelet service, an attacker could query the Kubelet API and retrieve pod information. For instance:

curl -sk https://<node-ip>:10250/pods
// curl -sk https://localhost:10250/pods

This command might return details of all running pods on the node in JSON format, exposing:

  • Pod names and namespaces

  • Resource configurations

  • Secrets or sensitive data (depending on the setup)

Following steps can be followed for securing kubelet.

Kubelet configuration file location

ps aux | grep kubelet

Search for - - config = …
That is the path for you kubelet configuration file.

Anonymous Authentication in Kubernetes Kubelet

Kubernetes allows you to configure authentication for the Kubelet, which is the agent running on each node responsible for managing pods and communicating with the control plane. One of the options for authentication is anonymous access, controlled by the authentication.anonymous.enabled setting in the Kubelet configuration file.

What Does authentication.anonymous.enabled: true Do?

When authentication.anonymous.enabled is set to true, it means that the Kubelet will accept requests from clients without requiring any form of authentication. In other words:

  • Anyone can send requests to the Kubelet without presenting credentials.

  • The requests will be treated as coming from an anonymous user with very limited permissions by default.

This type of authentication should be turned off and can be done from the config file by setting

enabled: false

vi <kubelet-config-file-path>

Restart the kubelet after updation. systemctl restart kubelet.service

Configuring Kubelet Authorization

In Kubernetes, the authorization setting determines how requests to the Kubelet are evaluated to decide whether they should be allowed or denied. The setting authorization.mode can significantly impact the security of your Kubernetes cluster. Mostly by default, the authorization mode is set to AlwaysAllow.

What Does authorization.mode: AlwaysAllow Do?

When the Kubelet authorization mode is set to AlwaysAllow, every request received by the Kubelet is automatically authorized. In other words:

  • No checks are performed to verify if the requester has the necessary permissions.

  • Every authenticated (or even anonymous, if authentication.anonymous.enabled: true is set) request is treated as valid and allowed to proceed.

This mode is inherently insecure, especially in production environments, as it grants unrestricted access to the Kubelet’s API. Malicious actors or misconfigured clients could potentially exploit this to access or disrupt your workloads.

Why Should You Use Webhook Mode?

The Webhook authorization mode is a more secure and robust alternative to AlwaysAllow. Here’s why:

  1. Centralized Authorization with RBAC:

    • In Webhook mode, the Kubelet delegates authorization decisions to the Kubernetes API server.

    • The API server evaluates requests against Role-Based Access Control (RBAC) policies configured in the cluster.

    • This ensures that only users or services with the appropriate roles and permissions can perform specific actions.

  2. Granular Access Control:

    • With RBAC, you can define precise rules for who can access the Kubelet API and what actions they can perform. For example:

      • Allowing only cluster administrators to retrieve pod logs.

      • Restricting exec or attach commands to specific namespaces or users.

  3. Enhanced Security:

    • By requiring every request to pass RBAC checks, Webhook mode minimizes the risk of unauthorized actions, data exposure, or system compromise.

    • This aligns with security best practices for production-grade clusters.

  4. Auditing and Monitoring:

    • RBAC policies provide clear logging and audit trails for all authorization decisions.

    • This helps in troubleshooting and compliance by tracking who accessed the Kubelet and what actions were performed.

Here is how you can configure Webhook as authorization mode.

vi <kubelet-config-file-path>

Set the mode to Webhook and restart the kubelet service.

systemctl restart kubelet.service

controlplane $ curl -sk https://localhost:10250/pods Unauthorized.

This is what you get after this.

Disable the Read-Only Port (10255):

The read-only port (10255) provides unauthenticated access to Kubelet metrics and pod details. While this port has been deprecated in recent Kubernetes versions, ensure it is explicitly disabled in your Kubelet configuration:

Set the readOnlyPort: 0 and restart the kubelet service.

Note:

If only authorization mode is set to Webhook and anonymous authentication is still on.
The output to the
curl command would be

Forbidden (user=system:anonymous, verb=get, resource=nodes, subresource=proxy)

Securing the Kubelet is important to maintain the integrity and confidentiality of your Kubernetes cluster. This is achieved by disabling unauthorized access, enabling secure communication, and applying fine-grained access control to minimize the risk of exploitation and ensure a robust security posture.