Kubelet Rotate Certificates Not Set To False

More Info:

The kubelet —rotate-certificates argument should not be false so that client certificates are automatically rotated before expiry, avoiding authentication outages.

Risk Level

High

Address

Security

Compliance Standards

• CIS AKS

Triage and Remediation

Remediation

Remediation

Manual Steps

1. On every worker node, inspect current kubelet process arguments to see if --rotate-certificates is explicitly set to false:

   ps -fC kubelet
   

2. On every worker node, edit the kubelet configuration file and ensure certificate rotation is enabled:

   sudo sed -i 's/^[[:space:]]*rotateCertificates:[[:space:]]*false/rotateCertificates: true/' /var/lib/kubelet/config.yaml
   

   Then open the file to confirm or add the setting if missing:

   sudo vi /var/lib/kubelet/config.yaml
   

   Make sure it contains:

   rotateCertificates: true
   

3. On every worker node, check for a systemd drop-in that might override this setting and remove any --rotate-certificates=false argument:

   sudo grep -R -- '--rotate-certificates' /etc/systemd/system/kubelet.service.d/ || echo "no override found"
   sudo vi /etc/systemd/system/kubelet.service.d/10-kubelet-args.conf
   

   In the 10-kubelet-args.conf file, remove --rotate-certificates=false if present, or change it to:

   --rotate-certificates=true
   

4. If your environment also uses /etc/kubernetes/kubelet/kubelet-config.json (per the benchmark guidance), ensure it has rotation enabled:

   if [ -f /etc/kubernetes/kubelet/kubelet-config.json ]; then
     sudo sed -i 's/"rotateCertificates"[[:space:]]*:[[:space:]]*false/"rotateCertificates": true/' /etc/kubernetes/kubelet/kubelet-config.json
     sudo vi /etc/kubernetes/kubelet/kubelet-config.json
   fi
   

   Confirm it contains:

   "rotateCertificates": true
   

5. On every worker node, reload systemd and restart kubelet (this will restart the kubelet and may briefly impact node status):

   sudo systemctl daemon-reload
   sudo systemctl restart kubelet.service
   sudo systemctl status kubelet -l
   

6. Verification on every worker node: confirm kubelet is running without --rotate-certificates=false:

   ps -fC kubelet
   

   Ensure there is no --rotate-certificates=false in the command line, and the configuration files still show rotateCertificates: true / "rotateCertificates": true.

Using kubectl

kubectl cannot change kubelet process flags or its config file on the node. This finding must be fixed directly on every worker node via host-level configuration (for example /etc/kubernetes/kubelet/kubelet-config.json, /etc/systemd/system/kubelet.service.d/10-kubelet-args.conf, or the kubelet arguments); see the Manual Steps section for how to do this.

Automation

#!/usr/bin/env bash
#
# Fix CIS AKS 3.2.8: Ensure kubelet rotate certificates is not set to false
# Targets: every worker node (run on each worker node, as root)
#
# This script:
#   - Ensures rotateCertificates: true in the kubelet config file
#   - Removes/neutralizes any --rotate-certificates=false flags from systemd drop-ins
#   - Restarts kubelet
#   - Verifies that kubelet is running without --rotate-certificates=false

set -euo pipefail

# CONFIG PATHS (adapt to your environment if needed)
KUBELET_CONFIG_FILE="/var/lib/kubelet/config.yaml"
ALT_KUBELET_CONFIG_FILE="/etc/kubernetes/kubelet/kubelet-config.json"
SYSTEMD_DIR="/etc/systemd/system/kubelet.service.d"

# Require root
if [[ "$(id -u)" -ne 0 ]]; then
  echo "ERROR: Run this script as root on each worker node."
  exit 1
fi

echo "==> Configuring kubelet certificate rotation on node: $(hostname)"

############################################
# 1. Ensure rotateCertificates: true in kubelet config
############################################

ensure_rotate_certificates_yaml() {
  local file="$1"
  echo "Processing kubelet YAML config: $file"

  if [[ ! -f "$file" ]]; then
    echo "  - File not found: $file, skipping YAML update."
    return 0
  fi

  # Create backup once per run
  if [[ ! -f "${file}.cis-backup" ]]; then
    cp -p "$file" "${file}.cis-backup"
    echo "  - Backup created at ${file}.cis-backup"
  fi

  # If rotateCertificates key exists, set to true; otherwise append at end (root level)
  if grep -Eq '^[[:space:]]*rotateCertificates:' "$file"; then
    sed -i -E 's/^([[:space:]]*rotateCertificates:).*/\1 true/' "$file"
    echo "  - Updated existing rotateCertificates to true"
  else
    echo "" >> "$file"
    echo "rotateCertificates: true" >> "$file"
    echo "  - Added rotateCertificates: true"
  fi
}

ensure_rotate_certificates_json() {
  local file="$1"
  echo "Processing kubelet JSON config: $file"

  if [[ ! -f "$file" ]]; then
    echo "  - File not found: $file, skipping JSON update."
    return 0
  fi

  # Create backup once per run
  if [[ ! -f "${file}.cis-backup" ]]; then
    cp -p "$file" "${file}.cis-backup"
    echo "  - Backup created at ${file}.cis-backup"
  fi

  # If rotateCertificates already present, set to true; otherwise insert before final '}'
  if grep -q '"rotateCertificates"' "$file"; then
    # Replace the value while preserving surrounding JSON syntax
    perl -pi -e 's/"rotateCertificates"\s*:\s*(true|false)/"rotateCertificates": true/' "$file"
    echo "  - Updated existing \"rotateCertificates\" to true"
  else
    # Insert `"rotateCertificates": true` before the last closing brace.
    # Handles cases with or without trailing newline.
    perl -0pi -e '
      s/
        \}\s*$
      /
        "rotateCertificates": true\n}
      /x' "$file"
    echo "  - Added \"rotateCertificates\": true"
  fi
}

if [[ -f "$KUBELET_CONFIG_FILE" ]]; then
  ensure_rotate_certificates_yaml "$KUBELET_CONFIG_FILE"
elif [[ -f "$ALT_KUBELET_CONFIG_FILE" ]]; then
  ensure_rotate_certificates_json "$ALT_KUBELET_CONFIG_FILE"
else
  echo "WARNING: No kubelet config file found at $KUBELET_CONFIG_FILE or $ALT_KUBELET_CONFIG_FILE."
  echo "         You may need to adjust paths in this script for your environment."
fi

############################################
# 2. Ensure no systemd drop-in sets --rotate-certificates=false
############################################

echo "==> Checking systemd kubelet drop-ins for --rotate-certificates=false"

if [[ -d "$SYSTEMD_DIR" ]]; then
  shopt -s nullglob
  for dropin in "$SYSTEMD_DIR"/*.conf; do
    echo "  - Inspecting $dropin"
    if grep -q -- '--rotate-certificates=false' "$dropin"; then
      if [[ ! -f "${dropin}.cis-backup" ]]; then
        cp -p "$dropin" "${dropin}.cis-backup"
        echo "    * Backup created at ${dropin}.cis-backup"
      fi
      # Remove or flip any occurrences of --rotate-certificates=false
      sed -i 's/--rotate-certificates=false/--rotate-certificates=true/g' "$dropin"
      echo "    * Replaced --rotate-certificates=false with --rotate-certificates=true"
    else
      echo "    * No --rotate-certificates=false found"
    fi
  done
  shopt -u nullglob
else
  echo "  - Systemd directory $SYSTEMD_DIR not found; skipping drop-in check."
fi

############################################
# 3. Reload systemd and restart kubelet
############################################

echo "==> Restarting kubelet to apply changes"
systemctl daemon-reload

if systemctl is-enabled --quiet kubelet 2>/dev/null || systemctl status kubelet >/dev/null 2>&1; then
  systemctl restart kubelet.service
  sleep 5
  systemctl status kubelet -l --no-pager || {
    echo "ERROR: kubelet is not healthy after restart. Check logs with: journalctl -u kubelet -xe"
    exit 1
  }
  echo "  - kubelet restarted successfully"
else
  echo "WARNING: kubelet service not found or not managed by systemd on this node."
fi

############################################
# 4. Verification (adapted from audit)
############################################

echo "==> Verification: checking kubelet process flags and config"

if /bin/ps -fC kubelet >/dev/null 2>&1; then
  /bin/ps -fC kubelet || true
  if /bin/ps -fC kubelet | grep -q -- '--rotate-certificates=false'; then
    echo "FAIL: kubelet still running with --rotate-certificates=false"
    exit 1
  fi
else
  echo "ERROR: kubelet process not found after restart."
  exit 1
fi

# Confirm config setting where applicable
if [[ -f "$KUBELET_CONFIG_FILE" ]]; then
  echo "==> Checking $KUBELET_CONFIG_FILE for rotateCertificates: true"
  if ! grep -Eq '^[[:space:]]*rotateCertificates:[[:space:]]*true' "$KUBELET_CONFIG_FILE"; then
    echo "FAIL: rotateCertificates: true not found in $KUBELET_CONFIG_FILE"
    exit 1
  fi
elif [[ -f "$ALT_KUBELET_CONFIG_FILE" ]]; then
  echo "==> Checking $ALT_KUBELET_CONFIG_FILE for \"rotateCertificates\": true"
  if ! grep -q '"rotateCertificates"[[:space:]]*:[[:space:]]*true' "$ALT_KUBELET_CONFIG_FILE"; then
    echo "FAIL: \"rotateCertificates\": true not found in $ALT_KUBELET_CONFIG_FILE"
    exit 1
  fi
fi

echo "SUCCESS: kubelet certificate rotation is enabled and --rotate-certificates is not set to false."