Ensure Streaming Connection Idle Timeout Argument Is Not Set 0

More Info:

Do not disable timeouts on streaming connections.

Risk Level

Medium

Address

Security

Compliance Standards

• CIS Kubernetes

Triage and Remediation

Remediation

Remediation

Manual Steps

1. On every worker node, check whether kubelet uses a config file and/or flags:

   ps -fC kubelet
   

   Review the command line for --config= and --streaming-connection-idle-timeout=.

2. If /var/lib/kubelet/config.yaml is in use, edit it to ensure the timeout is non‑zero:

   sudo sed -i 's/^streamingConnectionIdleTimeout: 0$/streamingConnectionIdleTimeout: 5m/' /var/lib/kubelet/config.yaml
   

   If the key does not exist or is set differently, open the file and add or adjust it under the top-level config:

   sudo vi /var/lib/kubelet/config.yaml
   

   and set:

   streamingConnectionIdleTimeout: 5m
   

3. If kubelet uses command-line flags via systemd, edit the drop-in file on each worker node:

   sudo vi /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
   

   In the Environment= line that defines kubelet args, either:

   • Replace any existing --streaming-connection-idle-timeout=0 with:
     --streaming-connection-idle-timeout=5m
     
   • Or, if it is missing, append:
     --streaming-connection-idle-timeout=5m
     

   to the list of arguments.

4. Reload systemd and restart kubelet on each worker node so changes take effect:

   sudo systemctl daemon-reload
   sudo systemctl restart kubelet.service
   

5. Verify kubelet is running with a non‑zero streaming idle timeout on each worker node:

   ps -fC kubelet
   

   Confirm in the output that:

   • --streaming-connection-idle-timeout=5m is present (if using flags), and
   • There is no --streaming-connection-idle-timeout=0.

Using kubectl

kubectl cannot modify kubelet process flags or its config file on worker nodes, so this finding cannot be fixed via the Kubernetes API. To remediate, edit /var/lib/kubelet/config.yaml or the kubelet systemd drop-in on each worker node as described in the Manual Steps section.

Automation

#!/usr/bin/env bash
# Automation: Fix kubelet streamingConnectionIdleTimeout on every worker node
# Usage: run as root on each worker node (or via SSH/Ansible). Safe to re-run.

set -euo pipefail

KUBELET_CONFIG="/var/lib/kubelet/config.yaml"
SYSTEMD_DROPIN="/etc/systemd/system/kubelet.service.d/10-kubeadm.conf"
DESIRED_TIMEOUT="5m"

echo "==> Ensuring kubelet streaming connection idle timeout is not 0"

changed=0

#############################################
# 1) Update Kubelet config file if present  #
#############################################
if [ -f "${KUBELET_CONFIG}" ]; then
  echo "-> Found kubelet config file at ${KUBELET_CONFIG}"

  # Backup once
  if [ ! -f "${KUBELET_CONFIG}.bak" ]; then
    cp "${KUBELET_CONFIG}" "${KUBELET_CONFIG}.bak"
    echo "   Backup created at ${KUBELET_CONFIG}.bak"
  fi

  if grep -q '^[[:space:]]*streamingConnectionIdleTimeout:' "${KUBELET_CONFIG}"; then
    # If explicitly set to 0, change to desired value
    if grep -q '^[[:space:]]*streamingConnectionIdleTimeout:[[:space:]]*0[[:space:]]*$' "${KUBELET_CONFIG}"; then
      echo "   Updating streamingConnectionIdleTimeout from 0 to ${DESIRED_TIMEOUT}"
      sed -i "s/^\([[:space:]]*streamingConnectionIdleTimeout:\)[[:space:]]*0[[:space:]]*$/\1 ${DESIRED_TIMEOUT}/" "${KUBELET_CONFIG}"
      changed=1
    else
      echo "   streamingConnectionIdleTimeout already set and not 0; leaving as-is"
    fi
  else
    # Add the setting under top-level if absent
    echo "   streamingConnectionIdleTimeout not present; adding with value ${DESIRED_TIMEOUT}"
    # Append at end; kubelet accepts it as a top-level field
    printf "\nstreamingConnectionIdleTimeout: %s\n" "${DESIRED_TIMEOUT}" >> "${KUBELET_CONFIG}"
    changed=1
  fi
else
  echo "-> Kubelet config file ${KUBELET_CONFIG} not found; skipping file-based config."
fi

#########################################################
# 2) Update systemd kubelet drop-in if flags are used   #
#########################################################
if [ -f "${SYSTEMD_DROPIN}" ]; then
  echo "-> Found systemd drop-in at ${SYSTEMD_DROPIN}"

  # Backup once
  if [ ! -f "${SYSTEMD_DROPIN}.bak" ]; then
    cp "${SYSTEMD_DROPIN}" "${SYSTEMD_DROPIN}.bak"
    echo "   Backup created at ${SYSTEMD_DROPIN}.bak"
  fi

  # Ensure KUBELET_SYSTEM_PODS_ARGS line exists
  if ! grep -q '^Environment="KUBELET_SYSTEM_PODS_ARGS=' "${SYSTEMD_DROPIN}"; then
    echo "   Adding KUBELET_SYSTEM_PODS_ARGS environment line"
    printf 'Environment="KUBELET_SYSTEM_PODS_ARGS="\n' >> "${SYSTEMD_DROPIN}"
    changed=1
  fi

  # Remove any existing streaming-connection-idle-timeout flag
  if grep -q -- '--streaming-connection-idle-timeout=' "${SYSTEMD_DROPIN}"; then
    echo "   Removing existing --streaming-connection-idle-timeout flag from KUBELET_SYSTEM_PODS_ARGS"
    sed -i 's/--streaming-connection-idle-timeout=[^" ]*//g' "${SYSTEMD_DROPIN}"
    changed=1
  fi

  # Normalize whitespace in KUBELET_SYSTEM_PODS_ARGS
  sed -i 's/^Environment="KUBELET_SYSTEM_PODS_ARGS=\(.*\)"/Environment="KUBELET_SYSTEM_PODS_ARGS=\1"/' "${SYSTEMD_DROPIN}"
  sed -i 's/Environment="KUBELET_SYSTEM_PODS_ARGS=[[:space:]]*/Environment="KUBELET_SYSTEM_PODS_ARGS=/g' "${SYSTEMD_DROPIN}"
  sed -i 's/[[:space:]]\{2,\}/ /g' "${SYSTEMD_DROPIN}"

  # Append desired flag if not already present
  if ! grep -q -- "--streaming-connection-idle-timeout=${DESIRED_TIMEOUT}" "${SYSTEMD_DROPIN}"; then
    echo "   Adding --streaming-connection-idle-timeout=${DESIRED_TIMEOUT} to KUBELET_SYSTEM_PODS_ARGS"
    sed -i "s/^Environment=\"KUBELET_SYSTEM_PODS_ARGS=\(.*\)\"/Environment=\"KUBELET_SYSTEM_PODS_ARGS=\1 --streaming-connection-idle-timeout=${DESIRED_TIMEOUT}\"/" "${SYSTEMD_DROPIN}"
    changed=1
  else
    echo "   Desired --streaming-connection-idle-timeout already present in systemd config"
  fi
else
  echo "-> Systemd drop-in ${SYSTEMD_DROPIN} not found; skipping flag-based config."
fi

##########################################
# 3) Restart kubelet if anything changed #
##########################################
if [ "${changed}" -eq 1 ]; then
  echo "-> Changes detected; reloading systemd and restarting kubelet"
  systemctl daemon-reload
  systemctl restart kubelet.service
  echo "   kubelet restarted (note: this is disruptive for pods on this node)"
else
  echo "-> No changes required; kubelet restart not needed"
fi

##########################################
# 4) Verification (same node)            #
##########################################
echo "==> Verifying kubelet process flags and config"

# Verify process arguments
/bin/ps -fC kubelet || true

echo
echo "-> Checking for streaming-connection-idle-timeout flag in kubelet process:"
/bin/ps -fC kubelet | grep -E -- '--streaming-connection-idle-timeout' || echo "   No streaming-connection-idle-timeout flag visible (may be using config file only)."

# If config file exists, verify field
if [ -f "${KUBELET_CONFIG}" ]; then
  echo
  echo "-> Checking streamingConnectionIdleTimeout in ${KUBELET_CONFIG}:"
  grep -n 'streamingConnectionIdleTimeout' "${KUBELET_CONFIG}" || echo "   streamingConnectionIdleTimeout not explicitly set in config file."
fi

echo
echo "==> Automation complete on this worker node."

Additional Reading:

• https://github.com/kubernetes/kubernetes/pull/18552