Kubectl logging

This page describes how to look at various logs by using some deployer commands that wrap the most common kubectl commands or by using kubectl directly.

Look at logs via deployer sub-commands¶

There are some deployer debug sub-commands that wrap up the most relevant kubectl logs arguments that allow conveniently checking logs with only one command.

Look at hub component logs¶

The JupyterHub component’s logs can be fetched with the deployer debug component-logs command, ran for each hub component.

These commands are standalone and don’t require running deployer use-cluster-credentials before.

Hub pod logs¶

deployer debug component-logs $CLUSTER_NAME $HUB_NAME hub

Proxy pod logs¶

deployer debug component-logs $CLUSTER_NAME $HUB_NAME proxy

Traefik pod logs¶

deployer debug component-logs $CLUSTER_NAME $HUB_NAME traefik

Look at dask-gateway logs¶

Display the logs from the dask-gateway’s most important component pods.

Dask-gateway-api pod logs¶

deployer debug component-logs $CLUSTER_NAME $HUB_NAME dask-gateway-api

Dask-gateway-controller pod logs¶

deployer debug component-logs $CLUSTER_NAME $HUB_NAME dask-gateway-controller

Look at a specific user’s logs¶

Display logs from the notebook pod of a given user with the following command:

deployer debug user-logs  $CLUSTER_NAME $HUB_NAME <username>

Note that you don’t need the escaped username, with this command.

Look at logs via kubectl¶

Pre-requisites¶

Get the name of the cluster you want to debug and export its name as env vars. Then use the deployer to gain kubectl access into this specific cluster.

Example:

export CLUSTER_NAME=2i2c;
deployer use-cluster-credentials $CLUSTER_NAME

Kubernetes autoscaler logs¶

You can find scale up or scale down events by looking for decision events

kubectl describe -n kube-system configmap cluster-autoscaler-status

Kubernetes node events and status¶

1. Running nodes and their status

   kubectl get nodes

2. Get a node’s events from the past 1h

   kubectl get events --field-selector involvedObject.kind=Node --field-selector involvedObject.name=<some-node-name>

3. Describe a node and any related events

   kubectl describe node <some-node-name> --show-events=true

Kubernetes pod events and status¶

1. Running pods in a namespace and their status

   kubectl get pods -n <namespace>

2. Running pods in all namespaces of the cluster and their status

   kubectl get pods --all-namespaces

3. Get a pod’s events from the past 1h

   kubectl get events --field-selector involvedObject.kind=Pod --field-selector involvedObject.name=<some-pod-name>

4. Describe a pod and any related events

   kubectl describe pod <some-pod-name> --show-events=true

Kubernetes pod logs¶

You can access any pod’s logs by using the kubectl logs commands. Below are some of the most common debugging commands.

1. Print the logs of a pod

   kubectl logs <pod_name> --namespace <pod_namespace>

2. Print the logs for a container in a pod

   kubectl logs -c <container_name> <pod_name> --namespace <pod_namespace>

3. View the logs for a previously failed pod

   kubectl logs --previous <pod_name> --namespace <pod_namespace>

4. View the logs for all containers in a pod

   kubectl logs <pod_name> --all-containers --namespace <pod_namespace>