Register the cluster’s Prometheus server with the central Grafana

Once you have deployed the support chart, you must also register this cluster as a datasource for the central Grafana dashboard. This will allow you to visualize cluster statistics not only from the cluster-specific Grafana deployment but also from the central dashboard, that aggregates data from all the clusters.

Create a support.secret.values.yaml file¶

Only 2i2c staff and our centralized grafana should be able to access the prometheus data on a cluster from outside the cluster. If you would like to provision extra credentials for users, please see User access to Prometheus endpoint The basic auth feature of nginx-ingress is used to restrict this. A enc-support.secret.values.yaml file is used to provide these secret credentials, which we create under the relevant config/clusters/<cluster-name>/ folder. It requires the following configuration:

prometheus:
  server:
    probeHeaders:
      - name: Authorization
        value: Basic <USER-PASSWORD>
  serverFiles:
    web.yml:
      basic_auth_users:
    <USER>: <SALTED>
prometheusAuthSecret:
  username: <USER>
  password: <PASSWORD>

where <USER> : The random username <PASSWORD> : The random password <SALTED> : See Prometheus docs:

import getpass
import bcrypt

password = getpass.getpass("password: ")
hashed_password = bcrypt.hashpw(password.encode("utf-8"), bcrypt.gensalt())
print(hashed_password.decode())  

<USER-PASSWORD>

echo -n "<USER>:<PASSWORD>" | base64

Once you create the file, encrypt it with sops.

export CLUSTER_NAME=<cluster-name>

sops --output config/clusters/$CLUSTER_NAME/enc-support.secret.values.yaml --encrypt config/clusters/$CLUSTER_NAME/support.secret.values.yaml

Update your cluster.yaml file¶

Update the support config in the cluster’s cluster.yaml file to include the encrypted secret file.

support:
  helm_chart_values_files:
    - support.values.yaml
    - enc-support.secret.values.yaml

Then redeploy the support chart.

deployer deploy-support $CLUSTER_NAME

Link the cluster’s Prometheus server to the central Grafana¶

To register the new prometheus with the default central grafana, run the command below.

deployer grafana central-ds add $CLUSTER_NAME