Register the cluster’s Prometheus server with the central Grafana

Once you have deployed the support chart, you must also register this cluster as a datasource for the central Grafana dashboard. This will allow you to visualize cluster statistics not only from the cluster-specific Grafana deployment but also from the central dashboard, that aggregates data from all the clusters.

Create a support.secret.values.yaml file¶

Only 2i2c staff and our centralized grafana should be able to access the prometheus data on a cluster from outside the cluster. If you would like to share these credentials with communities, please see User access to Prometheus endpoint The basic auth feature of prometheus is used to restrict this. A enc-support.secret.values.yaml file is used to provide these secret credentials, which we create under the relevant config/clusters/<cluster-name>/ folder. It requires the following configuration:

prometheus:
  server:
    probeHeaders:
      - name: Authorization
        value: Basic <USER-PASSWORD>
  serverFiles:
    web.yml:
      basic_auth_users:
    <USER>: <SALTED>
prometheusAuthSecret:
  username: <USER>
  password: <PASSWORD>

where <USER> : The random username <PASSWORD> : The random password <SALTED> : See Prometheus docs:

import getpass
import bcrypt

password = getpass.getpass("password: ")
hashed_password = bcrypt.hashpw(password.encode("utf-8"), bcrypt.gensalt())
print(hashed_password.decode())  

<USER-PASSWORD>

echo -n "<USER>:<PASSWORD>" | base64

Once you create the file, encrypt it with sops.

export CLUSTER_NAME=<cluster-name>

sops --output config/clusters/$CLUSTER_NAME/enc-support.secret.values.yaml --encrypt config/clusters/$CLUSTER_NAME/support.secret.values.yaml

Update your cluster.yaml file¶

Update the support config in the cluster’s cluster.yaml file to include the encrypted secret file.

support:
  helm_chart_values_files:
    - support.values.yaml
    - enc-support.secret.values.yaml

Then redeploy the support chart.

deployer deploy-support $CLUSTER_NAME

Link the cluster’s Prometheus server to the central Grafana¶

To register the new prometheus with the default central grafana, run the command below.

deployer grafana central-ds add $CLUSTER_NAME