Setting up your local environment to work on this repo#
This tutorial will guide you through all the steps needed to have a fully functional local environment that can perform tasks on our clusters and hubs using the tooling in this repo.
Step 1: Install required tools#
We’ll need a bunch of different tools that are focused around interacting with kubernetes and various cloud providers.
Kubernetes access tools#
We interact with kubernetes a lot, and these are the primary tools we use to interact with them:
On a Mac, you can install these easily with
brew install helm kubectl
On other operating systems, the documentation links above should help you find ways of installing them.
Secret decryption tools#
The wonderful sops tool is used to encrypt and keep secrets in our repository.
On a Mac, you can install this easily with
brew install sops
You can download releases for other platforms from the sops github releases page
Cloud provider tools#
We primarily interact with Google Cloud, Amazon Web Services (AWS) and Azure. All these providers provide decent command line tools that we need to have locally installed.
On a Mac, you can easily install them all with
brew install google-cloud-sdk awscli azure-cli
For other platforms, consult the documentation in the links above to find installation methods.
We use terraform to manage our infrastructure in the cloud. So in order to update existing clusters or add new ones, you’ll need to install this tool.
On a Mac, you can easily install it with
brew tap hashicorp/tap brew install hashicorp/tap/terraform
Checkout this information about terraform for how to configure and use it.
Step 2: Setup the python environment#
Our deployment scripts are all written in python, so you’ll need to have a recent
version of Python 3 installed. There are innumerable ways to install python on your
system, so we will not go into them here. Two quick suggestions are to either use
miniforge if you are already familiar with
conda, or use
brew install python3 on a Mac.
Once Python is installed, you need to create a virtual environment to install the specific
libraries we will use. Again this depends on how you installed python, and whether you
want to use
Once you have a virtual environment setup and activated, install the libraries
pip install -r requirements.txt -r dev-requirements.txt -e .
Now you are all ready to use our deployer scripts! Note if dependencies get added you will to run the installation again.
Remember you need to activate the environment you installed these libraries into each time you use any of our scripts.
Step 3: Setup the git pre-commit hooks#
Install pre-commit pre-commit installation instruction
In the root of the checked out
infrastructure repo run the below to install
the local pre-commit hooks.
pre-commit install --install-hooks
Step 4: Authenticate with Google Cloud to decrypt our secret files#
Permission to decrypt the secret files in this repo is managed via
Google Cloud’s Key Management Service,
regardless of the cloud provider running the cluster you are interested in.
So to decrypt our secrets, you must authenticatie to google cloud via
You need to already have permissions to the
two-eye-two-see project via
2i2c.org Google Account. If you don’t have this, please ask a team member.
gcloud auth login, and when the browser window pops up, login with your
gcloud auth application-default loginand repeat the process again.
Tada, now you’re logged in!
Step 5: Access kubernetes clusters with the
You should have already been given access to the
two-eye-two-see Google Cloud
Project as part of onboarding. If not, you might get errors from
sops about missing permissions
during this step. Please chat with whoever onboarded you to make sure you have the
required cloud access.
Our deployer has a convenient way to authenticate you in order
to interact with a kubernetes cluster from the commandline - the
Look for the cluster you want to access - there is one cluster per directory inside
deployer use-cluster-credentials CLUSTER_NAMEfrom the terminal, and this will authenticate you to the correct kubernetes cluster!
Test that you’ve been correctly authenticated by running
kubectl get node, which should list the nodes in the
kubernetes cluster selected.
use-cluster-credentials subcommand actually creates a new shell and sets environment variables to allow access to the chosen cluster, so make sure to run
D when you are finished.