Skip to content
Snippets Groups Projects
Commit 8129ecd0 authored by Pierre Smeyers's avatar Pierre Smeyers
Browse files

doc: reorganise chapters

parent 6e6dd777
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
Showing
with 138 additions and 95 deletions
README.md
+ 138
− 95
View file @ 8129ecd0
......@@ -2,14 +2,29 @@
This project implements a generic GitLab CI template for [Helm](https://helm.sh/).
## Overview
## Usage
In order to include this template in your project, add the following to your `gitlab-ci.yml`:
```yaml
include:
- project: 'to-be-continuous/helm'
ref: '3.2.0'
file: '/templates/gitlab-ci-helm.yml'
```
## Understand
This chapter introduces key notions and principle to understand how this template works.
### Managed deployment environments
This template implements continuous delivery/continuous deployment based on [Helm](https://helm.sh/) for projects hosted
on [Kubernetes](https://kubernetes.io) platforms.
It provides several features, usable in different modes (by configuration).
### Review environments
#### Review environments
The template supports **review** environments: those are dynamic and ephemeral environments to deploy your
_ongoing developments_ (a.k.a. _feature_ or _topic_ branches).
......@@ -21,7 +36,7 @@ It is a strict equivalent of GitLab's [Review Apps](https://docs.gitlab.com/ee/c
It also comes with a _cleanup_ job (accessible either from the _environments_ page, or from the pipeline view).
### Integration environment
#### Integration environment
If you're using a Git Workflow with an integration branch (such as [Gitflow](https://www.atlassian.com/git/tutorials/comparing-workflows/gitflow-workflow)),
the template supports an **integration** environment.
......@@ -29,7 +44,7 @@ the template supports an **integration** environment.
When enabled, it deploys the result from upstream build stages to a dedicated environment.
It is only active for your integration branch (`develop` by default).
### Production environments
#### Production environments
Lastly, the template supports 2 environments associated to your production branch (`master` by default):
......@@ -41,51 +56,106 @@ You're free to enable whichever or both, and you can also choose your deployment
* **continuous deployment**: automatic deployment to production (when the upstream pipeline is successful),
* **continuous delivery**: deployment to production can be triggered manually (when the upstream pipeline is successful).
## Usage
<!--
### Supported authentication methods
-->
### Include
### Deployment context variables
In order to include this template in your project, add the following to your `gitlab-ci.yml`:
In order to manage the various deployment environments, this template provides a couple of **dynamic variables**
that you might use in your hook scripts and Helm charts (as [values](https://helm.sh/docs/chart_best_practices/values/)):
```yaml
include:
- project: 'to-be-continuous/helm'
ref: '3.2.0'
file: '/templates/gitlab-ci-helm.yml'
```
| environment variable | template directive | description |
|----------------------|--------------------|-------------|
| `$environment_name` | `{{ .Release.Name }}` | a generated application name to use for the current deployment environment (ex: `myproject-review-fix-bug-12` or `myproject-staging`). This is used as the **Helm release name** in deploy &amp; delete jobs - _details below_ |
| `$environment_type` | `{{ .Values.environment_type }}` | the current deployment environment type (`review`, `integration`, `staging` or `production`) |
| `$hostname` | `{{ .Values.hostname }}` | the environment hostame, if you [specified the environment url statically](#environments-url-management) |
### Global configuration
#### Generated environment name
The Helm template uses some global configuration used throughout all jobs.
The `${environment_name}` variable is generated to designate each deployment environment with a unique and meaningful application name.
By construction, it is suitable for inclusion in DNS, URLs, Kubernetes labels...
It is built from:
| Name | description | default value |
| --------------------- | -------------------------------------- | ----------------- |
| `HELM_CLI_IMAGE` | The Docker image used to run Helm <br/>:warning: **set the version required by your Kubernetes server** | `alpine/helm:latest` |
| `HELM_CHART_DIR` | The folder where the Helm chart is located | `.` _(root project dir)_ |
| `HELM_SCRIPTS_DIR` | The folder where hook scripts are located | `.` _(root project dir)_ |
| `HELM_COMMON_VALUES` | Common values file (used for all environments, overridden by specific per-env values files) | undefined (none) |
| `HELM_ENV_VALUE_NAME` | The environment type variable set to helm | `environment_type` |
| `HELM_HOSTNAME_VALUE_NAME` | The hostname variable set to helm | `hostname` |
* the application _base name_ (defaults to `$CI_PROJECT_NAME` but can be overridden globally and/or per deployment environment - _see configuration variables_)
* GitLab predefined `$CI_ENVIRONMENT_SLUG` variable ([sluggified](https://en.wikipedia.org/wiki/Clean_URL#Slug) name, truncated to 24 characters)
### Charts publishing
The `${environment_name}` variable is then evaluated as:
The template builds a chart package that may be pushed as two distinct packages, depending on a certain _workflow_:
* `<app base name>` for the production environment
* `<app base name>-$CI_ENVIRONMENT_SLUG` for all other deployment environments
* :bulb: `${environment_name}` can also be overriden per environment with the appropriate configuration variable
1. **snapshot**: the chart is first packaged and then pushed to some registry as
the **snapshot** image. It can be seen as the raw result of the build, but still **untested and unreliable**.
2. **release**: once the snapshot chart has been thoroughly tested (both by `package-test` stage jobs and/or `acceptance`
stage jobs after being deployed to some server), then the chart is pushed one more time as the **release** chart.
This second push can be seen as the **promotion** of the snapshot chart being now **tested and reliable**.
Examples (with an application's base name `myapp`):
Common variables for `helm-package` and `helm-pusblish`:
| `$environment_type` | Branch | `$CI_ENVIRONMENT_SLUG` | `$environment_name` |
|---------------------|---------------|-------------------------|---------------------|
| `review` | `feat/blabla` | `review-feat-bla-xmuzs6`| `myapp-review-feat-bla-xmuzs6` |
| `integration` | `develop` | `integration` | `myapp-integration` |
| `staging` | `main` | `staging` | `myapp-staging` |
| `production` | `main` | `production` | `myapp` |
| Name | description | default value |
| --------------------------------- | -------------------------------------------- | ----------------------- |
| `HELM_REPO_PUBLISH_METHOD` | HTTP method to use to push the package | `POST` |
| :lock: `HELM_REPO_USER` | Helm registry username | `$CI_REGISTRY_USER` |
| :lock: `HELM_REPO_PASSWORD` | Helm registry password | `$CI_REGISTRY_PASSWORD` |
### Deployment and cleanup scripts
The Helm template requires you to provide a Helm chart (either in the project or located in an external repository) to deploy and delete the application.
The environment deployment is processed as follows:
1. _optionally_ executes the `helm-pre-deploy.sh` script in your project to perform specific environment pre-initialization (for e.g. create required services),
2. [`helm upgrade`](https://helm.sh/docs/helm/helm_upgrade/) the chart with the configured parameters, using [`$environment_name`](#using-variables) as release name,
3. _optionally_ executes the `helm-post-deploy.sh` script in your project to perform specific environment post-initialization stuff,
The environment deletion is processed as follows:
1. _optionally_ executes the `helm-pre-delete.sh` script in your project to perform specific environment pre-cleanup stuff,
2. [`helm uninstall`](https://helm.sh/docs/helm/helm_uninstall/), using [`$environment_name`](#using-variables) as release name,
3. _optionally_ executes the `helm-post-delete.sh` script in your project to perform specific environment post-cleanup (for e.g. delete bound services).
:warning: each of the above hook scripts needs to be executable, you can add flag execution with: `git update-index --chmod=+x helm-pre-cleanup.sh`
### Environments URL management
The Helm template supports two ways of providing your environments url:
* a **static way**: when the environments url can be determined in advance, probably because you're exposing your routes through a DNS you manage,
* a [**dynamic way**](https://docs.gitlab.com/ee/ci/environments/#set-dynamic-environment-urls-after-a-job-finishes): when the url cannot be known before the
deployment job is executed.
The **static way** can be implemented simply by setting the appropriate configuration variable(s) depending on the environment (see environments configuration chapters):
* `$HELM_ENVIRONMENT_URL` to define a default url pattern for all your envs,
* `$HELM_REVIEW_ENVIRONMENT_URL`, `$HELM_INTEG_ENVIRONMENT_URL`, `$HELM_STAGING_ENVIRONMENT_URL` and `$HELM_PROD_ENVIRONMENT_URL` to override the default.
> :information_source: Each of those variables support a **late variable expansion mechanism** with the `%{somevar}` syntax,
> allowing you to use any dynamically evaluated variables such as `${environment_name}`.
>
> Example:
>
> ```yaml
> variables:
> HELM_BASE_APP_NAME: "wonderapp"
> # global url for all environments
> HELM_ENVIRONMENT_URL: "https://%{environment_name}.nonprod.acme.domain"
> # override for prod (late expansion of $HELM_BASE_APP_NAME not needed here)
> HELM_PROD_ENVIRONMENT_URL: "https://$HELM_BASE_APP_NAME.acme.domain"
> # override for review (using separate resource paths)
> HELM_REVIEW_ENVIRONMENT_URL: "https://wonderapp-review.nonprod.acme.domain/%{environment_name}"
> ```
To implement the **dynamic way**, your deployment script shall simply generate a `environment_url.txt` file in the working directory, containing only
the dynamically generated url. When detected by the template, it will use it as the newly deployed environment url.
### Deployment output variables
Each deployment job produces _output variables_ that are propagated to downstream jobs (using [dotenv artifacts](https://docs.gitlab.com/ee/ci/pipelines/job_artifacts.html#artifactsreportsdotenv)):
* `$environment_type`: set to the type of environment (`review`, `integration`, `staging` or `production`),
* `$environment_name`: the application name (see below),
* `$environment_url`: set to the environment URL (whether determined statically or dynamically).
Those variables may be freely used in downstream jobs (for instance to run acceptance tests against the latest deployed environment).
## Configuration reference
### Secrets management
......@@ -101,22 +171,18 @@ Here are some advices about your **secrets** (variables marked with a :lock:):
it will then be possible to mask it and the template will automatically decode it prior to using it.
3. Don't forget to escape special characters (ex: `$` -> `$$`).
:warning: your [Values files](https://helm.sh/docs/chart_template_guide/values_files/) **may** contain variable patterns such as `${MY_SECRET}`.
If so, those patterns will be evaluated (replaced) with actual environment values. This is a safe way of managing your application secrets.
### Deploy &amp; cleanup jobs
The Helm template declares deployment &amp; cleanup jobs for each supported environment.
It supports 2 deployment cases:
* using an **external** Helm chart (retrieved from a repository),
* using an **internal** Helm chart (located in the project).
### Global configuration
Here are global configuration variables for deploy jobs.
The Helm template uses some global configuration used throughout all jobs.
| Name | description | default value |
| --------------------- | -------------------------------------- | ----------------- |
| `HELM_CLI_IMAGE` | The Docker image used to run Helm <br/>:warning: **set the version required by your Kubernetes server** | `alpine/helm:latest` |
| `HELM_CHART_DIR` | The folder where the Helm chart is located | `.` _(root project dir)_ |
| `HELM_SCRIPTS_DIR` | The folder where hook scripts are located | `.` _(root project dir)_ |
| `HELM_COMMON_VALUES` | Common values file (used for all environments, overridden by specific per-env values files) | undefined (none) |
| `HELM_ENV_VALUE_NAME` | The environment type variable set to helm | `environment_type` |
| `HELM_HOSTNAME_VALUE_NAME` | The hostname variable set to helm | `hostname` |
| `KUBE_NAMESPACE` | The default Kubernetes namespace to use | _none_ but this variable is automatically set by [GitLab Kubernetes integration](https://docs.gitlab.com/ee/user/project/clusters/index.html) when enabled |
| :lock: `HELM_DEFAULT_KUBE_CONFIG` | The default kubeconfig content to use | `$KUBECONFIG` (thus supports the [GitLab Kubernetes integration](https://docs.gitlab.com/ee/user/project/clusters/index.html) when enabled) |
| `HELM_DEPLOY_ARGS` | The Helm [command with options](https://helm.sh/docs/helm/helm_upgrade/) to deploy the application (_without dynamic arguments such as release name and chart_) | `upgrade --install --atomic --timeout 120s` |
......@@ -126,17 +192,7 @@ Here are global configuration variables for deploy jobs.
| `HELM_BASE_APP_NAME` | Base application name | `$CI_PROJECT_NAME` ([see GitLab doc](https://docs.gitlab.com/ee/ci/variables/predefined_variables.html)) |
| `HELM_ENVIRONMENT_URL` | Default environments url _(only define for static environment URLs declaration)_<br/>_supports late variable expansion (ex: `https://%{environment_name}.helm.acme.com`)_ | _none_ |
Each deployment job produces _output variables_ that are propagated to downstream jobs (using [dotenv artifacts](https://docs.gitlab.com/ee/ci/pipelines/job_artifacts.html#artifactsreportsdotenv)):
* `environment_type`: set to the type of environment (`review`, `integration`, `staging` or `production`),
* `environment_name`: the application name (see below),
* `environment_url`: set to `$CI_ENVIRONMENT_URL`.
They may be freely used in downstream jobs (for instance to run acceptance tests against the latest deployed environment).
Here are configuration details for each environment.
#### Review environments
### Review environments configuration
Review environments are dynamic and ephemeral environments to deploy your _ongoing developments_ (a.k.a. _feature_ or _topic_ branches).
......@@ -153,7 +209,7 @@ Here are variables supported to configure review environments:
| :lock: `HELM_REVIEW_KUBE_CONFIG` | kubeconfig content used for `review` env _(only define to override default)_ | `$HELM_DEFAULT_KUBE_CONFIG` |
| `HELM_REVIEW_VALUES` | The [Values file](https://helm.sh/docs/chart_template_guide/values_files/) to use with `review` environments | _none_ |
#### Integration environment
### Integration environment configuration
The integration environment is the environment associated to your integration branch (`develop` by default).
......@@ -170,7 +226,7 @@ Here are variables supported to configure the integration environment:
| :lock: `HELM_INTEG_KUBE_CONFIG` | kubeconfig content used for `integration` env _(only define to override default)_ | `$HELM_DEFAULT_KUBE_CONFIG` |
| `HELM_INTEG_VALUES` | The [Values file](https://helm.sh/docs/chart_template_guide/values_files/) to use with the `integration` environment | _none_ |
#### Staging environment
### Staging environment configuration
The staging environment is an iso-prod environment meant for testing and validation purpose associated to your production branch (`master` by default).
......@@ -187,7 +243,7 @@ Here are variables supported to configure the staging environment:
| :lock: `HELM_STAGING_KUBE_CONFIG` | kubeconfig content used for `staging` env _(only define to override default)_ | `$HELM_DEFAULT_KUBE_CONFIG` |
| `HELM_STAGING_VALUES` | The [Values file](https://helm.sh/docs/chart_template_guide/values_files/) to use with the staging environment | _none_ |
#### Production environment
### Production environment configuration
The production environment is the final deployment environment associated with your production branch (`master` by default).
......@@ -205,37 +261,6 @@ Here are variables supported to configure the production environment:
| `AUTODEPLOY_TO_PROD` | Set this variable to auto-deploy to production. If not set deployment to production will be `manual` (default behaviour). | _none_ (disabled) |
| `HELM_PROD_VALUES` | The [Values file](https://helm.sh/docs/chart_template_guide/values_files/) to use with the production environment | _none_ |
#### Dynamic Values and Variables
You have to be aware that your deployment (and cleanup) scripts have to be able to cope with various environments
(`review`, `integration`, `staging` and `production`), each with different application names, exposed routes, settings, ...
Part of this complexity can be handled by the lookup policies described above (ex: one resource per env).
In order to be able to implement some **genericity** in your scripts and templates:
1. you should use generic [values](https://helm.sh/docs/chart_best_practices/values/) dynamically set and passed by the template:
* `$HELM_ENV_VALUE_NAME` (set by default to `environment_type`): the environment type (`review`, `integration`, `staging` or `production`)
* `$HELM_HOSTNAME_VALUE_NAME` (set by default to `hostname`): the environment hostname, extracted from `${CI_ENVIRONMENT_URL}` (got from [`environment:url`](https://docs.gitlab.com/ee/ci/yaml/#environmenturl) - see `OS_REVIEW_ENVIRONMENT_SCHEME`, `OS_REVIEW_ENVIRONMENT_DOMAIN`, `OS_STAGING_ENVIRONMENT_URL` and `OS_PROD_ENVIRONMENT_URL`)
2. you should use available environment variables:
* any [GitLab CI variable](https://docs.gitlab.com/ee/ci/variables/predefined_variables.html)
(ex: `${CI_ENVIRONMENT_URL}` to retrieve the actual environment exposed route)
* any [custom variable](https://docs.gitlab.com/ee/ci/variables/#add-a-cicd-variable-to-a-project)
(ex: `${SECRET_TOKEN}` that you have set in your project CI/CD variables)
> :warning:
>
> In order to be properly replaced, variables in your YAML value file shall be written with curly braces (ex: `${MYVAR}` and not `$MYVAR`).
>
> Multiline variables must be surrounded by double quotes and you might have to disable line-length rule of yamllint as they are rewritten on a single line.
>
> ```yaml
> tlsKey: "${MYKEY}" # yamllint disable-line rule:line-length
> ```
### `helm-lint` job
This job [examines your chart for possible issues](https://helm.sh/docs/helm/helm_lint/) and uses the following variables:
......@@ -267,7 +292,25 @@ This job runs [Kube-Score](https://kube-score.com/) on the resources to be creat
| `HELM_KUBE_SCORE_IMAGE` | The Docker image used to run [Kube-Score](https://kube-score.com/) | `zegl/kube-score:latest-helm3` |
| `HELM_KUBE_SCORE_ARGS` | Arguments used by the helm-score job | _none_ |
### `helm-package` job
### Charts publishing
The template builds a chart package that may be pushed as two distinct packages, depending on a certain _workflow_:
1. **snapshot**: the chart is first packaged and then pushed to some registry as
the **snapshot** image. It can be seen as the raw result of the build, but still **untested and unreliable**.
2. **release**: once the snapshot chart has been thoroughly tested (both by `package-test` stage jobs and/or `acceptance`
stage jobs after being deployed to some server), then the chart is pushed one more time as the **release** chart.
This second push can be seen as the **promotion** of the snapshot chart being now **tested and reliable**.
Common variables for `helm-package` and `helm-pusblish`:
| Name | description | default value |
| --------------------------------- | -------------------------------------------- | ----------------------- |
| `HELM_REPO_PUBLISH_METHOD` | HTTP method to use to push the package | `POST` |
| :lock: `HELM_REPO_USER` | Helm registry username | `$CI_REGISTRY_USER` |
| :lock: `HELM_REPO_PASSWORD` | Helm registry password | `$CI_REGISTRY_PASSWORD` |
#### `helm-package` job
This job [packages your chart into an archive](https://helm.sh/docs/helm/helm_package/), optionaly push it to a snapshot repository and uses the following variables:
......@@ -288,7 +331,7 @@ If no next version info is generated by `semantic-release`, the package will be
Note: You can disable the `semantic-release` integration described herebefore the `HELM_SEMREL_RELEASE_DISABLED` variable.
### `helm-publish` job
#### `helm-publish` job
This job push helm package to a release repository and uses the following variables:
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment