diff --git a/README.md b/README.md
index 4c9d4b385e71093c9d3b2b95773d099e2069b5c3..ecc6712c98b283b766039003f34dd3700aa8108d 100644
--- a/README.md
+++ b/README.md
@@ -2,14 +2,29 @@
This project implements a generic GitLab CI template for [Helm](https://helm.sh/).
-## Overview
+## Usage
+
+In order to include this template in your project, add the following to your `gitlab-ci.yml`:
+
+```yaml
+include:
+ - project: 'to-be-continuous/helm'
+ ref: '3.2.0'
+ file: '/templates/gitlab-ci-helm.yml'
+```
+
+## Understand
+
+This chapter introduces key notions and principle to understand how this template works.
+
+### Managed deployment environments
This template implements continuous delivery/continuous deployment based on [Helm](https://helm.sh/) for projects hosted
on [Kubernetes](https://kubernetes.io) platforms.
It provides several features, usable in different modes (by configuration).
-### Review environments
+#### Review environments
The template supports **review** environments: those are dynamic and ephemeral environments to deploy your
_ongoing developments_ (a.k.a. _feature_ or _topic_ branches).
@@ -21,7 +36,7 @@ It is a strict equivalent of GitLab's [Review Apps](https://docs.gitlab.com/ee/c
It also comes with a _cleanup_ job (accessible either from the _environments_ page, or from the pipeline view).
-### Integration environment
+#### Integration environment
If you're using a Git Workflow with an integration branch (such as [Gitflow](https://www.atlassian.com/git/tutorials/comparing-workflows/gitflow-workflow)),
the template supports an **integration** environment.
@@ -29,7 +44,7 @@ the template supports an **integration** environment.
When enabled, it deploys the result from upstream build stages to a dedicated environment.
It is only active for your integration branch (`develop` by default).
-### Production environments
+#### Production environments
Lastly, the template supports 2 environments associated to your production branch (`master` by default):
@@ -41,81 +56,133 @@ You're free to enable whichever or both, and you can also choose your deployment
* **continuous deployment**: automatic deployment to production (when the upstream pipeline is successful),
* **continuous delivery**: deployment to production can be triggered manually (when the upstream pipeline is successful).
-## Usage
+<!--
+### Supported authentication methods
+-->
-### Include
+### Deployment context variables
-In order to include this template in your project, add the following to your `gitlab-ci.yml`:
+In order to manage the various deployment environments, this template provides a couple of **dynamic variables**
+that you might use in your hook scripts and Helm charts (as [values](https://helm.sh/docs/chart_best_practices/values/)):
-```yaml
-include:
- - project: 'to-be-continuous/helm'
- ref: '3.2.0'
- file: '/templates/gitlab-ci-helm.yml'
-```
+| environment variable | template directive | description |
+|----------------------|--------------------|-------------|
+| `$environment_name` | `{{ .Release.Name }}` | a generated application name to use for the current deployment environment (ex: `myproject-review-fix-bug-12` or `myproject-staging`). This is used as the **Helm release name** in deploy & delete jobs - _details below_ |
+| `$environment_type` | `{{ .Values.environment_type }}` | the current deployment environment type (`review`, `integration`, `staging` or `production`) |
+| `$hostname` | `{{ .Values.hostname }}` | the environment hostame, if you [specified the environment url statically](#environments-url-management) |
-### Global configuration
+#### Generated environment name
-The Helm template uses some global configuration used throughout all jobs.
+The `${environment_name}` variable is generated to designate each deployment environment with a unique and meaningful application name.
+By construction, it is suitable for inclusion in DNS, URLs, Kubernetes labels...
+It is built from:
-| Name | description | default value |
-| --------------------- | -------------------------------------- | ----------------- |
-| `HELM_CLI_IMAGE` | The Docker image used to run Helm <br/>:warning: **set the version required by your Kubernetes server** | `alpine/helm:latest` |
-| `HELM_CHART_DIR` | The folder in which is stored the Helm chart | `.` |
-| `HELM_COMMON_VALUES` | Common values file (used for all environments, overridden by specific per-env values files) | undefined (none) |
-| `HELM_ENV_VALUE_NAME` | The environment type variable set to helm | `env` |
-| `HELM_HOSTNAME_VALUE_NAME` | The hostname variable set to helm | `hostname` |
+* the application _base name_ (defaults to `$CI_PROJECT_NAME` but can be overridden globally and/or per deployment environment - _see configuration variables_)
+* GitLab predefined `$CI_ENVIRONMENT_SLUG` variable ([sluggified](https://en.wikipedia.org/wiki/Clean_URL#Slug) name, truncated to 24 characters)
-### Charts publishing
+The `${environment_name}` variable is then evaluated as:
-The template builds a chart package that may be pushed as two distinct packages, depending on a certain _workflow_:
+* `<app base name>` for the production environment
+* `<app base name>-$CI_ENVIRONMENT_SLUG` for all other deployment environments
+* :bulb: `${environment_name}` can also be overriden per environment with the appropriate configuration variable
-1. **snapshot**: the chart is first packaged and then pushed to some registry as
- the **snapshot** image. It can be seen as the raw result of the build, but still **untested and unreliable**.
-2. **release**: once the snapshot chart has been thoroughly tested (both by `package-test` stage jobs and/or `acceptance`
- stage jobs after being deployed to some server), then the chart is pushed one more time as the **release** chart.
- This second push can be seen as the **promotion** of the snapshot chart being now **tested and reliable**.
+Examples (with an application's base name `myapp`):
-Common variables for `helm-package` and `helm-pusblish`:
+| `$environment_type` | Branch | `$CI_ENVIRONMENT_SLUG` | `$environment_name` |
+|---------------------|---------------|-------------------------|---------------------|
+| `review` | `feat/blabla` | `review-feat-bla-xmuzs6`| `myapp-review-feat-bla-xmuzs6` |
+| `integration` | `develop` | `integration` | `myapp-integration` |
+| `staging` | `main` | `staging` | `myapp-staging` |
+| `production` | `main` | `production` | `myapp` |
-| Name | description | default value |
-| --------------------------------- | -------------------------------------------- | ----------------------- |
-| `HELM_REPO_PUBLISH_METHOD` | HTTP method to use to push the package | `POST` |
-| :lock: `HELM_REPO_USER` | Helm registry username | `$CI_REGISTRY_USER` |
-| :lock: `HELM_REPO_PASSWORD` | Helm registry password | `$CI_REGISTRY_PASSWORD` |
+### Deployment and cleanup scripts
+
+The Helm template requires you to provide a Helm chart (either in the project or located in an external repository) to deploy and delete the application.
+
+The environment deployment is processed as follows:
+
+1. _optionally_ executes the `helm-pre-deploy.sh` script in your project to perform specific environment pre-initialization (for e.g. create required services),
+2. [`helm upgrade`](https://helm.sh/docs/helm/helm_upgrade/) the chart with the configured parameters, using [`$environment_name`](#using-variables) as release name,
+3. _optionally_ executes the `helm-post-deploy.sh` script in your project to perform specific environment post-initialization stuff,
+
+The environment deletion is processed as follows:
+
+1. _optionally_ executes the `helm-pre-delete.sh` script in your project to perform specific environment pre-cleanup stuff,
+2. [`helm uninstall`](https://helm.sh/docs/helm/helm_uninstall/), using [`$environment_name`](#using-variables) as release name,
+3. _optionally_ executes the `helm-post-delete.sh` script in your project to perform specific environment post-cleanup (for e.g. delete bound services).
+
+:warning: each of the above hook scripts needs to be executable, you can add flag execution with: `git update-index --chmod=+x helm-pre-cleanup.sh`
+### Environments URL management
+The Helm template supports two ways of providing your environments url:
+
+* a **static way**: when the environments url can be determined in advance, probably because you're exposing your routes through a DNS you manage,
+* a [**dynamic way**](https://docs.gitlab.com/ee/ci/environments/#set-dynamic-environment-urls-after-a-job-finishes): when the url cannot be known before the
+ deployment job is executed.
+
+The **static way** can be implemented simply by setting the appropriate configuration variable(s) depending on the environment (see environments configuration chapters):
+
+* `$HELM_ENVIRONMENT_URL` to define a default url pattern for all your envs,
+* `$HELM_REVIEW_ENVIRONMENT_URL`, `$HELM_INTEG_ENVIRONMENT_URL`, `$HELM_STAGING_ENVIRONMENT_URL` and `$HELM_PROD_ENVIRONMENT_URL` to override the default.
+
+> :information_source: Each of those variables support a **late variable expansion mechanism** with the `%{somevar}` syntax,
+> allowing you to use any dynamically evaluated variables such as `${environment_name}`.
+>
+> Example:
+>
+> ```yaml
+> variables:
+> HELM_BASE_APP_NAME: "wonderapp"
+> # global url for all environments
+> HELM_ENVIRONMENT_URL: "https://%{environment_name}.nonprod.acme.domain"
+> # override for prod (late expansion of $HELM_BASE_APP_NAME not needed here)
+> HELM_PROD_ENVIRONMENT_URL: "https://$HELM_BASE_APP_NAME.acme.domain"
+> # override for review (using separate resource paths)
+> HELM_REVIEW_ENVIRONMENT_URL: "https://wonderapp-review.nonprod.acme.domain/%{environment_name}"
+> ```
+
+To implement the **dynamic way**, your deployment script shall simply generate a `environment_url.txt` file in the working directory, containing only
+the dynamically generated url. When detected by the template, it will use it as the newly deployed environment url.
+
+### Deployment output variables
+
+Each deployment job produces _output variables_ that are propagated to downstream jobs (using [dotenv artifacts](https://docs.gitlab.com/ee/ci/pipelines/job_artifacts.html#artifactsreportsdotenv)):
+
+* `$environment_type`: set to the type of environment (`review`, `integration`, `staging` or `production`),
+* `$environment_name`: the application name (see below),
+* `$environment_url`: set to the environment URL (whether determined statically or dynamically).
+
+Those variables may be freely used in downstream jobs (for instance to run acceptance tests against the latest deployed environment).
+
+## Configuration reference
### Secrets management
Here are some advices about your **secrets** (variables marked with a :lock:):
-1. Manage them as [project or group CI/CD variables](https://docs.gitlab.com/ee/ci/variables/#create-a-custom-variable-in-the-ui):
- * [**masked**](https://docs.gitlab.com/ee/ci/variables/#mask-a-custom-variable) to prevent them from being inadvertently
+1. Manage them as [project or group CI/CD variables](https://docs.gitlab.com/ee/ci/variables/#add-a-cicd-variable-to-a-project):
+ * [**masked**](https://docs.gitlab.com/ee/ci/variables/#mask-a-cicd-variable) to prevent them from being inadvertently
displayed in your job logs,
- * [**protected**](https://docs.gitlab.com/ee/ci/variables/#protect-a-custom-variable) if you want to secure some secrets
+ * [**protected**](https://docs.gitlab.com/ee/ci/variables/#protected-cicd-variables) if you want to secure some secrets
you don't want everyone in the project to have access to (for instance production secrets).
-2. In case a secret contains [characters that prevent it from being masked](https://docs.gitlab.com/ee/ci/variables/#masked-variable-requirements),
+2. In case a secret contains [characters that prevent it from being masked](https://docs.gitlab.com/ee/ci/variables/#mask-a-cicd-variable),
simply define its value as the [Base64](https://en.wikipedia.org/wiki/Base64) encoded value prefixed with `@b64@`:
it will then be possible to mask it and the template will automatically decode it prior to using it.
3. Don't forget to escape special characters (ex: `$` -> `$$`).
-:warning: your [Values files](https://helm.sh/docs/chart_template_guide/values_files/) **may** contain variable patterns such as `${MY_SECRET}`.
-If so, those patterns will be evaluated (replaced) with actual environment values. This is a safe way of managing your application secrets.
-
-### Deploy & cleanup jobs
-
-The Helm template declares deployment & cleanup jobs for each supported environment.
-
-It supports 2 deployment cases:
-
-* using an **external** Helm chart (retrieved from a repository),
-* using an **internal** Helm chart (located in the project).
+### Global configuration
-Here are global configuration variables for deploy jobs.
+The Helm template uses some global configuration used throughout all jobs.
| Name | description | default value |
| --------------------- | -------------------------------------- | ----------------- |
+| `HELM_CLI_IMAGE` | The Docker image used to run Helm <br/>:warning: **set the version required by your Kubernetes server** | `alpine/helm:latest` |
+| `HELM_CHART_DIR` | The folder where the Helm chart is located | `.` _(root project dir)_ |
+| `HELM_SCRIPTS_DIR` | The folder where hook scripts are located | `.` _(root project dir)_ |
+| `HELM_COMMON_VALUES` | Common values file (used for all environments, overridden by specific per-env values files) | undefined (none) |
+| `HELM_ENV_VALUE_NAME` | The environment type variable set to helm | `environment_type` |
+| `HELM_HOSTNAME_VALUE_NAME` | The hostname variable set to helm | `hostname` |
| `KUBE_NAMESPACE` | The default Kubernetes namespace to use | _none_ but this variable is automatically set by [GitLab Kubernetes integration](https://docs.gitlab.com/ee/user/project/clusters/index.html) when enabled |
| :lock: `HELM_DEFAULT_KUBE_CONFIG` | The default kubeconfig content to use | `$KUBECONFIG` (thus supports the [GitLab Kubernetes integration](https://docs.gitlab.com/ee/user/project/clusters/index.html) when enabled) |
| `HELM_DEPLOY_ARGS` | The Helm [command with options](https://helm.sh/docs/helm/helm_upgrade/) to deploy the application (_without dynamic arguments such as release name and chart_) | `upgrade --install --atomic --timeout 120s` |
@@ -123,18 +190,9 @@ Here are global configuration variables for deploy jobs.
| `HELM_DEPLOY_CHART` | The Helm [chart](https://helm.sh/docs/topics/charts/) to deploy. _Only required if you want to deploy an **external** chart._ | _none_ |
| `HELM_REPOS` | The Helm [chart repositories](https://helm.sh/docs/topics/chart_repository/) to use (formatted as `repo_name_1@:repo_url_1 repo_name_2@:repo_url_2 ...`) | `stable@https://charts.helm.sh/stable bitnami@https://charts.bitnami.com/bitnami` |
| `HELM_BASE_APP_NAME` | Base application name | `$CI_PROJECT_NAME` ([see GitLab doc](https://docs.gitlab.com/ee/ci/variables/predefined_variables.html)) |
+|Â `HELM_ENVIRONMENT_URL` | Default environments url _(only define for static environment URLs declaration)_<br/>_supports late variable expansion (ex: `https://%{environment_name}.helm.acme.com`)_ | _none_ |
-Each deployment job produces _output variables_ that are propagated to downstream jobs (using [dotenv artifacts](https://docs.gitlab.com/ee/ci/pipelines/job_artifacts.html#artifactsreportsdotenv)):
-
-* `environment_type`: set to the type of environment (`review`, `integration`, `staging` or `production`),
-* `environment_name`: the application name (see below),
-* `environment_url`: set to `$CI_ENVIRONMENT_URL`.
-
-They may be freely used in downstream jobs (for instance to run acceptance tests against the latest deployed environment).
-
-Here are configuration details for each environment.
-
-#### Review environments
+### Review environments configuration
Review environments are dynamic and ephemeral environments to deploy your _ongoing developments_ (a.k.a. _feature_ or _topic_ branches).
@@ -146,15 +204,12 @@ Here are variables supported to configure review environments:
| ------------------------ | -------------------------------------- | ----------------- |
| `HELM_REVIEW_DISABLED` | Set to `true` to disable `review` env | _none_ (enabled) |
| `HELM_REVIEW_APP_NAME` | Application name for `review` env | `"${HELM_BASE_APP_NAME}-${CI_ENVIRONMENT_SLUG}"` (ex: `myproject-review-fix-bug-12`) |
+|Â `HELM_REVIEW_ENVIRONMENT_URL`| The review environments url _(only define for static environment URLs declaration and if different from default)_ | `$HELM_ENVIRONMENT_URL` |
| `HELM_REVIEW_NAMESPACE` | The Kubernetes namespace to use for `review` env _(only define to override default)_ | `$KUBE_NAMESPACE` |
| :lock: `HELM_REVIEW_KUBE_CONFIG` | kubeconfig content used for `review` env _(only define to override default)_ | `$HELM_DEFAULT_KUBE_CONFIG` |
| `HELM_REVIEW_VALUES` | The [Values file](https://helm.sh/docs/chart_template_guide/values_files/) to use with `review` environments | _none_ |
-| `HELM_REVIEW_ENVIRONMENT_SCHEME` | The review environment protocol scheme | `https` |
-| `HELM_REVIEW_ENVIRONMENT_DOMAIN` | The review environment domain | _none_ |
-Note: By default review `environment.url` will be built as `${HELM_REVIEW_ENVIRONMENT_SCHEME}://${$CI_PROJECT_NAME}-${CI_ENVIRONMENT_SLUG}.${HELM_REVIEW_ENVIRONMENT_DOMAIN}`
-
-#### Integration environment
+### Integration environment configuration
The integration environment is the environment associated to your integration branch (`develop` by default).
@@ -166,12 +221,12 @@ Here are variables supported to configure the integration environment:
| ------------------------ | -------------------------------------- | ----------------- |
| `HELM_INTEG_DISABLED` | Set to `true` to disable `integration` env | _none_ (enabled) |
| `HELM_INTEG_APP_NAME` | Application name for `integration` env | `$HELM_BASE_APP_NAME-integration` |
+|Â `HELM_INTEG_ENVIRONMENT_URL`| The integration environment url _(only define for static environment URLs declaration and if different from default)_ | `$HELM_ENVIRONMENT_URL` |
| `HELM_INTEG_NAMESPACE` | The Kubernetes namespace to use for `integration` env _(only define to override default)_ | `$KUBE_NAMESPACE` |
| :lock: `HELM_INTEG_KUBE_CONFIG` | kubeconfig content used for `integration` env _(only define to override default)_ | `$HELM_DEFAULT_KUBE_CONFIG` |
| `HELM_INTEG_VALUES` | The [Values file](https://helm.sh/docs/chart_template_guide/values_files/) to use with the `integration` environment | _none_ |
-|Â `HELM_INTEG_ENVIRONMENT_URL` | The integration environment url **including scheme** (ex: `https://my-application-integration.nonpublic.k8s.domain.com`). Do not use variable inside variable definition as it will result in a two level cascade variable and gitlab does not allow that. | _none_ |
-#### Staging environment
+### Staging environment configuration
The staging environment is an iso-prod environment meant for testing and validation purpose associated to your production branch (`master` by default).
@@ -183,12 +238,12 @@ Here are variables supported to configure the staging environment:
| ------------------------ | -------------------------------------- | ----------------- |
| `HELM_STAGING_DISABLED` | Set to `true` to disable `staging` env | _none_ (enabled) |
| `HELM_STAGING_APP_NAME` | Application name for `staging` env | `$HELM_BASE_APP_NAME-staging` |
+|Â `HELM_STAGING_ENVIRONMENT_URL`| The staging environment url _(only define for static environment URLs declaration and if different from default)_ | `$HELM_ENVIRONMENT_URL` |
| `HELM_STAGING_NAMESPACE` | The Kubernetes namespace to use for `staging` env _(only define to override default)_ | `$KUBE_NAMESPACE` |
| :lock: `HELM_STAGING_KUBE_CONFIG` | kubeconfig content used for `staging` env _(only define to override default)_ | `$HELM_DEFAULT_KUBE_CONFIG` |
| `HELM_STAGING_VALUES` | The [Values file](https://helm.sh/docs/chart_template_guide/values_files/) to use with the staging environment | _none_ |
-|Â `HELM_STAGING_ENVIRONMENT_URL` | The staging environment url **including scheme** (ex: `https://my-application-staging.nonpublic.k8s.domain.com`). Do not use variable inside variable definition as it will result in a two level cascade variable and gitlab does not allow that. | _none_ |
-#### Production environment
+### Production environment configuration
The production environment is the final deployment environment associated with your production branch (`master` by default).
@@ -200,42 +255,11 @@ Here are variables supported to configure the production environment:
| ------------------------ | -------------------------------------- | ----------------- |
| `HELM_PROD_DISABLED` | Set to `true` to disable `production` env | _none_ (enabled) |
| `HELM_PROD_APP_NAME` | Application name for `production` env | `$HELM_BASE_APP_NAME` |
+|Â `HELM_PROD_ENVIRONMENT_URL`| The production environment url _(only define for static environment URLs declaration and if different from default)_ | `$HELM_ENVIRONMENT_URL` |
| `HELM_PROD_NAMESPACE` | The Kubernetes namespace to use for `production` env _(only define to override default)_ | `$KUBE_NAMESPACE` |
| :lock: `HELM_PROD_KUBE_CONFIG` | kubeconfig content used for `production` env _(only define to override default)_ | `$HELM_DEFAULT_KUBE_CONFIG` |
| `AUTODEPLOY_TO_PROD` | Set this variable to auto-deploy to production. If not set deployment to production will be `manual` (default behaviour). | _none_ (disabled) |
| `HELM_PROD_VALUES` | The [Values file](https://helm.sh/docs/chart_template_guide/values_files/) to use with the production environment | _none_ |
-| `HELM_PROD_ENVIRONMENT_URL` |Â The production environment url **including scheme** (ex: `https://my-application.public.k8s.domain.com`) Do not use variable inside variable definition as it will result in a two level cascade variable and gitlab does not allow that. | _none_ |
-
-#### Dynamic Values and Variables
-
-You have to be aware that your deployment (and cleanup) scripts have to be able to cope with various environments
-(`review`, `integration`, `staging` and `production`), each with different application names, exposed routes, settings, ...
-
-Part of this complexity can be handled by the lookup policies described above (ex: one resource per env).
-
-In order to be able to implement some **genericity** in your scripts and templates:
-
-1. you should use generic [values](https://helm.sh/docs/chart_best_practices/values/) dynamically set and passed by the template:
-
-* `$HELM_ENV_VALUE_NAME` (set by default to `env`): the environment type (`review`, `integration`, `staging` or `production`)
-* `$HELM_HOSTNAME_VALUE_NAME` (set by default to `hostname`): the environment hostname, extracted from `${CI_ENVIRONMENT_URL}` (got from [`environment:url`](https://docs.gitlab.com/ee/ci/yaml/#environmenturl) - see `OS_REVIEW_ENVIRONMENT_SCHEME`, `OS_REVIEW_ENVIRONMENT_DOMAIN`, `OS_STAGING_ENVIRONMENT_URL` and `OS_PROD_ENVIRONMENT_URL`)
-
-2. you should use available environment variables:
-
- * any [GitLab CI variable](https://docs.gitlab.com/ee/ci/variables/#predefined-environment-variables)
- (ex: `${CI_ENVIRONMENT_URL}` to retrieve the actual environment exposed route)
- * any [custom variable](https://docs.gitlab.com/ee/ci/variables/#custom-environment-variables)
- (ex: `${SECRET_TOKEN}` that you have set in your project CI/CD variables)
-
-> :warning:
->
-> In order to be properly replaced, variables in your YAML value file shall be written with curly braces (ex: `${MYVAR}` and not `$MYVAR`).
->
-> Multiline variables must be surrounded by double quotes and you might have to disable line-length rule of yamllint as they are rewritten on a single line.
->
-> ```yaml
-> tlsKey: "${MYKEY}" # yamllint disable-line rule:line-length
-> ```
### `helm-lint` job
@@ -268,7 +292,25 @@ This job runs [Kube-Score](https://kube-score.com/) on the resources to be creat
| `HELM_KUBE_SCORE_IMAGE` | The Docker image used to run [Kube-Score](https://kube-score.com/) | `zegl/kube-score:latest-helm3` |
| `HELM_KUBE_SCORE_ARGS` | Arguments used by the helm-score job | _none_ |
-### `helm-package` job
+### Charts publishing
+
+The template builds a chart package that may be pushed as two distinct packages, depending on a certain _workflow_:
+
+1. **snapshot**: the chart is first packaged and then pushed to some registry as
+ the **snapshot** image. It can be seen as the raw result of the build, but still **untested and unreliable**.
+2. **release**: once the snapshot chart has been thoroughly tested (both by `package-test` stage jobs and/or `acceptance`
+ stage jobs after being deployed to some server), then the chart is pushed one more time as the **release** chart.
+ This second push can be seen as the **promotion** of the snapshot chart being now **tested and reliable**.
+
+Common variables for `helm-package` and `helm-pusblish`:
+
+| Name | description | default value |
+| --------------------------------- | -------------------------------------------- | ----------------------- |
+| `HELM_REPO_PUBLISH_METHOD` | HTTP method to use to push the package | `POST` |
+| :lock: `HELM_REPO_USER` | Helm registry username | `$CI_REGISTRY_USER` |
+| :lock: `HELM_REPO_PASSWORD` | Helm registry password | `$CI_REGISTRY_PASSWORD` |
+
+#### `helm-package` job
This job [packages your chart into an archive](https://helm.sh/docs/helm/helm_package/), optionaly push it to a snapshot repository and uses the following variables:
@@ -289,7 +331,7 @@ If no next version info is generated by `semantic-release`, the package will be
Note: You can disable the `semantic-release` integration described herebefore the `HELM_SEMREL_RELEASE_DISABLED` variable.
-### `helm-publish` job
+#### `helm-publish` job
This job push helm package to a release repository and uses the following variables:
diff --git a/kicker.json b/kicker.json
index a15e725c7f01b8e80d2559d6a790bbe829bc034c..12130535a1917138ab953a89f117a670e6f6f0cc 100644
--- a/kicker.json
+++ b/kicker.json
@@ -11,7 +11,12 @@
},
{
"name": "HELM_CHART_DIR",
- "description": "The folder in which is stored the Helm chart",
+ "description": "The folder where the Helm chart is located",
+ "default": "."
+ },
+ {
+ "name": "HELM_SCRIPTS_DIR",
+ "description": "The folder where hook scripts are located",
"default": "."
},
{
@@ -43,6 +48,11 @@
"default": "$CI_PROJECT_NAME",
"advanced": true
},
+ {
+ "name": "HELM_ENVIRONMENT_URL",
+ "type": "url",
+ "description": "The default environments url _(only define for static environment URLs declaration)_\n\n_supports late variable expansion (ex: `https://%{environment_name}.helm.acme.com`)_"
+ },
{
"name": "HELM_DEPLOY_ARGS",
"description": "The Helm [command with options](https://helm.sh/docs/helm/helm_upgrade/) to deploy the application (_without dynamic arguments such as release name and chart_)",
@@ -63,13 +73,13 @@
},
{
"name": "HELM_ENV_VALUE_NAME",
- "description": "The environment type variable set to helm",
- "default": "env",
+ "description": "The environment type variable set to Helm",
+ "default": "environment_type",
"advanced": true
},
{
"name": "HELM_HOSTNAME_VALUE_NAME",
- "description": "The hostname variable set to helm",
+ "description": "The hostname variable set to Helm",
"default": "hostname",
"advanced": true
},
@@ -257,6 +267,12 @@
"default": "${HELM_BASE_APP_NAME}-${CI_ENVIRONMENT_SLUG}",
"advanced": true
},
+ {
+ "name": "HELM_REVIEW_ENVIRONMENT_URL",
+ "type": "url",
+ "description": "The review environments url _(only define for static environment URLs declaration and if different from default)_",
+ "advanced": true
+ },
{
"name": "HELM_REVIEW_VALUES",
"description": "The Values file to use with `review` environment"
@@ -288,6 +304,12 @@
"default": "${HELM_BASE_APP_NAME}-integration",
"advanced": true
},
+ {
+ "name": "HELM_INTEG_ENVIRONMENT_URL",
+ "type": "url",
+ "description": "The integration environment url _(only define for static environment URLs declaration and if different from default)_",
+ "advanced": true
+ },
{
"name": "HELM_INTEG_VALUES",
"description": "The Values file to use with `integration` environment"
@@ -319,6 +341,12 @@
"default": "${HELM_BASE_APP_NAME}-staging",
"advanced": true
},
+ {
+ "name": "HELM_STAGING_ENVIRONMENT_URL",
+ "type": "url",
+ "description": "The staging environment url _(only define for static environment URLs declaration and if different from default)_",
+ "advanced": true
+ },
{
"name": "HELM_STAGING_VALUES",
"description": "The Values file to use with `staging` environment"
@@ -355,6 +383,12 @@
"default": "${HELM_BASE_APP_NAME}",
"advanced": true
},
+ {
+ "name": "HELM_PROD_ENVIRONMENT_URL",
+ "type": "url",
+ "description": "The production environment url _(only define for static environment URLs declaration and if different from default)_",
+ "advanced": true
+ },
{
"name": "HELM_PROD_VALUES",
"description": "The Values file to use with `production` environment"
diff --git a/templates/gitlab-ci-helm.yml b/templates/gitlab-ci-helm.yml
index 97c8c345304a3473b2014f661e3161115c7999af..6df4e1709f0f16583b7e84d6abe5665328364340 100644
--- a/templates/gitlab-ci-helm.yml
+++ b/templates/gitlab-ci-helm.yml
@@ -56,6 +56,7 @@ variables:
HELM_KUBE_SCORE_IMAGE: "zegl/kube-score:latest-helm3"
HELM_CHART_DIR: "."
+ HELM_SCRIPTS_DIR: "."
HELM_PACKAGE_ARGS: "package --dependency-update"
HELM_PUBLISH_URL: "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/helm/api/release/charts"
HELM_PUBLISH_SNAPSHOT_URL: "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/helm/api/snapshot/charts"
@@ -63,7 +64,7 @@ variables:
HELM_REPOS: "stable@https://charts.helm.sh/stable bitnami@https://charts.bitnami.com/bitnami"
- HELM_ENV_VALUE_NAME: env
+ HELM_ENV_VALUE_NAME: environment_type
HELM_HOSTNAME_VALUE_NAME: hostname
# Will work with gitlab Kubernetes integration (per env variables)
@@ -363,47 +364,85 @@ stages:
}
function awkenvsubst() {
- awk '{while(match($0,"[$]{[^}]*}")) {var=substr($0,RSTART+2,RLENGTH -3);val=ENVIRON[var];gsub(/["\\]/,"\\\\&", val);gsub("\n", "\\n", val);gsub("\r", "\\r", val);gsub("[$]{"var"}",val)}}1'
+ awk '{while(match($0,"[$%]{[^}]*}")) {g0=substr($0,RSTART,RLENGTH); val=ENVIRON[substr(g0,3,RLENGTH-3)]; gsub(/["\\]/,"\\\\&",val); gsub("\n","\\n",val);gsub("\r","\\r",val); gsub(g0,val)}}1'
+ }
+
+ function exec_hook() {
+ if [[ ! -x "$1" ]] && ! chmod +x "$1"
+ then
+ log_warn "... could not make \\e[33;1m${1}\\e[0m executable: please do it (chmod +x)"
+ # fallback technique
+ sh "$1"
+ else
+ "$1"
+ fi
}
# deploy application
function deploy() {
- export env=$1
- export appname=$2
- # extract hostname from $CI_ENVIRONMENT_URL
- hostname=$(echo "$CI_ENVIRONMENT_URL" | awk -F[/:] '{print $4}')
- export hostname
-
+ export environment_type=$1
+ export environment_name=$2
namespace=$3
values_files=$4
+ environment_url=$5
- log_info "--- \\e[32mdeploy\\e[0m (env: \\e[33;1m${env}\\e[0m)"
- log_info "--- appname: \\e[33;1m${appname}\\e[0m"
- log_info "--- env: \\e[33;1m${env}\\e[0m"
- log_info "--- hostname: \\e[33;1m${hostname}\\e[0m"
+ # backwards compatibility
+ export env=$environment_type
+ export appname=$environment_name
+
+ # variables expansion in $environment_url
+ environment_url=$(echo "$environment_url" | awkenvsubst)
+ export environment_url
+ # extract hostname from $environment_url
+ hostname=$(echo "$environment_url" | awk -F[/:] '{print $4}')
+ export hostname
+
+ log_info "--- \\e[32mdeploy\\e[0m"
+ log_info "--- \$namespace: \\e[33;1m${namespace}\\e[0m"
+ log_info "--- \$environment_type: \\e[33;1m${environment_type}\\e[0m (Helm variable '$HELM_ENV_VALUE_NAME')"
+ log_info "--- \$environment_name: \\e[33;1m${environment_name}\\e[0m (used as release name)"
+ log_info "--- \$hostname: \\e[33;1m${hostname}\\e[0m (Helm variable '$HELM_HOSTNAME_VALUE_NAME')"
+
+ # unset any upstream deployment env & artifacts
+ rm -f helm.env
+ rm -f environment_url.txt
+
+ # maybe execute pre deploy script
+ prescript="$HELM_SCRIPTS_DIR/helm-pre-deploy.sh"
+ if [[ -f "$prescript" ]]; then
+ log_info "--- \\e[32mpre-deploy hook\\e[0m (\\e[33;1m${prescript}\\e[0m) found: execute"
+ exec_hook "$prescript"
+ else
+ log_info "--- \\e[32mpre-deploy hook\\e[0m (\\e[33;1m${prescript}\\e[0m) not found: skip"
+ fi
helm_opts=$(get_helm_config_opt)
+
+ helm_opts="$helm_opts --set ${HELM_ENV_VALUE_NAME}=$environment_type"
+ helm_opts="$helm_opts --set ${HELM_HOSTNAME_VALUE_NAME}=$hostname"
+ # backward compatibility
+ helm_opts="$helm_opts --set env=$environment_type"
if [ -n "$HELM_COMMON_VALUES" ]; then
log_info "--- using \\e[32mcommon values\\e[0m file: \\e[33;1m${HELM_COMMON_VALUES}\\e[0m"
awkenvsubst < "$HELM_COMMON_VALUES" > generated-values-common.yml
- helm_values_opt="--values generated-values-common.yml"
+ helm_opts="$helm_opts --values generated-values-common.yml"
fi
if [ -n "$values_files" ]; then
log_info "--- using \\e[32mvalues\\e[0m file: \\e[33;1m${values_files}\\e[0m"
awkenvsubst < "$values_files" > generated-values.yml
- helm_values_opt="$helm_values_opt --values generated-values.yml"
+ helm_opts="$helm_opts --values generated-values.yml"
fi
if [ -f "$CI_PROJECT_DIR/.kubeconfig" ]; then
log_info "--- using \\e[32mkubeconfig\\e[0m: \\e[33;1m$CI_PROJECT_DIR/.kubeconfig\\e[0m"
- helm_namespace_opt="--kubeconfig $CI_PROJECT_DIR/.kubeconfig"
+ helm_opts="$helm_opts --kubeconfig $CI_PROJECT_DIR/.kubeconfig"
fi
if [ -n "$namespace" ]; then
log_info "--- using \\e[32mnamespace\\e[0m: \\e[33;1m${namespace}\\e[0m"
- helm_namespace_opt="$helm_namespace_opt --namespace $namespace"
+ helm_opts="$helm_opts --namespace $namespace"
fi
package=$(ls -1 ./helm_packages/*.tgz 2>/dev/null || echo "")
@@ -415,65 +454,104 @@ stages:
fi
log_info "--- using \\e[32mpackage\\e[0m: \\e[33;1m${package}\\e[0m"
-
# shellcheck disable=SC2086
- helm ${TRACE+--debug} $helm_opts $helm_namespace_opt $helm_values_opt --set "${HELM_ENV_VALUE_NAME}=$env,${HELM_HOSTNAME_VALUE_NAME}=$hostname" $HELM_DEPLOY_ARGS $appname $package
+ helm ${TRACE+--debug} $helm_opts $HELM_DEPLOY_ARGS $environment_name $package
- # finally persist environment url
- echo "$CI_ENVIRONMENT_URL" > environment_url.txt
- echo -e "environment_type=$env\\nenvironment_name=$appname\\nenvironment_url=$CI_ENVIRONMENT_URL" > helm.env
+ # maybe execute post deploy script
+ postscript="$HELM_SCRIPTS_DIR/helm-post-deploy.sh"
+ if [[ -f "$postscript" ]]; then
+ log_info "--- \\e[32mpost-deploy hook\\e[0m (\\e[33;1m${postscript}\\e[0m) found: execute"
+ exec_hook "$postscript"
+ else
+ log_info "--- \\e[32mpost-deploy hook\\e[0m (\\e[33;1m${postscript}\\e[0m) not found: skip"
+ fi
+
+ # persist environment url
+ if [[ -f environment_url.txt ]]
+ then
+ environment_url=$(cat environment_url.txt)
+ export environment_url
+ log_info "--- dynamic environment url found: (\\e[33;1m$environment_url\\e[0m)"
+ else
+ echo "$environment_url" > environment_url.txt
+ fi
+ echo -e "environment_type=$environment_type\\nenvironment_name=$environment_name\\nenvironment_url=$environment_url" > helm.env
}
# delete application (and dependencies)
function delete() {
- export env=$1
- export appname=$2
+ export environment_type=$1
+ export environment_name=$2
namespace=$3
- log_info "--- \\e[32mdelete\\e[0m (env: ${env})"
- log_info "--- appname: \\e[33;1m${appname}\\e[0m"
- log_info "--- env: \\e[33;1m${env}\\e[0m"
+ # backwards compatibility
+ export env=$environment_type
+ export appname=$environment_name
+
+ log_info "--- \\e[32mdelete"
+ log_info "--- \$namespace: \\e[33;1m${namespace}\\e[0m"
+ log_info "--- \$environment_type: \\e[33;1m${environment_type}\\e[0m"
+ log_info "--- \$environment_name: \\e[33;1m${environment_name}\\e[0m (used as release name)"
+
+ # maybe execute pre delete script
+ prescript="$HELM_SCRIPTS_DIR/helm-pre-delete.sh"
+ if [[ -f "$prescript" ]]; then
+ log_info "--- \\e[32mpre-delete hook\\e[0m (\\e[33;1m${prescript}\\e[0m) found: execute"
+ exec_hook "$prescript"
+ else
+ log_info "--- \\e[32mpre-delete hook\\e[0m (\\e[33;1m${prescript}\\e[0m) not found: skip"
+ fi
helm_opts=$(get_helm_config_opt)
if [ -f "$CI_PROJECT_DIR/.kubeconfig" ]; then
log_info "--- using \\e[32mkubeconfig\\e[0m: \\e[33;1m$CI_PROJECT_DIR/.kubeconfig\\e[0m"
- helm_namespace_opt="--kubeconfig $CI_PROJECT_DIR/.kubeconfig"
+ helm_opts="$helm_opts --kubeconfig $CI_PROJECT_DIR/.kubeconfig"
fi
if [ -n "$namespace" ]; then
log_info "--- using \\e[32mnamespace\\e[0m: \\e[33;1m${namespace}\\e[0m"
- helm_namespace_opt="--namespace $namespace"
+ helm_opts="$helm_opts --namespace $namespace"
fi
# shellcheck disable=SC2086
- helm ${TRACE+--debug} $helm_opts $helm_namespace_opt $HELM_DELETE_ARGS $appname
+ helm ${TRACE+--debug} $helm_opts $HELM_DELETE_ARGS $environment_name
+
+ # maybe execute post delete script
+ postscript="$HELM_SCRIPTS_DIR/helm-post-delete.sh"
+ if [[ -f "$postscript" ]]; then
+ log_info "--- \\e[32mpost-delete hook\\e[0m (\\e[33;1m${postscript}\\e[0m) found: execute"
+ exec_hook "$postscript"
+ else
+ log_info "--- \\e[32mpost-delete hook\\e[0m (\\e[33;1m${postscript}\\e[0m) not found: skip"
+ fi
}
# test application (and dependencies)
function test() {
- export env=$1
- export appname=$2
+ export environment_type=$1
+ export environment_name=$2
namespace=$3
- log_info "--- \\e[32mtest\\e[0m (env: ${env})"
- log_info "--- appname: \\e[33;1m${appname}\\e[0m"
- log_info "--- env: \\e[33;1m${env}\\e[0m"
+ log_info "--- \\e[32mtest\\e[0m (env: ${environment_type})"
+ log_info "--- \$namespace: \\e[33;1m${namespace}\\e[0m"
+ log_info "--- \$environment_name: \\e[33;1m${environment_name}\\e[0m"
+ log_info "--- \$environment_type: \\e[33;1m${environment_type}\\e[0m"
helm_opts=$(get_helm_config_opt)
if [ -f "$CI_PROJECT_DIR/.kubeconfig" ]; then
log_info "--- using \\e[32mkubeconfig\\e[0m: \\e[33;1m$CI_PROJECT_DIR/.kubeconfig\\e[0m"
- helm_namespace_opt="--kubeconfig $CI_PROJECT_DIR/.kubeconfig"
+ helm_opts="$helm_opts --kubeconfig $CI_PROJECT_DIR/.kubeconfig"
fi
if [ -n "$namespace" ]; then
log_info "--- using \\e[32mnamespace\\e[0m: \\e[33;1m${namespace}\\e[0m"
- helm_namespace_opt="--namespace $namespace"
+ helm_opts="$helm_opts --namespace $namespace"
fi
# shellcheck disable=SC2086
- helm ${TRACE+--debug} $helm_opts $helm_namespace_opt $HELM_TEST_ARGS $appname
+ helm ${TRACE+--debug} $helm_opts $HELM_TEST_ARGS $environment_name
}
function maybe_install_curl() {
@@ -650,7 +728,7 @@ helm-integration-score:
- awkenvsubst < "$HELM_INTEG_VALUES" > generated-values-integration.yml
- helm template $helm_package --values generated-values-common.yml --values generated-values-integration.yml | kube-score score ${HELM_KUBE_SCORE_ARGS} -
rules:
- # exclude when $K8S_SCORE_DISABLED is set
+ # exclude when $HELM_SCORE_DISABLED is set
- if: '$HELM_KUBE_SCORE_DISABLED == "true"'
when: never
- if: '$HELM_INTEG_VALUES == null || $HELM_INTEG_VALUES == ""'
@@ -668,7 +746,7 @@ helm-staging-score:
- awkenvsubst < "$HELM_STAGING_VALUES" > generated-values-staging.yml
- helm template $helm_package --values generated-values-common.yml --values generated-values-staging.yml | kube-score score ${HELM_KUBE_SCORE_ARGS} -
rules:
- # exclude when $K8S_SCORE_DISABLED is set
+ # exclude when $HELM_SCORE_DISABLED is set
- if: '$HELM_KUBE_SCORE_DISABLED == "true"'
when: never
- if: '$HELM_STAGING_VALUES == null || $HELM_STAGING_VALUES == ""'
@@ -683,7 +761,7 @@ helm-prod-score:
- awkenvsubst < "$HELM_PROD_VALUES" > generated-values-prod.yml
- helm template $helm_package --values generated-values-common.yml --values generated-values-prod.yml | kube-score score ${HELM_KUBE_SCORE_ARGS} -
rules:
- # exclude when $K8S_SCORE_DISABLED is set
+ # exclude when $HELM_SCORE_DISABLED is set
- if: '$HELM_KUBE_SCORE_DISABLED == "true"'
when: never
- if: '$HELM_PROD_VALUES == null || $HELM_PROD_VALUES == ""'
@@ -776,6 +854,7 @@ helm-publish:
# @arg ENV_TYPE : environment type
# @arg ENV_APP_NAME : env-specific application name
# @arg ENV_APP_SUFFIX: env-specific application suffix
+# @arg ENV_URL : env-specific application url
# @arg ENV_KUBE_CONFIG: env-specific Kubeconfig
# @arg ENV_NAMESPACE : env-specific Kubernetes namespace
# @arg ENV_VALUES : env-specific Helm values
@@ -791,7 +870,7 @@ helm-publish:
- add_helm_repositories
- setup_kubeconfig "${ENV_KUBE_CONFIG:-${HELM_DEFAULT_KUBE_CONFIG}}"
script:
- - deploy $ENV_TYPE "${ENV_APP_NAME:-${HELM_BASE_APP_NAME}${ENV_APP_SUFFIX}}" "${ENV_NAMESPACE:-${KUBE_NAMESPACE}}" "$ENV_VALUES"
+ - deploy $ENV_TYPE "${ENV_APP_NAME:-${HELM_BASE_APP_NAME}${ENV_APP_SUFFIX}}" "${ENV_NAMESPACE:-${KUBE_NAMESPACE}}" "$ENV_VALUES" "${ENV_URL:-${HELM_ENVIRONMENT_URL:-$ENV_URL_LEGACY}}"
artifacts:
name: "$ENV_TYPE env url for $CI_PROJECT_NAME on $CI_COMMIT_REF_SLUG"
paths:
@@ -799,6 +878,8 @@ helm-publish:
reports:
dotenv: helm.env
resource_group: $CI_ENVIRONMENT_NAME
+ environment:
+ url: "$environment_url" # can be either static or dynamic
# Cleanup job prototype
# Can be extended for each deletable environment
@@ -853,12 +934,13 @@ helm-review:
variables:
ENV_TYPE: review
ENV_APP_NAME: "$HELM_REVIEW_APP_NAME"
+ ENV_URL: "${HELM_REVIEW_ENVIRONMENT_URL}"
+ ENV_URL_LEGACY: "${HELM_REVIEW_ENVIRONMENT_SCHEME}://${CI_PROJECT_NAME}-${CI_ENVIRONMENT_SLUG}.${HELM_REVIEW_ENVIRONMENT_DOMAIN}"
ENV_KUBE_CONFIG: "$HELM_REVIEW_KUBE_CONFIG"
ENV_NAMESPACE: "$HELM_REVIEW_NAMESPACE"
ENV_VALUES: "$HELM_REVIEW_VALUES"
environment:
name: review/$CI_COMMIT_REF_NAME
- url: "${HELM_REVIEW_ENVIRONMENT_SCHEME}://${CI_PROJECT_NAME}-${CI_ENVIRONMENT_SLUG}.${HELM_REVIEW_ENVIRONMENT_DOMAIN}"
on_stop: helm-cleanup-review
resource_group: review/$CI_COMMIT_REF_NAME
rules:
@@ -920,12 +1002,12 @@ helm-integration:
variables:
ENV_TYPE: integration
ENV_APP_NAME: "$HELM_INTEG_APP_NAME"
+ ENV_URL: "${HELM_INTEG_ENVIRONMENT_URL}"
ENV_KUBE_CONFIG: "$HELM_INTEG_KUBE_CONFIG"
ENV_NAMESPACE: "$HELM_INTEG_NAMESPACE"
ENV_VALUES: "$HELM_INTEG_VALUES"
environment:
name: integration
- url: "${HELM_INTEG_ENVIRONMENT_URL}"
on_stop: helm-cleanup-integration
resource_group: integration
rules:
@@ -986,12 +1068,12 @@ helm-staging:
variables:
ENV_TYPE: staging
ENV_APP_NAME: "$HELM_STAGING_APP_NAME"
+ ENV_URL: "${HELM_STAGING_ENVIRONMENT_URL}"
ENV_KUBE_CONFIG: "$HELM_STAGING_KUBE_CONFIG"
ENV_NAMESPACE: "$HELM_STAGING_NAMESPACE"
ENV_VALUES: "$HELM_STAGING_VALUES"
environment:
name: staging
- url: "${HELM_STAGING_ENVIRONMENT_URL}"
on_stop: helm-cleanup-staging
resource_group: staging
rules:
@@ -1052,12 +1134,12 @@ helm-production:
ENV_TYPE: production
ENV_APP_NAME: "$HELM_PROD_APP_NAME"
ENV_APP_SUFFIX: ""
+ ENV_URL: "${HELM_PROD_ENVIRONMENT_URL}"
ENV_KUBE_CONFIG: "$HELM_PROD_KUBE_CONFIG"
ENV_NAMESPACE: "$HELM_PROD_NAMESPACE"
ENV_VALUES: "$HELM_PROD_VALUES"
environment:
name: production
- url: "${HELM_PROD_ENVIRONMENT_URL}"
resource_group: production
rules:
# exclude non-production branches