diff --git a/README.md b/README.md index 69918360f3967d8ebbeb2058cc04d532e2d474ce..96c5bc40defeb0fd8ba964468f9b4c6c883a6441 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,22 @@ This project implements a generic GitLab CI template for [Google Cloud Platform](https://cloud.google.com/) environments. -## Overview: managed environments +## Usage + +In order to include this template in your project, add the following to your `gitlab-ci.yml`: + +```yaml +include: + - project: 'to-be-continuous/gcloud' + ref: '2.2.0' + file: '/templates/gitlab-ci-gcloud.yml' +``` + +## Understand + +This chapter introduces key notions and principle to understand how this template works. + +### Managed deployment environments This template implements continuous delivery/continuous deployment for projects hosted on Google Cloud Platform. @@ -13,7 +28,7 @@ workflow, by reusing/extending the base (hidden) jobs. This is advanced usage an The following chapters present the managed predefined environments and their associated Git workflow. -### Review environments +#### Review environments The template supports **review** environments: those are dynamic and ephemeral environments to deploy your _ongoing developments_ (a.k.a. _feature_ or _topic_ branches). @@ -25,7 +40,7 @@ It is a strict equivalent of GitLab's [Review Apps](https://docs.gitlab.com/ee/c It also comes with a _cleanup_ job (accessible either from the _environments_ page, or from the pipeline view). -### Integration environment +#### Integration environment If you're using a Git Workflow with an integration branch (such as [Gitflow](https://www.atlassian.com/git/tutorials/comparing-workflows/gitflow-workflow)), the template supports an **integration** environment. @@ -33,7 +48,7 @@ the template supports an **integration** environment. When enabled, it deploys the result from upstream build stages to a dedicated environment. It is only active for your integration branch (`develop` by default). -### Production environments +#### Production environments Lastly, the template supports 2 environments associated to your production branch (`master` or `main` by default): @@ -45,47 +60,20 @@ You're free to enable whichever or both, and you can also choose your deployment * **continuous deployment**: automatic deployment to production (when the upstream pipeline is successful), * **continuous delivery**: deployment to production can be triggered manually (when the upstream pipeline is successful). -## Usage +### Supported authentication methods -### Include +The Google Cloud Platform template supports two kinds of authentication: -In order to include this template in your project, add the following to your `gitlab-ci.yml`: +1. basic authentication with [Service Account key file](https://cloud.google.com/bigquery/docs/authentication/service-account-file), +2. or [federated authentication using OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/). -```yaml -include: - - project: 'to-be-continuous/gcloud' - ref: '2.2.0' - file: '/templates/gitlab-ci-gcloud.yml' -``` +#### Service account authentication -### Global configuration +To use this authentication method, simply generate and provide [Service Account key file](https://cloud.google.com/bigquery/docs/authentication/service-account-file) as secret GitLab CI/CD variables (of type File), using the appropriate variables (see doc below). -The Google Cloud template uses some global configuration used throughout all jobs. +Can be provided globally and/or per environment. -| Name | description | default value | -| ------------------------ | -------------------------------------- | ----------------- | -| `GCP_CLI_IMAGE` | the Docker image used to run Google Cloud CLI commands| `google/cloud-sdk:latest` | -| :lock: `GCP_KEY_FILE` | Default [Service Account key file](https://cloud.google.com/bigquery/docs/authentication/service-account-file) | _none_ | -| `GCP_OIDC_PROVIDER` | Default Workload Identity Provider associated with GitLab to [authenticate with OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/) | none| -| `GCP_OIDC_ACCOUNT` | Default Service Account to which impersonate with OpenID Connect authentication | none | -| `GCP_BASE_APP_NAME` | Base application name | `$CI_PROJECT_NAME` ([see GitLab doc](https://docs.gitlab.com/ee/ci/variables/predefined_variables.html)) | -| `GCP_SCRIPTS_DIR` | Directory where Google Cloud scripts (deploy & cleanup) are located | `.` _(root project dir)_ | - -### Secrets management - -Here are some advices about your **secrets** (variables marked with a :lock:): - -1. Manage them as [project or group CI/CD variables](https://docs.gitlab.com/ee/ci/variables/#create-a-custom-variable-in-the-ui): - * [**masked**](https://docs.gitlab.com/ee/ci/variables/#mask-a-custom-variable) to prevent them from being inadvertently - displayed in your job logs, - * [**protected**](https://docs.gitlab.com/ee/ci/variables/#protect-a-custom-variable) if you want to secure some secrets - you don't want everyone in the project to have access to (for instance production secrets). -2. In case a secret contains [characters that prevent it from being masked](https://docs.gitlab.com/ee/ci/variables/#masked-variable-requirements), - simply define its value as the [Base64](https://en.wikipedia.org/wiki/Base64) encoded value prefixed with `@b64@`: - it will then be possible to mask it and the template will automatically decode it prior to using it. -3. Don't forget to escape special characters (ex: `$` -> `$$`). - -### Federated authentication using OpenID Connect +#### Federated authentication using OpenID Connect The GCP template supports a [federated authentication using OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/). @@ -106,79 +94,139 @@ The following commands may help you retrieve the different values: - `gcloud iam workload-identity-pools list --location=global --format="value(name)"` will list you POOL_IDs available on your `GCP_PROJECT` - `gcloud iam workload-identity-pools providers list --workload-identity-pool=<my-pool> --location=global --format="value(name)"` will return the list of available `PROVIDER_ID` for one `POOL_ID` -### Deployment and cleanup jobs +### Deployment context variables + +In order to manage the various deployment environments, this template provides a couple of **dynamic variables** +that you might use in your hook scripts, deployment manifests and other deployment resources: -The GitLab CI template for Google Cloud requires you to provide a shell script that fully implements your application -deployment and cleanup using the [`gcloud` CLI](https://cloud.google.com/sdk/gcloud). +* `${environment_type}`: the current deployment environment type (`review`, `integration`, `staging` or `production`) +* `${environment_name}`: a generated application name to use for the current deployment environment (ex: `myapp-review-fix-bug-12` or `myapp-staging`) - _details below_ -#### Lookup policy +#### Generated environment name + +The `${environment_name}` variable is generated to designate each deployment environment with a unique and meaningful application name. +By construction, it is suitable for inclusion in DNS, URLs, Kubernetes labels... +It is built from: + +* the application _base name_ (defaults to `$CI_PROJECT_NAME` but can be overridden globally and/or per deployment environment - _see configuration variables_) +* GitLab predefined `$CI_ENVIRONMENT_SLUG` variable ([sluggified](https://en.wikipedia.org/wiki/Clean_URL#Slug) name, truncated to 24 characters) + +The `${environment_name}` variable is then evaluated as: + +* `<app base name>` for the production environment +* `<app base name>-$CI_ENVIRONMENT_SLUG` for all other deployment environments +* :bulb: `${environment_name}` can also be overriden per environment with the appropriate configuration variable + +Examples (with an application's base name `myapp`): + +| `$environment_type` | Branch | `$CI_ENVIRONMENT_SLUG` | `$environment_name` | +|---------------------|---------------|-------------------------|---------------------| +| `review` | `feat/blabla` | `review-feat-bla-xmuzs6`| `myapp-review-feat-bla-xmuzs6` | +| `integration` | `develop` | `integration` | `myapp-integration` | +| `staging` | `main` | `staging` | `myapp-staging` | +| `production` | `main` | `production` | `myapp` | + +### Deployment and cleanup scripts + +The Google Cloud template requires you to provide a shell script that fully implements your application +deployment and cleanup using the [`gcloud` CLI](https://cloud.google.com/sdk/gcloud) and all other tools available in the selected Docker image. The deployment script is searched as follows: -1. look for a specific `gcp-deploy-$env.sh` in the `$GCP_SCRIPTS_DIR` directory in your project (e.g. `gcp-deploy-staging.sh` for staging environment), +1. look for a specific `gcp-deploy-$environment_type.sh` in the `$GCP_SCRIPTS_DIR` directory in your project (e.g. `gcp-deploy-staging.sh` for staging environment), 2. if not found: look for a default `gcp-deploy.sh` in the `$GCP_SCRIPTS_DIR` directory in your project, 3. if not found: the deployment job will fail. The cleanup script is searched as follows: -1. look for a specific `gcp-cleanup-$env.sh` in the `$GCP_SCRIPTS_DIR` directory in your project (e.g. `gcp-cleanup-staging.sh` for staging environment), +1. look for a specific `gcp-cleanup-$environment_type.sh` in the `$GCP_SCRIPTS_DIR` directory in your project (e.g. `gcp-cleanup-staging.sh` for staging environment), 2. if not found: look for a default `gcp-cleanup.sh` in the `$GCP_SCRIPTS_DIR` directory in your project, 3. if not found: the cleanup job will fail. -Your script(s) shall use available [dynamic variables](#dynamic-variables). - -#### Dynamic Variables - -You have to be aware that your deployment (and cleanup) scripts have to be able to cope with various environments -(`review`, `integration`, `staging` and `production`), each with different application names, exposed routes, settings, ... - -Part of this complexity can be handled by the lookup policies described above (ex: one resource per env). - -In order to be able to implement some **genericity** in your scripts and templates, you should use available environment variables: +> :information_source: Your deployment (and cleanup) scripts have to be able to cope with various environments, each with different application names, exposed routes, settings, ... +> Part of this complexity can be handled by the lookup policies described above (ex: one script per env) and also by using available environment variables: +> +> 1. [deployment context variables](#deployment-context-variables) provided by the template: +> * `${environment_type}`: the current environment type (`review`, `integration`, `staging` or `production`) +> * `${environment_name}`: the application name to use for the current environment (ex: `myproject-review-fix-bug-12` or `myproject-staging`) +> * `${hostname}`: the environment hostname, extracted from the current environment url (after late variable expansion - see below) +> 2. any [GitLab CI variable](https://docs.gitlab.com/ee/ci/variables/predefined_variables.html) +> 3. any [custom variable](https://docs.gitlab.com/ee/ci/variables/#add-a-cicd-variable-to-a-project) +> (ex: `${SECRET_TOKEN}` that you have set in your project CI/CD variables) -1. any [GitLab CI variable](https://docs.gitlab.com/ee/ci/variables/#predefined-environment-variables) - (ex: `${CI_ENVIRONMENT_URL}` to retrieve the actual environment exposed route) -2. any [custom variable](https://docs.gitlab.com/ee/ci/variables/#custom-environment-variables) - (ex: `${SECRET_TOKEN}` that you have set in your project CI/CD variables) -3. **dynamic variables** set by the template: - * `${appname}`: the application target name to use in this environment (ex: `myproject-review-fix-bug-12` or `myproject-staging`) - * `${env}`: the environment type (`review`, `integration`, `staging` or `production`) - * `${hostname}`: the environment hostname, extracted from `${CI_ENVIRONMENT_URL}` (has to be explicitly declared as [`environment:url`](https://docs.gitlab.com/ee/ci/yaml/#environmenturl) in your `.gitlab-ci.yml` file) - * `${gcp_project_id}`: the current Google Cloud project ID associated to your environment +### Environments URL management -#### Static vs. Dynamic environment URLs +The GCP template supports two ways of providing your environments url: -The Google Cloud template supports two ways of defining your environments url: - -* a **static way**: when you know your environments url in advance, probably because you're exposing your routes through a DNS you manage, +* a **static way**: when the environments url can be determined in advance, probably because you're exposing your routes through a DNS you manage, * a [**dynamic way**](https://docs.gitlab.com/ee/ci/environments/#set-dynamic-environment-urls-after-a-job-finishes): when the url cannot be known before the deployment job is executed. -The static way can be implemented simply by setting the appropriate configuration variables depending on the environments (see environments configuration chapters below): +The **static way** can be implemented simply by setting the appropriate configuration variable(s) depending on the environment (see environments configuration chapters): + +* `$GCP_ENVIRONMENT_URL` to define a default url pattern for all your envs, +* `$GCP_REVIEW_ENVIRONMENT_URL`, `$GCP_INTEG_ENVIRONMENT_URL`, `$GCP_STAGING_ENVIRONMENT_URL` and `$GCP_PROD_ENVIRONMENT_URL` to override the default. -* `$GCP_REVIEW_ENVIRONMENT_SCHEME` and`$GCP_REVIEW_ENVIRONMENT_DOMAIN` for the review environments, -* `$GCP_INTEG_ENVIRONMENT_URL`, `$GCP_STAGING_ENVIRONMENT_URL` and `$GCP_PROD_ENVIRONMENT_URL` for others. +> :information_source: Each of those variables support a **late variable expansion mechanism** with the `%{somevar}` syntax, +> allowing you to use any dynamically evaluated variables such as `${environment_name}`. +> +> Example: +> +> ```yaml +> variables: +> GCP_BASE_APP_NAME: "wonderapp" +> # global url for all environments +> GCP_ENVIRONMENT_URL: "https://%{environment_name}.nonprod.acme.domain" +> # override for prod (late expansion of $GCP_BASE_APP_NAME not needed here) +> GCP_PROD_ENVIRONMENT_URL: "https://$GCP_BASE_APP_NAME.acme.domain" +> # override for review (using separate resource paths) +> GCP_REVIEW_ENVIRONMENT_URL: "https://wonderapp-review.nonprod.acme.domain/%{environment_name}" +> ``` -To implement the dynamic way, your deployment script shall simply generate a `environment_url.txt` file, containing only -the dynamically generated url. +To implement the **dynamic way**, your deployment script shall simply generate a `environment_url.txt` file in the working directory, containing only +the dynamically generated url. When detected by the template, it will use it as the newly deployed environment url. -#### Deployment output variables +### Deployment output variables Each deployment job produces _output variables_ that are propagated to downstream jobs (using [dotenv artifacts](https://docs.gitlab.com/ee/ci/pipelines/job_artifacts.html#artifactsreportsdotenv)): * `$environment_type`: set to the type of environment (`review`, `integration`, `staging` or `production`), * `$environment_name`: the application name (see below), -* `$environment_url`: set to `$CI_ENVIRONMENT_URL`. +* `$environment_url`: set to the environment URL (whether determined statically or dynamically). Those variables may be freely used in downstream jobs (for instance to run acceptance tests against the latest deployed environment). -### Environments configuration +## Confuguration reference -As seen above, the Google Cloud template may support up to 4 environments (`review`, `integration`, `staging` and `production`). +### Secrets management -Here are configuration details for each environment. +Here are some advices about your **secrets** (variables marked with a :lock:): -#### Review environments +1. Manage them as [project or group CI/CD variables](https://docs.gitlab.com/ee/ci/variables/#create-a-custom-variable-in-the-ui): + * [**masked**](https://docs.gitlab.com/ee/ci/variables/#mask-a-custom-variable) to prevent them from being inadvertently + displayed in your job logs, + * [**protected**](https://docs.gitlab.com/ee/ci/variables/#protect-a-custom-variable) if you want to secure some secrets + you don't want everyone in the project to have access to (for instance production secrets). +2. In case a secret contains [characters that prevent it from being masked](https://docs.gitlab.com/ee/ci/variables/#masked-variable-requirements), + simply define its value as the [Base64](https://en.wikipedia.org/wiki/Base64) encoded value prefixed with `@b64@`: + it will then be possible to mask it and the template will automatically decode it prior to using it. +3. Don't forget to escape special characters (ex: `$` -> `$$`). + +### Global configuration + +The Google Cloud template uses some global configuration used throughout all jobs. + +| Name | description | default value | +| ------------------------ | -------------------------------------- | ----------------- | +| `GCP_CLI_IMAGE` | the Docker image used to run Google Cloud CLI commands| `google/cloud-sdk:latest` | +| :lock: `GCP_KEY_FILE` | Default [Service Account key file](https://cloud.google.com/bigquery/docs/authentication/service-account-file) | _none_ | +| `GCP_OIDC_PROVIDER` | Default Workload Identity Provider associated with GitLab to [authenticate with OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/) | none| +| `GCP_OIDC_ACCOUNT` | Default Service Account to which impersonate with OpenID Connect authentication | none | +| `GCP_BASE_APP_NAME` | Base application name | `$CI_PROJECT_NAME` ([see GitLab doc](https://docs.gitlab.com/ee/ci/variables/predefined_variables.html)) | +|Â `GCP_ENVIRONMENT_URL` | Default environments url _(only define for static environment URLs declaration)_<br/>_supports late variable expansion (ex: `https://%{environment_name}.gcloud.acme.com`)_ | _none_ | +| `GCP_SCRIPTS_DIR` | Directory where Google Cloud scripts (deploy & cleanup) are located | `.` _(root project dir)_ | + +### Review environments configuration Review environments are dynamic and ephemeral environments to deploy your _ongoing developments_ (a.k.a. _feature_ or _topic_ branches). @@ -192,17 +240,11 @@ Here are variables supported to configure review environments: | `GCP_REVIEW_PROJECT` | Google Cloud project ID for `review` env | _none_ (disabled) | | :lock: `GCP_REVIEW_KEY_FILE`| [Service Account key file](https://cloud.google.com/bigquery/docs/authentication/service-account-file) to authenticate on `review` env _(only define if different from default)_ | `$GCP_KEY_FILE` | | `GCP_REVIEW_APP_NAME` | Application name for `review` env | `"${GCP_BASE_APP_NAME}-${CI_ENVIRONMENT_SLUG}"` (ex: `myproject-review-fix-bug-12`) | -| `GCP_REVIEW_ENVIRONMENT_SCHEME`| The review environment protocol scheme.<br/>_For static environment URLs declaration_ | `https` | -| `GCP_REVIEW_ENVIRONMENT_DOMAIN`| The review environment domain.<br/>_For static environment URLs declaration_ | _none_ | +|Â `GCP_REVIEW_ENVIRONMENT_URL`| The review environments url _(only define for static environment URLs declaration and if different from default)_ | `$GCP_ENVIRONMENT_URL` | | `GCP_REVIEW_OIDC_PROVIDER` | Workload Identity Provider associated with GitLab to [authenticate with OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/) on `review` environment | none| | `GCP_REVIEW_OIDC_ACCOUNT` | Service Account to which impersonate with OpenID Connect authentication on `review` environment | none | - - - -Note: If you're managing your environment URLs statically, review environment URLs will be built as `${AWS_REVIEW_ENVIRONMENT_SCHEME}://${$CI_PROJECT_NAME}-${CI_ENVIRONMENT_SLUG}.${AWS_REVIEW_ENVIRONMENT_DOMAIN}` - -#### Integration environment +### Integration environment configuration The integration environment is the environment associated to your integration branch (`develop` by default). @@ -215,12 +257,11 @@ Here are variables supported to configure the integration environment: | `GCP_INTEG_PROJECT` | Google Cloud project ID for `integration` env | _none_ (disabled) | | :lock: `GCP_INTEG_KEY_FILE`|[Service Account key file](https://cloud.google.com/bigquery/docs/authentication/service-account-file) to authenticate on `integration` env _(only define if different from default)_ | `$GCP_KEY_FILE` | | `GCP_INTEG_APP_NAME` | Application name for `integration` env | `${GCP_BASE_APP_NAME}-integration` | -|Â `GCP_INTEG_ENVIRONMENT_URL`| The integration environment url (ex: `https://my-application-integration.nonpublic.domain.com`).<br/>_For static environment URLs declaration_ | _none_ | +|Â `GCP_INTEG_ENVIRONMENT_URL`| The integration environment url _(only define for static environment URLs declaration and if different from default)_ | `$GCP_ENVIRONMENT_URL` | | `GCP_INTEG_OIDC_PROVIDER` | Workload Identity Provider associated with GitLab to [authenticate with OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/) on `integration` environment | none| | `GCP_INTEG_OIDC_ACCOUNT` | Service Account to which impersonate with OpenID Connect authentication on `integration` environment | none | - -#### Staging environment +### Staging environment configuration The staging environment is an iso-prod environment meant for testing and validation purpose associated to your production branch (`master` by default). @@ -234,12 +275,11 @@ Here are variables supported to configure the staging environment: | `GCP_STAGING_PROJECT` | Google Cloud project ID for `staging` env | _none_ (disabled) | | :lock: `GCP_STAGING_KEY_FILE`|[Service Account key file](https://cloud.google.com/bigquery/docs/authentication/service-account-file) to authenticate on `staging` env _(only define if different from default)_ | `$GCP_KEY_FILE` | | `GCP_STAGING_APP_NAME` | Application name for `staging` env | `${GCP_BASE_APP_NAME}-staging` | -|Â `GCP_STAGING_ENVIRONMENT_URL` | The staging environment url (ex: `https://my-application-staging.nonpublic.domain.com`).<br/>_For static environment URLs declaration_ | _none_ | +|Â `GCP_STAGING_ENVIRONMENT_URL`| The staging environment url _(only define for static environment URLs declaration and if different from default)_ | `$GCP_ENVIRONMENT_URL` | | `GCP_STAGING_OIDC_PROVIDER` | Workload Identity Provider associated with GitLab to [authenticate with OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/) on `staging` environment | none| | `GCP_STAGING_OIDC_ACCOUNT` | Service Account to which impersonate with OpenID Connect authentication on `staging` environment | none | - -#### Production environment +### Production environment configuration The production environment is the final deployment environment associated with your production branch (`master` by default). @@ -252,10 +292,11 @@ Here are variables supported to configure the production environment: | `GCP_PROD_PROJECT` | Google Cloud project ID for `production` env | _none_ (disabled) | | :lock: `GCP_PROD_KEY_FILE`|[Service Account key file](https://cloud.google.com/bigquery/docs/authentication/service-account-file) to authenticate on `production` env _(only define if different from default)_ | `$GCP_KEY_FILE` | | `GCP_PROD_APP_NAME` | Application name for `production` env | `$GCP_BASE_APP_NAME` | -| `GCP_PROD_ENVIRONMENT_URL`|Â The production environment url (ex: `https://my-application.public.domain.com`).<br/>_For static environment URLs declaration_ | _none_ | +|Â `GCP_PROD_ENVIRONMENT_URL`| The production environment url _(only define for static environment URLs declaration and if different from default)_ | `$GCP_ENVIRONMENT_URL` | | `AUTODEPLOY_TO_PROD` | Set this variable to auto-deploy to production. If not set deployment to production will be `manual` (default behaviour). | _none_ (disabled) | | `GCP_PROD_OIDC_PROVIDER` | Workload Identity Provider associated with GitLab to [authenticate with OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/) on `production ` environment | none| | `GCP_PROD_OIDC_ACCOUNT` | Service Account to which impersonate with OpenID Connect authentication on `production ` environment | none | + ## Examples ### Google AppEngine application diff --git a/kicker.json b/kicker.json index 2a9a7ec9ce7cdaa782e5b71e4d54065d30a37aaf..9122500e2a71ca836ac93e318701651c5177d18a 100644 --- a/kicker.json +++ b/kicker.json @@ -41,6 +41,11 @@ "default": "$CI_PROJECT_NAME", "advanced": true }, + { + "name": "GCP_ENVIRONMENT_URL", + "type": "url", + "description": "The default environments url _(only define for static environment URLs declaration)_\n\n_supports late variable expansion (ex: `https://%{environment_name}.gcloud.acme.com`)_" + }, { "name": "GCP_SCRIPTS_DIR", "description": "Directory where Google Cloud scripts (deploy & cleanup) are located", @@ -66,13 +71,10 @@ "advanced": true }, { - "name": "GCP_REVIEW_ENVIRONMENT_SCHEME", - "description": "The review environment protocol scheme", - "default": "https" - }, - { - "name": "GCP_REVIEW_ENVIRONMENT_DOMAIN", - "description": "The review environment domain (ex: `noprod-gcloud.domain.com`).\n\nRequired for static environment URLs declaration only (see doc).\n\nBy default review `environment.url` will be built as `${GCP_REVIEW_ENVIRONMENT_SCHEME}://${$CI_PROJECT_NAME}-${CI_ENVIRONMENT_SLUG}.${GCP_REVIEW_ENVIRONMENT_DOMAIN}`" + "name": "GCP_REVIEW_ENVIRONMENT_URL", + "type": "url", + "description": "The review environments url _(only define for static environment URLs declaration and if different from default)_", + "advanced": true }, { "name": "GCP_REVIEW_KEY_FILE", @@ -109,7 +111,8 @@ { "name": "GCP_INTEG_ENVIRONMENT_URL", "type": "url", - "description": "The integration environment url including scheme (ex: `https://my-application-integration.noprod-gcloud.domain.com`).\n\nRequired for static environment URLs declaration only (see doc).\n\nDo not use variable inside variable definition as it will result in a two level cascade variable and gitlab does not allow that." + "description": "The integration environment url _(only define for static environment URLs declaration and if different from default)_", + "advanced": true }, { "name": "GCP_INTEG_KEY_FILE", @@ -146,7 +149,8 @@ { "name": "GCP_STAGING_ENVIRONMENT_URL", "type": "url", - "description": "The staging environment url including scheme (ex: `https://my-application-staging.noprod-gcloud.domain.com`).\n\nRequired for static environment URLs declaration only (see doc).\n\nDo not use variable inside variable definition as it will result in a two level cascade variable and gitlab does not allow that." + "description": "The staging environment url _(only define for static environment URLs declaration and if different from default)_", + "advanced": true }, { "name": "GCP_STAGING_KEY_FILE", @@ -183,7 +187,8 @@ { "name": "GCP_PROD_ENVIRONMENT_URL", "type": "url", - "description": "The production environment url including scheme (ex: `https://my-application.gcloud.domain.com`).\n\nRequired for static environment URLs declaration only (see doc).\n\nDo not use variable inside variable definition as it will result in a two level cascade variable and gitlab does not allow that." + "description": "The production environment url _(only define for static environment URLs declaration and if different from default)_", + "advanced": true }, { "name": "AUTODEPLOY_TO_PROD", diff --git a/templates/gitlab-ci-gcloud.yml b/templates/gitlab-ci-gcloud.yml index 18bb27bf6517a27a8283fa147d1c5ec691c4d99e..f8531162a7e661dfd8543bfa19ba5bcce77991ab 100644 --- a/templates/gitlab-ci-gcloud.yml +++ b/templates/gitlab-ci-gcloud.yml @@ -254,9 +254,8 @@ stages: } function awkenvsubst() { - awk '{while(match($0,"[$]{[^}]*}")) {var=substr($0,RSTART+2,RLENGTH -3);val=ENVIRON[var];gsub(/["\\]/,"\\\\&", val);gsub("\n", "\\n", val);gsub("\r", "\\r", val);gsub("[$]{"var"}",val)}}1' + awk '{while(match($0,"[$%]{[^}]*}")) {g0=substr($0,RSTART,RLENGTH); val=ENVIRON[substr(g0,3,RLENGTH-3)]; gsub(g0,val)}}1' } - # Google Cloud Authentication function gcp_auth() { @@ -288,27 +287,33 @@ stages: # application deployment function function deploy() { - export env=$1 - export appname=$2 + export environment_type=$1 + export environment_name=$2 export gcp_project_id=$3 - export environment_url=$4 + environment_url=$4 + + # backwards compatibility + export env=$environment_type + export appname=$environment_name + + # variables expansion in $environment_url + environment_url=$(echo "$environment_url" | awkenvsubst) + export environment_url # extract hostname from $environment_url hostname=$(echo "$environment_url" | awk -F[/:] '{print $4}') export hostname - log_info "--- \\e[32mdeploy\\e[0m (env: \\e[33;1m${env}\\e[0m)" - log_info "--- \$appname: \\e[33;1m${appname}\\e[0m" - log_info "--- \$env: \\e[33;1m${env}\\e[0m" + log_info "--- \\e[32mdeploy\\e[0m" + log_info "--- \$environment_type: \\e[33;1m${environment_type}\\e[0m" + log_info "--- \$environment_name: \\e[33;1m${environment_name}\\e[0m" log_info "--- \$hostname: \\e[33;1m${hostname}\\e[0m" log_info "--- \$gcp_project_id: \\e[33;1m${gcp_project_id}\\e[0m" # unset any upstream deployment env & artifacts - unset environment_name - unset environment_type rm -f gcloud.env rm -f environment_url.txt - deployscript=$(ls -1 "$GCP_SCRIPTS_DIR/gcp-deploy-${env}.sh" 2>/dev/null || ls -1 "$GCP_SCRIPTS_DIR/gcp-deploy.sh" 2>/dev/null || echo "") + deployscript=$(ls -1 "$GCP_SCRIPTS_DIR/gcp-deploy-${environment_type}.sh" 2>/dev/null || ls -1 "$GCP_SCRIPTS_DIR/gcp-deploy.sh" 2>/dev/null || echo "") if [[ -f "$deployscript" ]] then log_info "--- deploy script (\\e[33;1m${deployscript}\\e[0m) found: execute" @@ -328,21 +333,25 @@ stages: else echo "$environment_url" > environment_url.txt fi - echo -e "environment_type=$env\\nenvironment_name=$appname\\nenvironment_url=$environment_url" > gcloud.env + echo -e "environment_type=$environment_type\\nenvironment_name=$environment_name\\nenvironment_url=$environment_url" > gcloud.env } # environment cleanup function function delete() { - export env=$1 - export appname=$2 + export environment_type=$1 + export environment_name=$2 export gcp_project_id=$3 - log_info "--- \\e[32mdelete\\e[0m (env: ${env})" - log_info "--- \$appname: \\e[33;1m${appname}\\e[0m" - log_info "--- \$env: \\e[33;1m${env}\\e[0m" + # backwards compatibility + export env=$environment_type + export appname=$environment_name + + log_info "--- \\e[32mdelete" + log_info "--- \$environment_type: \\e[33;1m${environment_type}\\e[0m" + log_info "--- \$environment_name: \\e[33;1m${environment_name}\\e[0m" log_info "--- \$gcp_project_id: \\e[33;1m${gcp_project_id}\\e[0m" - cleanupscript=$(ls -1 "$GCP_SCRIPTS_DIR/gcp-cleanup-${env}.sh" 2>/dev/null || ls -1 "$GCP_SCRIPTS_DIR/gcp-cleanup.sh" 2>/dev/null || echo "") + cleanupscript=$(ls -1 "$GCP_SCRIPTS_DIR/gcp-cleanup-${environment_type}.sh" 2>/dev/null || ls -1 "$GCP_SCRIPTS_DIR/gcp-cleanup.sh" 2>/dev/null || echo "") if [[ -f "$cleanupscript" ]] then log_info "--- cleanup script (\\e[33;1m${cleanupscript}\\e[0m) found: execute" @@ -394,7 +403,7 @@ stages: - gcp_auth "${ENV_KEY_FILE:-$GCP_KEY_FILE}" "${ENV_OIDC_PROVIDER:-$GCP_OIDC_PROVIDER}" "${ENV_OIDC_ACCOUNT:-$GCP_OIDC_ACCOUNT}" script: - - deploy "$ENV_TYPE" "${ENV_APP_NAME:-${GCP_BASE_APP_NAME}${ENV_APP_SUFFIX}}" "$ENV_PROJECT" "$ENV_URL" + - deploy "$ENV_TYPE" "${ENV_APP_NAME:-${GCP_BASE_APP_NAME}${ENV_APP_SUFFIX}}" "$ENV_PROJECT" "${ENV_URL:-${GCP_ENVIRONMENT_URL:-$ENV_URL_LEGACY}}" artifacts: name: "$ENV_TYPE env url for $CI_PROJECT_NAME on $CI_COMMIT_REF_SLUG" paths: @@ -440,7 +449,8 @@ gcp-review: ENV_OIDC_PROVIDER: "$GCP_REVIEW_OIDC_PROVIDER" ENV_OIDC_ACCOUNT: "$GCP_REVIEW_OIDC_ACCOUNT" ENV_KEY_FILE: "$GCP_REVIEW_KEY_FILE" - ENV_URL: "${GCP_REVIEW_ENVIRONMENT_SCHEME}://${CI_PROJECT_NAME}-${CI_ENVIRONMENT_SLUG}.${GCP_REVIEW_ENVIRONMENT_DOMAIN}" + ENV_URL: "${GCP_REVIEW_ENVIRONMENT_URL}" + ENV_URL_LEGACY: "${GCP_REVIEW_ENVIRONMENT_SCHEME}://${CI_PROJECT_NAME}-${CI_ENVIRONMENT_SLUG}.${GCP_REVIEW_ENVIRONMENT_DOMAIN}" environment: name: review/$CI_COMMIT_REF_NAME on_stop: gcp-cleanup-review