# GitLab CI template for Google Cloud Platform This project implements a GitLab CI/CD template to deploy your application to the [Google Cloud](https://cloud.google.com/) platform. ## Usage In order to include this template in your project, add the following to your `gitlab-ci.yml`: ```yaml include: - project: 'to-be-continuous/gcloud' ref: '3.1.0' file: '/templates/gitlab-ci-gcloud.yml' ``` ## Understand This chapter introduces key notions and principle to understand how this template works. ### Managed deployment environments This template implements continuous delivery/continuous deployment for projects hosted on Google Cloud Platform. It allows you to manage automatic deployment & cleanup of standard predefined environments. Each environment can be enabled/disabled by configuration. If you're not satisfied with predefined environments and/or their associated Git workflow, you may implement you own environments and workflow, by reusing/extending the base (hidden) jobs. This is advanced usage and will not be covered by this documentation. The following chapters present the managed predefined environments and their associated Git workflow. #### Review environments The template supports **review** environments: those are dynamic and ephemeral environments to deploy your _ongoing developments_ (a.k.a. _feature_ or _topic_ branches). When enabled, it deploys the result from upstream build stages to a dedicated and temporary environment. It is only active for non-production, non-integration branches. It is a strict equivalent of GitLab's [Review Apps](https://docs.gitlab.com/ee/ci/review_apps/) feature. It also comes with a _cleanup_ job (accessible either from the _environments_ page, or from the pipeline view). #### Integration environment If you're using a Git Workflow with an integration branch (such as [Gitflow](https://www.atlassian.com/git/tutorials/comparing-workflows/gitflow-workflow)), the template supports an **integration** environment. When enabled, it deploys the result from upstream build stages to a dedicated environment. It is only active for your integration branch (`develop` by default). #### Production environments Lastly, the template supports 2 environments associated to your production branch (`master` or `main` by default): * a **staging** environment (an iso-prod environment meant for testing and validation purpose), * the **production** environment. You're free to enable whichever or both, and you can also choose your deployment-to-production policy: * **continuous deployment**: automatic deployment to production (when the upstream pipeline is successful), * **continuous delivery**: deployment to production can be triggered manually (when the upstream pipeline is successful). ### Supported authentication methods The Google Cloud Platform template supports two kinds of authentication: 1. basic authentication with [Service Account key file](https://cloud.google.com/bigquery/docs/authentication/service-account-file), 2. or [federated authentication using OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/). #### Service account authentication To use this authentication method, simply generate and provide [Service Account key file](https://cloud.google.com/bigquery/docs/authentication/service-account-file) as secret GitLab CI/CD variables (of type File), using the appropriate variables (see doc below). Can be provided globally and/or per environment. #### Federated authentication using OpenID Connect The GCP template supports a [federated authentication using OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/). If you wish to use this authentication mode, please follow carefully [the GitLab guide](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/), then configure appropriately the related variables: * `GPC_OIDC_PROVIDER` / `GPC_OIDC_ACCOUNT` for any global/common access, * `GPC_<env>_OIDC_PROVIDER` / `GPC_<env>_OIDC_ACCOUNT` if you wish to use separate settings with any of your environments. The `GPC_OIDC_PROVIDER` & `GPC_<env>_OIDC_PROVIDER` variable shall be of the form: ``` projects/<PROJECT_NUMBER>/locations/global/workloadIdentityPools/<POOL_ID>/providers/<PROVIDER_ID> ``` The following commands may help you retrieve the different values: - `gcloud projects describe $GCP_PROJECT --format="value(projectNumber)"` will return the `PROJECT_NUMBER` value - `gcloud iam workload-identity-pools list --location=global --format="value(name)"` will list you POOL_IDs available on your `GCP_PROJECT` - `gcloud iam workload-identity-pools providers list --workload-identity-pool=<my-pool> --location=global --format="value(name)"` will return the list of available `PROVIDER_ID` for one `POOL_ID` The template supports two ways to retrieve the JSON web token (JWT): * using GitLab's `CI_JOB_JWT_V2` variable - _default_<br/> :warning: deprecated, scheduled to be removed in GitLab 16.5 * using an [ID token](https://docs.gitlab.com/ee/ci/yaml/index.html#id_tokens) named `GCP_JWT` - _configurable_ The ID token can be configured as follows in your `.gitlab-ci.yml` file: ```yaml # enable GCP_JWT ID token for GCP jobs only .gcp-base: id_tokens: GCP_JWT: # use your own audience url here aud: https://gitlab.com ``` ### Deployment context variables In order to manage the various deployment environments, this template provides a couple of **dynamic variables** that you might use in your hook scripts, deployment manifests and other deployment resources: * `${environment_type}`: the current deployment environment type (`review`, `integration`, `staging` or `production`) * `${environment_name}`: a generated application name to use for the current deployment environment (ex: `myapp-review-fix-bug-12` or `myapp-staging`) - _details below_ #### Generated environment name The `${environment_name}` variable is generated to designate each deployment environment with a unique and meaningful application name. By construction, it is suitable for inclusion in DNS, URLs, Kubernetes labels... It is built from: * the application _base name_ (defaults to `$CI_PROJECT_NAME` but can be overridden globally and/or per deployment environment - _see configuration variables_) * GitLab predefined `$CI_ENVIRONMENT_SLUG` variable ([sluggified](https://en.wikipedia.org/wiki/Clean_URL#Slug) name, truncated to 24 characters) The `${environment_name}` variable is then evaluated as: * `<app base name>` for the production environment * `<app base name>-$CI_ENVIRONMENT_SLUG` for all other deployment environments * :bulb: `${environment_name}` can also be overriden per environment with the appropriate configuration variable Examples (with an application's base name `myapp`): | `$environment_type` | Branch | `$CI_ENVIRONMENT_SLUG` | `$environment_name` | |---------------------|---------------|-------------------------|---------------------| | `review` | `feat/blabla` | `review-feat-bla-xmuzs6`| `myapp-review-feat-bla-xmuzs6` | | `integration` | `develop` | `integration` | `myapp-integration` | | `staging` | `main` | `staging` | `myapp-staging` | | `production` | `main` | `production` | `myapp` | ### Deployment and cleanup scripts The Google Cloud template requires you to provide a shell script that fully implements your application deployment and cleanup using the [`gcloud` CLI](https://cloud.google.com/sdk/gcloud) and all other tools available in the selected Docker image. The deployment script is searched as follows: 1. look for a specific `gcp-deploy-$environment_type.sh` in the `$GCP_SCRIPTS_DIR` directory in your project (e.g. `gcp-deploy-staging.sh` for staging environment), 2. if not found: look for a default `gcp-deploy.sh` in the `$GCP_SCRIPTS_DIR` directory in your project, 3. if not found: the deployment job will fail. The cleanup script is searched as follows: 1. look for a specific `gcp-cleanup-$environment_type.sh` in the `$GCP_SCRIPTS_DIR` directory in your project (e.g. `gcp-cleanup-staging.sh` for staging environment), 2. if not found: look for a default `gcp-cleanup.sh` in the `$GCP_SCRIPTS_DIR` directory in your project, 3. if not found: the cleanup job will fail. > :information_source: Your deployment (and cleanup) scripts have to be able to cope with various environments, each with different application names, exposed routes, settings, ... > Part of this complexity can be handled by the lookup policies described above (ex: one script per env) and also by using available environment variables: > > 1. [deployment context variables](#deployment-context-variables) provided by the template: > * `${environment_type}`: the current environment type (`review`, `integration`, `staging` or `production`) > * `${environment_name}`: the application name to use for the current environment (ex: `myproject-review-fix-bug-12` or `myproject-staging`) > * `${hostname}`: the environment hostname, extracted from the current environment url (after late variable expansion - see below) > 2. any [GitLab CI variable](https://docs.gitlab.com/ee/ci/variables/predefined_variables.html) > 3. any [custom variable](https://docs.gitlab.com/ee/ci/variables/#add-a-cicd-variable-to-a-project) > (ex: `${SECRET_TOKEN}` that you have set in your project CI/CD variables) ### Environments URL management The GCP template supports two ways of providing your environments url: * a **static way**: when the environments url can be determined in advance, probably because you're exposing your routes through a DNS you manage, * a [**dynamic way**](https://docs.gitlab.com/ee/ci/environments/#set-dynamic-environment-urls-after-a-job-finishes): when the url cannot be known before the deployment job is executed. The **static way** can be implemented simply by setting the appropriate configuration variable(s) depending on the environment (see environments configuration chapters): * `$GCP_ENVIRONMENT_URL` to define a default url pattern for all your envs, * `$GCP_REVIEW_ENVIRONMENT_URL`, `$GCP_INTEG_ENVIRONMENT_URL`, `$GCP_STAGING_ENVIRONMENT_URL` and `$GCP_PROD_ENVIRONMENT_URL` to override the default. > :information_source: Each of those variables support a **late variable expansion mechanism** with the `%{somevar}` syntax, > allowing you to use any dynamically evaluated variables such as `${environment_name}`. > > Example: > > ```yaml > variables: > GCP_BASE_APP_NAME: "wonderapp" > # global url for all environments > GCP_ENVIRONMENT_URL: "https://%{environment_name}.nonprod.acme.domain" > # override for prod (late expansion of $GCP_BASE_APP_NAME not needed here) > GCP_PROD_ENVIRONMENT_URL: "https://$GCP_BASE_APP_NAME.acme.domain" > # override for review (using separate resource paths) > GCP_REVIEW_ENVIRONMENT_URL: "https://wonderapp-review.nonprod.acme.domain/%{environment_name}" > ``` To implement the **dynamic way**, your deployment script shall simply generate a `environment_url.txt` file in the working directory, containing only the dynamically generated url. When detected by the template, it will use it as the newly deployed environment url. ### Deployment output variables Each deployment job produces _output variables_ that are propagated to downstream jobs (using [dotenv artifacts](https://docs.gitlab.com/ee/ci/pipelines/job_artifacts.html#artifactsreportsdotenv)): * `$environment_type`: set to the type of environment (`review`, `integration`, `staging` or `production`), * `$environment_name`: the application name (see below), * `$environment_url`: set to the environment URL (whether determined statically or dynamically). Those variables may be freely used in downstream jobs (for instance to run acceptance tests against the latest deployed environment). ## Confuguration reference ### Secrets management Here are some advices about your **secrets** (variables marked with a :lock:): 1. Manage them as [project or group CI/CD variables](https://docs.gitlab.com/ee/ci/variables/#add-a-cicd-variable-to-a-project): * [**masked**](https://docs.gitlab.com/ee/ci/variables/#mask-a-cicd-variable) to prevent them from being inadvertently displayed in your job logs, * [**protected**](https://docs.gitlab.com/ee/ci/variables/#protected-cicd-variables) if you want to secure some secrets you don't want everyone in the project to have access to (for instance production secrets). 2. In case a secret contains [characters that prevent it from being masked](https://docs.gitlab.com/ee/ci/variables/#mask-a-cicd-variable), simply define its value as the [Base64](https://en.wikipedia.org/wiki/Base64) encoded value prefixed with `@b64@`: it will then be possible to mask it and the template will automatically decode it prior to using it. 3. Don't forget to escape special characters (ex: `$` -> `$$`). ### Global configuration The Google Cloud template uses some global configuration used throughout all jobs. | Name | description | default value | | ------------------------ | -------------------------------------- | ----------------- | | `GCP_CLI_IMAGE` | the Docker image used to run Google Cloud CLI commands| `gcr.io/google.com/cloudsdktool/cloud-sdk:latest` | | :lock: `GCP_KEY_FILE` | Default [Service Account key file](https://cloud.google.com/bigquery/docs/authentication/service-account-file) | _none_ | | `GCP_OIDC_PROVIDER` | Default Workload Identity Provider associated with GitLab to [authenticate with OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/) | none| | `GCP_OIDC_ACCOUNT` | Default Service Account to which impersonate with OpenID Connect authentication | none | | `GCP_BASE_APP_NAME` | Base application name | `$CI_PROJECT_NAME` ([see GitLab doc](https://docs.gitlab.com/ee/ci/variables/predefined_variables.html)) | | `GCP_ENVIRONMENT_URL` | Default environments url _(only define for static environment URLs declaration)_<br/>_supports late variable expansion (ex: `https://%{environment_name}.gcloud.acme.com`)_ | _none_ | | `GCP_SCRIPTS_DIR` | Directory where Google Cloud scripts (deploy & cleanup) are located | `.` _(root project dir)_ | ### Review environments configuration Review environments are dynamic and ephemeral environments to deploy your _ongoing developments_ (a.k.a. _feature_ or _topic_ branches). They are **disabled by default** and can be enabled by setting the `GCP_REVIEW_PROJECT` variable (see below). Here are variables supported to configure review environments: | Name | description | default value | | ------------------------ | -------------------------------------- | ----------------- | | `GCP_REVIEW_PROJECT` | Google Cloud project ID for `review` env | _none_ (disabled) | | :lock: `GCP_REVIEW_KEY_FILE`| [Service Account key file](https://cloud.google.com/bigquery/docs/authentication/service-account-file) to authenticate on `review` env _(only define if different from default)_ | `$GCP_KEY_FILE` | | `GCP_REVIEW_APP_NAME` | Application name for `review` env | `"${GCP_BASE_APP_NAME}-${CI_ENVIRONMENT_SLUG}"` (ex: `myproject-review-fix-bug-12`) | | `GCP_REVIEW_ENVIRONMENT_URL`| The review environments url _(only define for static environment URLs declaration and if different from default)_ | `$GCP_ENVIRONMENT_URL` | | `GCP_REVIEW_OIDC_PROVIDER` | Workload Identity Provider associated with GitLab to [authenticate with OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/) on `review` environment | none| | `GCP_REVIEW_OIDC_ACCOUNT` | Service Account to which impersonate with OpenID Connect authentication on `review` environment | none | ### Integration environment configuration The integration environment is the environment associated to your integration branch (`develop` by default). It is **disabled by default** and can be enabled by setting the `GCP_INTEG_PROJECT` variable (see below). Here are variables supported to configure the integration environment: | Name | description | default value | | ------------------------ | -------------------------------------- | ----------------- | | `GCP_INTEG_PROJECT` | Google Cloud project ID for `integration` env | _none_ (disabled) | | :lock: `GCP_INTEG_KEY_FILE`|[Service Account key file](https://cloud.google.com/bigquery/docs/authentication/service-account-file) to authenticate on `integration` env _(only define if different from default)_ | `$GCP_KEY_FILE` | | `GCP_INTEG_APP_NAME` | Application name for `integration` env | `${GCP_BASE_APP_NAME}-integration` | | `GCP_INTEG_ENVIRONMENT_URL`| The integration environment url _(only define for static environment URLs declaration and if different from default)_ | `$GCP_ENVIRONMENT_URL` | | `GCP_INTEG_OIDC_PROVIDER` | Workload Identity Provider associated with GitLab to [authenticate with OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/) on `integration` environment | none| | `GCP_INTEG_OIDC_ACCOUNT` | Service Account to which impersonate with OpenID Connect authentication on `integration` environment | none | ### Staging environment configuration The staging environment is an iso-prod environment meant for testing and validation purpose associated to your production branch (`master` by default). It is **disabled by default** and can be enabled by setting the `GCP_STAGING_PROJECT` variable (see below). Here are variables supported to configure the staging environment: | Name | description | default value | | ------------------------ | -------------------------------------- | ----------------- | | `GCP_STAGING_PROJECT` | Google Cloud project ID for `staging` env | _none_ (disabled) | | :lock: `GCP_STAGING_KEY_FILE`|[Service Account key file](https://cloud.google.com/bigquery/docs/authentication/service-account-file) to authenticate on `staging` env _(only define if different from default)_ | `$GCP_KEY_FILE` | | `GCP_STAGING_APP_NAME` | Application name for `staging` env | `${GCP_BASE_APP_NAME}-staging` | | `GCP_STAGING_ENVIRONMENT_URL`| The staging environment url _(only define for static environment URLs declaration and if different from default)_ | `$GCP_ENVIRONMENT_URL` | | `GCP_STAGING_OIDC_PROVIDER` | Workload Identity Provider associated with GitLab to [authenticate with OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/) on `staging` environment | none| | `GCP_STAGING_OIDC_ACCOUNT` | Service Account to which impersonate with OpenID Connect authentication on `staging` environment | none | ### Production environment configuration The production environment is the final deployment environment associated with your production branch (`master` by default). It is **disabled by default** and can be enabled by setting the `GCP_PROD_PROJECT` variable (see below). Here are variables supported to configure the production environment: | Name | description | default value | | ------------------------- | -------------------------------------- | ----------------- | | `GCP_PROD_PROJECT` | Google Cloud project ID for `production` env | _none_ (disabled) | | :lock: `GCP_PROD_KEY_FILE`|[Service Account key file](https://cloud.google.com/bigquery/docs/authentication/service-account-file) to authenticate on `production` env _(only define if different from default)_ | `$GCP_KEY_FILE` | | `GCP_PROD_APP_NAME` | Application name for `production` env | `$GCP_BASE_APP_NAME` | | `GCP_PROD_ENVIRONMENT_URL`| The production environment url _(only define for static environment URLs declaration and if different from default)_ | `$GCP_ENVIRONMENT_URL` | | `GCP_PROD_DEPLOY_STRATEGY`| Defines the deployment to production strategy. One of `manual` (i.e. _one-click_) or `auto`. | `manual` | | `GCP_PROD_OIDC_PROVIDER` | Workload Identity Provider associated with GitLab to [authenticate with OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/) on `production ` environment | none| | `GCP_PROD_OIDC_ACCOUNT` | Service Account to which impersonate with OpenID Connect authentication on `production ` environment | none | ## Examples ### Google AppEngine application #### Context Let's imagine a backend service: * named **coockedoodledoo**, * developped in whichever language, * part of project named **farmvoices** * hosted on Google AppEngine with project ID `farmvoices-12345` * with review, staging and production environments enabled. #### `.gitlab-ci.yml` ```yaml include: # Include Google Cloud template - project: 'to-be-continuous/gcloud' ref: '3.1.0' file: '/templates/gitlab-ci-gcloud.yml' ... # Global variables variables: ... # Google Cloud # GCP_KEY_FILE defined as secret CI/CD variable GCP_REVIEW_PROJECT: "farm-12345" # enable review env GCP_STAGING_PROJECT: "farm-12345" # enable staging env GCP_PROD_PROJECT: "farm-12345" # enable production env GCP_STAGING_ENVIRONMENT_URL: "https://staging-dot-coockedoodledoo-dot-farmvoices-12345.ew.r.appspot.com" GCP_PROD_ENVIRONMENT_URL: "https://coockedoodledoo-dot-farmvoices-12345.ew.r.appspot.com" # Postman REVIEW_ENABLED: "true" # Pipeline steps stages: - build - test - deploy - acceptance - publish - production # define review environment url (uses $CI_ENVIRONMENT_SLUG as app version) gcp-review: environment: url: "https://$CI_ENVIRONMENT_SLUG-dot-coockedoodledoo-dot-farmvoices-12345.ew.r.appspot.com" ``` #### AppEngine manifest ```yaml # Google AppEngine manifest # see: https://cloud.google.com/appengine/docs/standard/java11/config/appref runtime: TODO # depends on languages instance_class: F2 service: coockedoodledoo ... variables: # this is an example of hardcoded (non-sensitive) configuration variable SOME_CONFIG: "some-value" # this is an example of variabilized (secret) configuration variable # will be replaced programmatically during deployment SOME_SECRET: "${SOME_SECRET}" ``` #### hook scripts ##### `gcp-deploy.sh` This script is executed by the template to perform the application(s) deployment based on `gcloud` CLI. ```bash #!/usr/bin/env bash echo "[gcp-deploy] Deploy burger/$CI_ENVIRONMENT_SLUG..." # prepare GAE deployment directory (copy build output) mkdir -p gae cp build/* gae # copy manifest with variables substitution awk '{while(match($0,"[$]{[^}]*}")) {var=substr($0,RSTART+2,RLENGTH-3);gsub("[$]{"var"}",ENVIRON[var])}}1' < src/app.yaml > gae/app.yaml # gcloud deploy # use $CI_ENVIRONMENT_SLUG as the version cd gae if [[ "$CI_ENVIRONMENT_SLUG" == "production" ]] then promote_opt="--promote" else promote_opt="--no-promote" fi gcloud --quiet app deploy --project=${gcp_project_id} --version=${CI_ENVIRONMENT_SLUG} $promote_opt ``` ##### `gcp-cleanup.sh` This script is executed by the template to perform the application(s) cleanup based on `gcloud` CLI (review env only). ```bash #!/usr/bin/env bash echo "[gcp-cleanup] Cleanup burger/$CI_ENVIRONMENT_SLUG..." # use $CI_ENVIRONMENT_SLUG as the version gcloud --quiet app versions delete --project=${gcp_project_id} --service=coockedoodledoo ${CI_ENVIRONMENT_SLUG} ``` ## Variants The Google Cloud template can be used in conjunction with template variants to cover specific cases. ### Vault variant This variant allows delegating your secrets management to a [Vault](https://www.vaultproject.io/) server. #### Configuration In order to be able to communicate with the Vault server, the variant requires the additional configuration parameters: | Name | description | default value | | ----------------- | -------------------------------------- | ----------------- | | `TBC_VAULT_IMAGE` | The [Vault Secrets Provider](https://gitlab.com/to-be-continuous/tools/vault-secrets-provider) image to use (can be overridden) | `$CI_REGISTRY/to-be-continuous/tools/vault-secrets-provider:master` | | `VAULT_BASE_URL` | The Vault server base API url | _none_ | | :lock: `VAULT_ROLE_ID` | The [AppRole](https://www.vaultproject.io/docs/auth/approle) RoleID | **must be defined** | | :lock: `VAULT_SECRET_ID` | The [AppRole](https://www.vaultproject.io/docs/auth/approle) SecretID | **must be defined** | #### Usage Then you may retrieve any of your secret(s) from Vault using the following syntax: ```text @url@http://vault-secrets-provider/api/secrets/{secret_path}?field={field} ``` With: | Name | description | | -------------------------------- | -------------------------------------- | | `secret_path` (_path parameter_) | this is your secret location in the Vault server | | `field` (_query parameter_) | parameter to access a single basic field from the secret JSON payload | The variant supports two ways to retrieve the JSON web token (JWT): * using GitLab's `CI_JOB_JWT` variable - _default_<br/> :warning: deprecated, scheduled to be removed in GitLab 16.5 * using an [ID token](https://docs.gitlab.com/ee/ci/yaml/index.html#id_tokens) named `VAULT_JWT_TOKEN` - _configurable_ The ID token can be configured as follows in your `.gitlab-ci.yml` file: ```yaml # enable VAULT_JWT_TOKEN ID token for GCP jobs only .gcp-base: id_tokens: VAULT_JWT_TOKEN: # use your own audience url here aud: https://gitlab.com ``` #### Example ```yaml include: # main template - project: 'to-be-continuous/gcloud' ref: '3.1.0' file: '/templates/gitlab-ci-gcloud.yml' # Vault variant - project: 'to-be-continuous/gcloud' ref: '3.1.0' file: '/templates/gitlab-ci-gcloud-vault.yml' variables: # Secrets managed by Vault SOME_SECRET_USED_IN_MY_APP: "@url@http://vault-secrets-provider/api/secrets/b7ecb6ebabc231/prod/gcloud/secret?field=my.app.secret" VAULT_BASE_URL: "https://vault.acme.host/v1" # $VAULT_ROLE_ID and $VAULT_SECRET_ID defined as a secret CI/CD variable # enable VAULT_JWT_TOKEN ID token for GCP jobs only .gcp-base: id_tokens: VAULT_JWT_TOKEN: # use your own audience url here aud: https://gitlab.com ```