Newer
Older
This project implements a GitLab CI/CD template to deploy your application to the [Google Cloud](https://cloud.google.com/) platform.
## Usage
In order to include this template in your project, add the following to your `gitlab-ci.yml`:
```yaml
include:
- project: 'to-be-continuous/gcloud'
file: '/templates/gitlab-ci-gcloud.yml'
```
## Understand
This chapter introduces key notions and principle to understand how this template works.
### Managed deployment environments
This template implements continuous delivery/continuous deployment for projects hosted on Google Cloud Platform.
It allows you to manage automatic deployment & cleanup of standard predefined environments.
Each environment can be enabled/disabled by configuration.
If you're not satisfied with predefined environments and/or their associated Git workflow, you may implement you own environments and
workflow, by reusing/extending the base (hidden) jobs. This is advanced usage and will not be covered by this documentation.
The following chapters present the managed predefined environments and their associated Git workflow.
The template supports **review** environments: those are dynamic and ephemeral environments to deploy your
_ongoing developments_ (a.k.a. _feature_ or _topic_ branches).
When enabled, it deploys the result from upstream build stages to a dedicated and temporary environment.
It is only active for non-production, non-integration branches.
It is a strict equivalent of GitLab's [Review Apps](https://docs.gitlab.com/ee/ci/review_apps/) feature.
It also comes with a _cleanup_ job (accessible either from the _environments_ page, or from the pipeline view).
If you're using a Git Workflow with an integration branch (such as [Gitflow](https://www.atlassian.com/git/tutorials/comparing-workflows/gitflow-workflow)),
the template supports an **integration** environment.
When enabled, it deploys the result from upstream build stages to a dedicated environment.
It is only active for your integration branch (`develop` by default).
Lastly, the template supports 2 environments associated to your production branch (`master` or `main` by default):
* a **staging** environment (an iso-prod environment meant for testing and validation purpose),
* the **production** environment.
You're free to enable whichever or both, and you can also choose your deployment-to-production policy:
* **continuous deployment**: automatic deployment to production (when the upstream pipeline is successful),
* **continuous delivery**: deployment to production can be triggered manually (when the upstream pipeline is successful).
The Google Cloud Platform template supports two kinds of authentication:
1. basic authentication with [Service Account key file](https://cloud.google.com/bigquery/docs/authentication/service-account-file),
2. or [federated authentication using OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/).
To use this authentication method, simply generate and provide [Service Account key file](https://cloud.google.com/bigquery/docs/authentication/service-account-file) as secret GitLab CI/CD variables (of type File), using the appropriate variables (see doc below).
The GCP template supports a [federated authentication using OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/).
If you wish to use this authentication mode, please follow carefully [the GitLab guide](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/), then configure appropriately the related variables:
* `GPC_OIDC_PROVIDER` / `GPC_OIDC_ACCOUNT` for any global/common access,
* `GPC_<env>_OIDC_PROVIDER` / `GPC_<env>_OIDC_ACCOUNT` if you wish to use separate settings with any of your environments.
The `GPC_OIDC_PROVIDER` & `GPC_<env>_OIDC_PROVIDER` variable shall be of the form:
```
projects/<PROJECT_NUMBER>/locations/global/workloadIdentityPools/<POOL_ID>/providers/<PROVIDER_ID>
```
The following commands may help you retrieve the different values:
- `gcloud projects describe $GCP_PROJECT --format="value(projectNumber)"` will return the `PROJECT_NUMBER` value
- `gcloud iam workload-identity-pools list --location=global --format="value(name)"` will list you POOL_IDs available on your `GCP_PROJECT`
- `gcloud iam workload-identity-pools providers list --workload-identity-pool=<my-pool> --location=global --format="value(name)"` will return the list of available `PROVIDER_ID` for one `POOL_ID`
### Deployment context variables
In order to manage the various deployment environments, this template provides a couple of **dynamic variables**
that you might use in your hook scripts, deployment manifests and other deployment resources:
* `${environment_type}`: the current deployment environment type (`review`, `integration`, `staging` or `production`)
* `${environment_name}`: a generated application name to use for the current deployment environment (ex: `myapp-review-fix-bug-12` or `myapp-staging`) - _details below_
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
#### Generated environment name
The `${environment_name}` variable is generated to designate each deployment environment with a unique and meaningful application name.
By construction, it is suitable for inclusion in DNS, URLs, Kubernetes labels...
It is built from:
* the application _base name_ (defaults to `$CI_PROJECT_NAME` but can be overridden globally and/or per deployment environment - _see configuration variables_)
* GitLab predefined `$CI_ENVIRONMENT_SLUG` variable ([sluggified](https://en.wikipedia.org/wiki/Clean_URL#Slug) name, truncated to 24 characters)
The `${environment_name}` variable is then evaluated as:
* `<app base name>` for the production environment
* `<app base name>-$CI_ENVIRONMENT_SLUG` for all other deployment environments
* :bulb: `${environment_name}` can also be overriden per environment with the appropriate configuration variable
Examples (with an application's base name `myapp`):
| `$environment_type` | Branch | `$CI_ENVIRONMENT_SLUG` | `$environment_name` |
|---------------------|---------------|-------------------------|---------------------|
| `review` | `feat/blabla` | `review-feat-bla-xmuzs6`| `myapp-review-feat-bla-xmuzs6` |
| `integration` | `develop` | `integration` | `myapp-integration` |
| `staging` | `main` | `staging` | `myapp-staging` |
| `production` | `main` | `production` | `myapp` |
### Deployment and cleanup scripts
The Google Cloud template requires you to provide a shell script that fully implements your application
deployment and cleanup using the [`gcloud` CLI](https://cloud.google.com/sdk/gcloud) and all other tools available in the selected Docker image.
The deployment script is searched as follows:
1. look for a specific `gcp-deploy-$environment_type.sh` in the `$GCP_SCRIPTS_DIR` directory in your project (e.g. `gcp-deploy-staging.sh` for staging environment),
2. if not found: look for a default `gcp-deploy.sh` in the `$GCP_SCRIPTS_DIR` directory in your project,
3. if not found: the deployment job will fail.
The cleanup script is searched as follows:
1. look for a specific `gcp-cleanup-$environment_type.sh` in the `$GCP_SCRIPTS_DIR` directory in your project (e.g. `gcp-cleanup-staging.sh` for staging environment),
2. if not found: look for a default `gcp-cleanup.sh` in the `$GCP_SCRIPTS_DIR` directory in your project,
3. if not found: the cleanup job will fail.
> :information_source: Your deployment (and cleanup) scripts have to be able to cope with various environments, each with different application names, exposed routes, settings, ...
> Part of this complexity can be handled by the lookup policies described above (ex: one script per env) and also by using available environment variables:
>
> 1. [deployment context variables](#deployment-context-variables) provided by the template:
> * `${environment_type}`: the current environment type (`review`, `integration`, `staging` or `production`)
> * `${environment_name}`: the application name to use for the current environment (ex: `myproject-review-fix-bug-12` or `myproject-staging`)
> * `${hostname}`: the environment hostname, extracted from the current environment url (after late variable expansion - see below)
> 2. any [GitLab CI variable](https://docs.gitlab.com/ee/ci/variables/predefined_variables.html)
> 3. any [custom variable](https://docs.gitlab.com/ee/ci/variables/#add-a-cicd-variable-to-a-project)
> (ex: `${SECRET_TOKEN}` that you have set in your project CI/CD variables)
The GCP template supports two ways of providing your environments url:
* a **static way**: when the environments url can be determined in advance, probably because you're exposing your routes through a DNS you manage,
* a [**dynamic way**](https://docs.gitlab.com/ee/ci/environments/#set-dynamic-environment-urls-after-a-job-finishes): when the url cannot be known before the
deployment job is executed.
The **static way** can be implemented simply by setting the appropriate configuration variable(s) depending on the environment (see environments configuration chapters):
* `$GCP_ENVIRONMENT_URL` to define a default url pattern for all your envs,
* `$GCP_REVIEW_ENVIRONMENT_URL`, `$GCP_INTEG_ENVIRONMENT_URL`, `$GCP_STAGING_ENVIRONMENT_URL` and `$GCP_PROD_ENVIRONMENT_URL` to override the default.
> :information_source: Each of those variables support a **late variable expansion mechanism** with the `%{somevar}` syntax,
> allowing you to use any dynamically evaluated variables such as `${environment_name}`.
>
> Example:
>
> ```yaml
> variables:
> GCP_BASE_APP_NAME: "wonderapp"
> # global url for all environments
> GCP_ENVIRONMENT_URL: "https://%{environment_name}.nonprod.acme.domain"
> # override for prod (late expansion of $GCP_BASE_APP_NAME not needed here)
> GCP_PROD_ENVIRONMENT_URL: "https://$GCP_BASE_APP_NAME.acme.domain"
> # override for review (using separate resource paths)
> GCP_REVIEW_ENVIRONMENT_URL: "https://wonderapp-review.nonprod.acme.domain/%{environment_name}"
> ```
To implement the **dynamic way**, your deployment script shall simply generate a `environment_url.txt` file in the working directory, containing only
the dynamically generated url. When detected by the template, it will use it as the newly deployed environment url.
Each deployment job produces _output variables_ that are propagated to downstream jobs (using [dotenv artifacts](https://docs.gitlab.com/ee/ci/pipelines/job_artifacts.html#artifactsreportsdotenv)):
* `$environment_type`: set to the type of environment (`review`, `integration`, `staging` or `production`),
* `$environment_name`: the application name (see below),
* `$environment_url`: set to the environment URL (whether determined statically or dynamically).
Those variables may be freely used in downstream jobs (for instance to run acceptance tests against the latest deployed environment).
Here are some advices about your **secrets** (variables marked with a :lock:):
1. Manage them as [project or group CI/CD variables](https://docs.gitlab.com/ee/ci/variables/#add-a-cicd-variable-to-a-project):
* [**masked**](https://docs.gitlab.com/ee/ci/variables/#mask-a-cicd-variable) to prevent them from being inadvertently
* [**protected**](https://docs.gitlab.com/ee/ci/variables/#protected-cicd-variables) if you want to secure some secrets
you don't want everyone in the project to have access to (for instance production secrets).
2. In case a secret contains [characters that prevent it from being masked](https://docs.gitlab.com/ee/ci/variables/#mask-a-cicd-variable),
simply define its value as the [Base64](https://en.wikipedia.org/wiki/Base64) encoded value prefixed with `@b64@`:
it will then be possible to mask it and the template will automatically decode it prior to using it.
3. Don't forget to escape special characters (ex: `$` -> `$$`).
### Global configuration
The Google Cloud template uses some global configuration used throughout all jobs.
| Name | description | default value |
| ------------------------ | -------------------------------------- | ----------------- |
| `GCP_CLI_IMAGE` | the Docker image used to run Google Cloud CLI commands| `gcr.io/google.com/cloudsdktool/cloud-sdk:latest` |
| :lock: `GCP_KEY_FILE` | Default [Service Account key file](https://cloud.google.com/bigquery/docs/authentication/service-account-file) | _none_ |
| `GCP_OIDC_PROVIDER` | Default Workload Identity Provider associated with GitLab to [authenticate with OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/) | none|
| `GCP_OIDC_ACCOUNT` | Default Service Account to which impersonate with OpenID Connect authentication | none |
| `GCP_BASE_APP_NAME` | Base application name | `$CI_PROJECT_NAME` ([see GitLab doc](https://docs.gitlab.com/ee/ci/variables/predefined_variables.html)) |
| `GCP_ENVIRONMENT_URL` | Default environments url _(only define for static environment URLs declaration)_<br/>_supports late variable expansion (ex: `https://%{environment_name}.gcloud.acme.com`)_ | _none_ |
| `GCP_SCRIPTS_DIR` | Directory where Google Cloud scripts (deploy & cleanup) are located | `.` _(root project dir)_ |
### Review environments configuration
Review environments are dynamic and ephemeral environments to deploy your _ongoing developments_ (a.k.a. _feature_ or
_topic_ branches).
They are **disabled by default** and can be enabled by setting the `GCP_REVIEW_PROJECT` variable (see below).
Here are variables supported to configure review environments:
| Name | description | default value |
| ------------------------ | -------------------------------------- | ----------------- |
| `GCP_REVIEW_PROJECT` | Google Cloud project ID for `review` env | _none_ (disabled) |
| :lock: `GCP_REVIEW_KEY_FILE`| [Service Account key file](https://cloud.google.com/bigquery/docs/authentication/service-account-file) to authenticate on `review` env _(only define if different from default)_ | `$GCP_KEY_FILE` |
| `GCP_REVIEW_APP_NAME` | Application name for `review` env | `"${GCP_BASE_APP_NAME}-${CI_ENVIRONMENT_SLUG}"` (ex: `myproject-review-fix-bug-12`) |
| `GCP_REVIEW_ENVIRONMENT_URL`| The review environments url _(only define for static environment URLs declaration and if different from default)_ | `$GCP_ENVIRONMENT_URL` |
| `GCP_REVIEW_OIDC_PROVIDER` | Workload Identity Provider associated with GitLab to [authenticate with OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/) on `review` environment | none|
| `GCP_REVIEW_OIDC_ACCOUNT` | Service Account to which impersonate with OpenID Connect authentication on `review` environment | none |
The integration environment is the environment associated to your integration branch (`develop` by default).
It is **disabled by default** and can be enabled by setting the `GCP_INTEG_PROJECT` variable (see below).
Here are variables supported to configure the integration environment:
| Name | description | default value |
| ------------------------ | -------------------------------------- | ----------------- |
| `GCP_INTEG_PROJECT` | Google Cloud project ID for `integration` env | _none_ (disabled) |
| :lock: `GCP_INTEG_KEY_FILE`|[Service Account key file](https://cloud.google.com/bigquery/docs/authentication/service-account-file) to authenticate on `integration` env _(only define if different from default)_ | `$GCP_KEY_FILE` |
| `GCP_INTEG_APP_NAME` | Application name for `integration` env | `${GCP_BASE_APP_NAME}-integration` |
| `GCP_INTEG_ENVIRONMENT_URL`| The integration environment url _(only define for static environment URLs declaration and if different from default)_ | `$GCP_ENVIRONMENT_URL` |
| `GCP_INTEG_OIDC_PROVIDER` | Workload Identity Provider associated with GitLab to [authenticate with OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/) on `integration` environment | none|
| `GCP_INTEG_OIDC_ACCOUNT` | Service Account to which impersonate with OpenID Connect authentication on `integration` environment | none |
The staging environment is an iso-prod environment meant for testing and validation purpose associated to your production
branch (`master` by default).
It is **disabled by default** and can be enabled by setting the `GCP_STAGING_PROJECT` variable (see below).
Here are variables supported to configure the staging environment:
| Name | description | default value |
| ------------------------ | -------------------------------------- | ----------------- |
| `GCP_STAGING_PROJECT` | Google Cloud project ID for `staging` env | _none_ (disabled) |
| :lock: `GCP_STAGING_KEY_FILE`|[Service Account key file](https://cloud.google.com/bigquery/docs/authentication/service-account-file) to authenticate on `staging` env _(only define if different from default)_ | `$GCP_KEY_FILE` |
| `GCP_STAGING_APP_NAME` | Application name for `staging` env | `${GCP_BASE_APP_NAME}-staging` |
| `GCP_STAGING_ENVIRONMENT_URL`| The staging environment url _(only define for static environment URLs declaration and if different from default)_ | `$GCP_ENVIRONMENT_URL` |
| `GCP_STAGING_OIDC_PROVIDER` | Workload Identity Provider associated with GitLab to [authenticate with OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/) on `staging` environment | none|
| `GCP_STAGING_OIDC_ACCOUNT` | Service Account to which impersonate with OpenID Connect authentication on `staging` environment | none |
The production environment is the final deployment environment associated with your production branch (`master` by default).
It is **disabled by default** and can be enabled by setting the `GCP_PROD_PROJECT` variable (see below).
Here are variables supported to configure the production environment:
| Name | description | default value |
| ------------------------- | -------------------------------------- | ----------------- |
| `GCP_PROD_PROJECT` | Google Cloud project ID for `production` env | _none_ (disabled) |
| :lock: `GCP_PROD_KEY_FILE`|[Service Account key file](https://cloud.google.com/bigquery/docs/authentication/service-account-file) to authenticate on `production` env _(only define if different from default)_ | `$GCP_KEY_FILE` |
| `GCP_PROD_APP_NAME` | Application name for `production` env | `$GCP_BASE_APP_NAME` |
| `GCP_PROD_ENVIRONMENT_URL`| The production environment url _(only define for static environment URLs declaration and if different from default)_ | `$GCP_ENVIRONMENT_URL` |
| `GCP_PROD_DEPLOY_STRATEGY`| Defines the deployment to production strategy. One of `manual` (i.e. _one-click_) or `auto`. | `manual` |
| `GCP_PROD_OIDC_PROVIDER` | Workload Identity Provider associated with GitLab to [authenticate with OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/) on `production ` environment | none|
| `GCP_PROD_OIDC_ACCOUNT` | Service Account to which impersonate with OpenID Connect authentication on `production ` environment | none |
## Examples
### Google AppEngine application
#### Context
Let's imagine a backend service:
* named **coockedoodledoo**,
* developped in whichever language,
* part of project named **farmvoices**
* hosted on Google AppEngine with project ID `farmvoices-12345`
* with review, staging and production environments enabled.
#### `.gitlab-ci.yml`
```yaml
include:
# Include Google Cloud template
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
file: '/templates/gitlab-ci-gcloud.yml'
...
# Global variables
variables:
...
# Google Cloud
# GCP_KEY_FILE defined as secret CI/CD variable
GCP_REVIEW_PROJECT: "farm-12345" # enable review env
GCP_STAGING_PROJECT: "farm-12345" # enable staging env
GCP_PROD_PROJECT: "farm-12345" # enable production env
GCP_STAGING_ENVIRONMENT_URL: "https://staging-dot-coockedoodledoo-dot-farmvoices-12345.ew.r.appspot.com"
GCP_PROD_ENVIRONMENT_URL: "https://coockedoodledoo-dot-farmvoices-12345.ew.r.appspot.com"
# Postman
REVIEW_ENABLED: "true"
# Pipeline steps
stages:
- build
- test
- deploy
- acceptance
- publish
- production
# define review environment url (uses $CI_ENVIRONMENT_SLUG as app version)
gcp-review:
environment:
url: "https://$CI_ENVIRONMENT_SLUG-dot-coockedoodledoo-dot-farmvoices-12345.ew.r.appspot.com"
```
#### AppEngine manifest
```yaml
# Google AppEngine manifest
# see: https://cloud.google.com/appengine/docs/standard/java11/config/appref
runtime: TODO # depends on languages
instance_class: F2
service: coockedoodledoo
...
variables:
# this is an example of hardcoded (non-sensitive) configuration variable
SOME_CONFIG: "some-value"
# this is an example of variabilized (secret) configuration variable
# will be replaced programmatically during deployment
SOME_SECRET: "${SOME_SECRET}"
```
#### hook scripts
##### `gcp-deploy.sh`
This script is executed by the template to perform the application(s) deployment based on `gcloud` CLI.
```bash
#!/usr/bin/env bash
echo "[gcp-deploy] Deploy burger/$CI_ENVIRONMENT_SLUG..."
# prepare GAE deployment directory (copy build output)
mkdir -p gae
cp build/* gae
# copy manifest with variables substitution
awk '{while(match($0,"[$]{[^}]*}")) {var=substr($0,RSTART+2,RLENGTH-3);gsub("[$]{"var"}",ENVIRON[var])}}1' < src/app.yaml > gae/app.yaml
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
# gcloud deploy
# use $CI_ENVIRONMENT_SLUG as the version
cd gae
if [[ "$CI_ENVIRONMENT_SLUG" == "production" ]]
then
promote_opt="--promote"
else
promote_opt="--no-promote"
fi
gcloud --quiet app deploy --project=${gcp_project_id} --version=${CI_ENVIRONMENT_SLUG} $promote_opt
```
##### `gcp-cleanup.sh`
This script is executed by the template to perform the application(s) cleanup based on `gcloud` CLI (review env only).
```bash
#!/usr/bin/env bash
echo "[gcp-cleanup] Cleanup burger/$CI_ENVIRONMENT_SLUG..."
# use $CI_ENVIRONMENT_SLUG as the version
gcloud --quiet app versions delete --project=${gcp_project_id} --service=coockedoodledoo ${CI_ENVIRONMENT_SLUG}
```
## Variants
The Google Cloud template can be used in conjunction with template variants to cover specific cases.
### Vault variant
This variant allows delegating your secrets management to a [Vault](https://www.vaultproject.io/) server.
#### Configuration
In order to be able to communicate with the Vault server, the variant requires the additional configuration parameters:
| Name | description | default value |
| ----------------- | -------------------------------------- | ----------------- |
| `TBC_VAULT_IMAGE` | The [Vault Secrets Provider](https://gitlab.com/to-be-continuous/tools/vault-secrets-provider) image to use (can be overridden) | `$CI_REGISTRY/to-be-continuous/tools/vault-secrets-provider:master` |
| `VAULT_BASE_URL` | The Vault server base API url | _none_ |
| :lock: `VAULT_ROLE_ID` | The [AppRole](https://www.vaultproject.io/docs/auth/approle) RoleID | **must be defined** |
| :lock: `VAULT_SECRET_ID` | The [AppRole](https://www.vaultproject.io/docs/auth/approle) SecretID | **must be defined** |
#### Usage
Then you may retrieve any of your secret(s) from Vault using the following syntax:
```text
@url@http://vault-secrets-provider/api/secrets/{secret_path}?field={field}
```
With:
| Name | description |
| -------------------------------- | -------------------------------------- |
| `secret_path` (_path parameter_) | this is your secret location in the Vault server |
| `field` (_query parameter_) | parameter to access a single basic field from the secret JSON payload |
#### Example
```yaml
include:
# main template
- project: 'to-be-continuous/gcloud'
file: '/templates/gitlab-ci-gcloud.yml'
# Vault variant
- project: 'to-be-continuous/gcloud'
file: '/templates/gitlab-ci-gcloud-vault.yml'
variables:
# Secrets managed by Vault
SOME_SECRET_USED_IN_MY_APP: "@url@http://vault-secrets-provider/api/secrets/b7ecb6ebabc231/prod/gcloud/secret?field=my.app.secret"
VAULT_BASE_URL: "https://vault.acme.host/v1"
# $VAULT_ROLE_ID and $VAULT_SECRET_ID defined as a secret CI/CD variable
```