diff --git a/README.md b/README.md index acb50c0311e6e54786170a9e04aaf3f3b35c7f1a..ebe7b79e2b8e9c01849d141fd2fcf97719f7e3ed 100644 --- a/README.md +++ b/README.md @@ -78,6 +78,62 @@ Here are some advices about your **secrets** (variables marked with a :lock:): it will then be possible to mask it and the template will automatically decode it prior to using it. 3. Don't forget to escape special characters (ex: `$` -> `$$`). +### Deployment and cleanup jobs + +The GitLab CI template for Google Cloud requires you to provide a shell script that fully implements your application +deployment and cleanup using the [`gcloud` CLI](https://cloud.google.com/sdk/gcloud). + +#### Lookup policy + +The deployment script is searched as follows: + +1. look for a specific `gcp-deploy-$env.sh` in the `$GCP_SCRIPTS_DIR` directory in your project (e.g. `gcp-deploy-staging.sh` for staging environment), +2. if not found: look for a default `gcp-deploy.sh` in the `$GCP_SCRIPTS_DIR` directory in your project, +3. if not found: the deployment job will fail. + +The cleanup script is searched as follows: + +1. look for a specific `gcp-cleanup-$env.sh` in the `$GCP_SCRIPTS_DIR` directory in your project (e.g. `gcp-cleanup-staging.sh` for staging environment), +2. if not found: look for a default `gcp-cleanup.sh` in the `$GCP_SCRIPTS_DIR` directory in your project, +3. if not found: the cleanup job will fail. + +Your script(s) shall use available [dynamic variables](#dynamic-variables). + +#### Dynamic Variables + +You have to be aware that your deployment (and cleanup) scripts have to be able to cope with various environments +(`review`, `integration`, `staging` and `production`), each with different application names, exposed routes, settings, ... + +Part of this complexity can be handled by the lookup policies described above (ex: one resource per env). + +In order to be able to implement some **genericity** in your scripts and templates, you should use available environment variables: + +1. any [GitLab CI variable](https://docs.gitlab.com/ee/ci/variables/#predefined-environment-variables) + (ex: `${CI_ENVIRONMENT_URL}` to retrieve the actual environment exposed route) +2. any [custom variable](https://docs.gitlab.com/ee/ci/variables/#custom-environment-variables) + (ex: `${SECRET_TOKEN}` that you have set in your project CI/CD variables) +3. **dynamic variables** set by the template: + * `${appname}`: the application target name to use in this environment (ex: `myproject-review-fix-bug-12` or `myproject-staging`) + * `${env}`: the environment type (`review`, `integration`, `staging` or `production`) + * `${hostname}`: the environment hostname, extracted from `${CI_ENVIRONMENT_URL}` (has to be explicitly declared as [`environment:url`](https://docs.gitlab.com/ee/ci/yaml/#environmenturl) in your `.gitlab-ci.yml` file) + * `${gcp_project_id}`: the current Google Cloud project ID associated to your environment + +#### Static vs. Dynamic environment URLs + +The Google Cloud template supports two ways of defining your environments url: + +* a **static way**: when you know your environments url in advance, probably because you're exposing your routes through a DNS you manage, +* a [**dynamic way**](https://docs.gitlab.com/ee/ci/environments/#set-dynamic-environment-urls-after-a-job-finishes): when the url cannot be known before the + deployment job is executed. + +The static way can be implemented simply by setting the appropriate configuration variables depending on the environments (see environments configuration chapters below): + +* `$GCP_REVIEW_ENVIRONMENT_SCHEME` and`$GCP_REVIEW_ENVIRONMENT_DOMAIN` for the review environments, +* `$GCP_INTEG_ENVIRONMENT_URL`, `$GCP_STAGING_ENVIRONMENT_URL` and `$GCP_PROD_ENVIRONMENT_URL` for others. + +To implement the dynamic way, your deployment script shall simply generate a `environment_url.txt` file, containing only +the dynamically generated url. + ### Environments configuration As seen above, the Google Cloud template may support up to 4 environments (`review`, `integration`, `staging` and `production`). @@ -158,51 +214,6 @@ Here are variables supported to configure the production environment: | `GCP_PROD_ENVIRONMENT_URL`|Â The production environment url **including scheme** (ex: `https://my-application.public.domain.com`) Do not use variable inside variable definition as it will result in a two level cascade variable and gitlab does not allow that. | _none_ | | `AUTODEPLOY_TO_PROD` | Set this variable to auto-deploy to production. If not set deployment to production will be `manual` (default behaviour). | _none_ (disabled) | -### Deployment jobs - -The GitLab CI template for Google Cloud requires you to provide a shell script that fully implements your application -deployment using the [`gcloud` CLI](https://cloud.google.com/sdk/gcloud). - -The deployment script is searched as follows: - -1. look for a specific `gcp-deploy-$env.sh` in the `$GCP_SCRIPTS_DIR` directory in your project (e.g. `gcp-deploy-staging.sh` for staging environment), -2. if not found: look for a default `gcp-deploy.sh` in the `$GCP_SCRIPTS_DIR` directory in your project, -3. if not found: the deployment job will fail. - -Your script(s) shall use available [dynamic variables](#dynamic-variables). - -### Cleanup jobs - -The GitLab CI template for Google Cloud requires you to provide a shell script that fully implements your application -cleanup using the [`gcloud` CLI](https://cloud.google.com/sdk/gcloud). - -The cleanup script is searched as follows: - -1. look for a specific `gcp-cleanup-$env.sh` in the `$GCP_SCRIPTS_DIR` directory in your project (e.g. `gcp-cleanup-staging.sh` for staging environment), -2. if not found: look for a default `gcp-cleanup.sh` in the `$GCP_SCRIPTS_DIR` directory in your project, -3. if not found: the cleanup job will fail. - -Your script(s) shall use available [dynamic variables](#dynamic-variables). - -### Dynamic Variables - -You have to be aware that your deployment (and cleanup) scripts have to be able to cope with various environments -(`review`, `integration`, `staging` and `production`), each with different application names, exposed routes, settings, ... - -Part of this complexity can be handled by the lookup policies described above (ex: one resource per env). - -In order to be able to implement some **genericity** in your scripts and templates, you should use available environment variables: - -1. any [GitLab CI variable](https://docs.gitlab.com/ee/ci/variables/#predefined-environment-variables) - (ex: `${CI_ENVIRONMENT_URL}` to retrieve the actual environment exposed route) -2. any [custom variable](https://docs.gitlab.com/ee/ci/variables/#custom-environment-variables) - (ex: `${SECRET_TOKEN}` that you have set in your project CI/CD variables) -3. **dynamic variables** set by the template: - * `${appname}`: the application target name to use in this environment (ex: `myproject-review-fix-bug-12` or `myproject-staging`) - * `${env}`: the environment type (`review`, `integration`, `staging` or `production`) - * `${hostname}`: the environment hostname, extracted from `${CI_ENVIRONMENT_URL}` (has to be explicitly declared as [`environment:url`](https://docs.gitlab.com/ee/ci/yaml/#environmenturl) in your `.gitlab-ci.yml` file) - * `${gcp_project_id}`: the current Google Cloud project ID associated to your environment - ## Examples ### Google AppEngine application diff --git a/templates/gitlab-ci-gcloud.yml b/templates/gitlab-ci-gcloud.yml index 2c830d77bf46dd44391b2b5444681ac186aa7042..3315cdb2230c9a7719548f76effc086f826e0382 100644 --- a/templates/gitlab-ci-gcloud.yml +++ b/templates/gitlab-ci-gcloud.yml @@ -241,8 +241,9 @@ stages: export env=$1 export appname=$2 export gcp_project_id=$3 - # extract hostname from $CI_ENVIRONMENT_URL - hostname=$(echo "$CI_ENVIRONMENT_URL" | awk -F[/:] '{print $4}') + export environment_url=$4 + # extract hostname from $environment_url + hostname=$(echo "$environment_url" | awk -F[/:] '{print $4}') export hostname log_info "--- \\e[32mdeploy\\e[0m (env: \\e[33;1m${env}\\e[0m)" @@ -251,6 +252,12 @@ stages: log_info "--- \$hostname: \\e[33;1m${hostname}\\e[0m" log_info "--- \$gcp_project_id: \\e[33;1m${gcp_project_id}\\e[0m" + # unset any upstream deployment env & artifacts + unset environment_name + unset environment_type + rm -f gcloud.env + rm -f environment_url.txt + deployscript=$(ls -1 "$GCP_SCRIPTS_DIR/gcp-deploy-${env}.sh" 2>/dev/null || ls -1 "$GCP_SCRIPTS_DIR/gcp-deploy.sh" 2>/dev/null || echo "") if [[ -f "$deployscript" ]] then @@ -262,8 +269,15 @@ stages: fi # finally persist environment url - echo "$CI_ENVIRONMENT_URL" > environment_url.txt - echo -e "environment_type=$env\\nenvironment_name=$appname\\nenvironment_url=$CI_ENVIRONMENT_URL" > gcloud.env + if [[ -f environment_url.txt ]] + then + environment_url=$(cat environment_url.txt) + export environment_url + log_info "--- dynamic environment url found: (\\e[33;1m$environment_url\\e[0m)" + else + echo "$environment_url" > environment_url.txt + fi + echo -e "environment_type=$env\\nenvironment_name=$appname\\nenvironment_url=$environment_url" > gcloud.env } # environment cleanup function @@ -335,6 +349,7 @@ stages: # @arg ENV_APP_SUFFIX: env-specific application suffix # @arg ENV_PROJECT : env-specific GCP Project ID # @arg ENV_KEY_FILE : env-specific GCP API key file (JSON) +# @arg ENV_URL : env-specific application url .gcp-deploy: extends: .gcp-base stage: deploy @@ -346,13 +361,15 @@ stages: - assert_defined "${ENV_KEY_FILE:-$GCP_KEY_FILE}" 'Missing required GCP key file (JSON)' - gcloud auth activate-service-account --key-file ${ENV_KEY_FILE:-$GCP_KEY_FILE} script: - - deploy "$ENV_TYPE" "${ENV_APP_NAME:-${GCP_BASE_APP_NAME}${ENV_APP_SUFFIX}}" "$ENV_PROJECT" + - deploy "$ENV_TYPE" "${ENV_APP_NAME:-${GCP_BASE_APP_NAME}${ENV_APP_SUFFIX}}" "$ENV_PROJECT" "$ENV_URL" artifacts: name: "$ENV_TYPE env url for $CI_PROJECT_NAME on $CI_COMMIT_REF_SLUG" paths: - environment_url.txt reports: dotenv: gcloud.env + environment: + url: "$environment_url" # can be either static or dynamic # Cleanup job prototype # Can be extended for each deletable environment @@ -389,9 +406,9 @@ gcp-review: ENV_APP_NAME: "$GCP_REVIEW_APP_NAME" ENV_PROJECT: "$GCP_REVIEW_PROJECT" ENV_KEY_FILE: "$GCP_REVIEW_KEY_FILE" + ENV_URL: "${GCP_REVIEW_ENVIRONMENT_SCHEME}://${CI_PROJECT_NAME}-${CI_ENVIRONMENT_SLUG}.${GCP_REVIEW_ENVIRONMENT_DOMAIN}" environment: name: review/$CI_COMMIT_REF_NAME - url: "${GCP_REVIEW_ENVIRONMENT_SCHEME}://${CI_PROJECT_NAME}-${CI_ENVIRONMENT_SLUG}.${GCP_REVIEW_ENVIRONMENT_DOMAIN}" on_stop: gcp-cleanup-review resource_group: review/$CI_COMMIT_REF_NAME rules: @@ -439,9 +456,9 @@ gcp-integration: ENV_APP_NAME: "$GCP_INTEG_APP_NAME" ENV_PROJECT: "$GCP_INTEG_PROJECT" ENV_KEY_FILE: "$GCP_INTEG_KEY_FILE" + ENV_URL: "${GCP_INTEG_ENVIRONMENT_URL}" environment: name: integration - url: "${GCP_INTEG_ENVIRONMENT_URL}" resource_group: integration rules: # exclude merge requests @@ -458,9 +475,9 @@ gcp-staging: ENV_APP_NAME: "$GCP_STAGING_APP_NAME" ENV_PROJECT: "$GCP_STAGING_PROJECT" ENV_KEY_FILE: "$GCP_STAGING_KEY_FILE" + ENV_URL: "${GCP_STAGING_ENVIRONMENT_URL}" environment: name: staging - url: "${GCP_STAGING_ENVIRONMENT_URL}" resource_group: staging rules: # exclude merge requests @@ -479,9 +496,9 @@ gcp-production: ENV_APP_NAME: "$GCP_PROD_APP_NAME" ENV_PROJECT: "$GCP_PROD_PROJECT" ENV_KEY_FILE: "$GCP_PROD_KEY_FILE" + ENV_URL: "${GCP_PROD_ENVIRONMENT_URL}" environment: name: production - url: "${GCP_PROD_ENVIRONMENT_URL}" resource_group: production rules: # exclude merge requests