feat: add helm publish to a repository

65c79247 · Gaël Bréard · Pierre Smeyers · e85c52a8 · 65c79247 · 65c79247
Commit 65c79247 authored 3 years ago by Gaël Bréard Committed by Pierre Smeyers 3 years ago
--- a/README.md
+++ b/README.md
@@ -65,6 +65,25 @@ The Helm template uses some global configuration used throughout all jobs.
 | `HELM_ENV_VALUE_NAME`      | The environment type variable set to helm | `env` |
 | `HELM_HOSTNAME_VALUE_NAME`      | The hostname variable set to helm | `hostname` |

+### Charts publishing
+
+The template builds a chart package that may be pushed as two distinct packages, depending on a certain _workflow_:
+
+1. **snapshot**: the chart is first packaged and then pushed to some registry as
+  the **snapshot** image. It can be seen as the raw result of the build, but still **untested and unreliable**.
+2. **release**: once the snapshot chart has been thoroughly tested (both by `package-test` stage jobs and/or `acceptance`
+  stage jobs after being deployed to some server), then the chart is pushed one more time as the **release** chart.
+  This second push can be seen as the **promotion** of the snapshot chart being now **tested and reliable**.
+
+Common variables for `helm-package` and `helm-pusblish`:
+
+| Name                              | description                                  | default value           |
+| --------------------------------- | -------------------------------------------- | ----------------------- |
+| `HELM_REPO_PUBLISH_METHOD`        | HTTP method to use to push the package       | `POST`                  |
+| :lock: `HELM_REPO_USER`           | Helm registry username                       | `$CI_REGISTRY_USER`     |
+| :lock: `HELM_REPO_PASSWORD`       | Helm registry password                       | `$CI_REGISTRY_PASSWORD` |
+
+

 ### Secrets management

@@ -250,13 +269,16 @@ This job runs [Kube-Score](https://kube-score.com/) on the resources to be creat

 ### `helm-package` job

-This job [packages your chart into an archive](https://helm.sh/docs/helm/helm_package/) and uses the following variables:
+This job [packages your chart into an archive](https://helm.sh/docs/helm/helm_package/), optionaly push it to a snapshot repository and uses the following variables:

-| Name                  | description                              | default value     |
-| --------------------- | ---------------------------------------- | ----------------- |
-| `HELM_PACKAGE_ARGS`   | The Helm [command with options](https://helm.sh/docs/helm/helm_package/) to perform the packaging (_without dynamic arguments such as the chart path_)   | `package --dependency-update` |
-
-| `HELM_SEMREL_RELEASE_DISABLED`   | Set to `true` to disable usage of semrel release info for helm package  | _none_ (enabled) |
+| Name                                  | description                              | default value     |
+| ------------------------------------- | ---------------------------------------- | ----------------- |
+| `HELM_PACKAGE_ARGS`                   | The Helm [command with options](https://helm.sh/docs/helm/helm_package/) to perform the packaging (_without dynamic arguments such as the chart path_)   | `package --dependency-update` |
+| `HELM_SEMREL_RELEASE_DISABLED`        | Set to `true` to disable usage of semrel release info for helm package  | _none_ (enabled) |
+| `HELM_PUBLISH_SNAPSHOT_URL`           | The URL of the Helm repository to publish your Helm package as a snapshot | _gitlab repo on snapshot channel_ `${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/helm/api/snapshot/charts` |
+| `HELM_REPO_SNAPSHOT_PUBLISH_METHOD`   | HTTP method to use to push the package        | `$HELM_REPO_PUBLISH_METHOD`       |
+| `HELM_REPO_SNAPSHOT_USER`             | Snapshot repo username | `$HELM_REPO_USER` |
+| :lock: `HELM_REPO_SNAPSHOT_PASSWORD`  | Snapshot repo password | `$HELM_REPO_PASSWORD` |

 #### `semantic-release` integration

@@ -268,13 +290,14 @@ Note: You can disable the `semantic-release` integration (as it's the `helm-pack

 ### `helm-publish` job

-This job uses the following variables:
+This job push helm package to a release repository and uses the following variables:

-| Name                  | description                              | default value     |
-| --------------------- | ---------------------------------------- | ----------------- |
-| `HELM_PUBLISH_ARGS`   | Arguments used by the Helm publish job   | _none_ |
-| `HELM_PUBLISH_DIR`    | The folder in which the job will publish the chart  | `.` |
-| `HELM_PUBLISH_URL`    | The URL of the the chart to publish      | _none_ |
+| Name                                | description                                   | default value                     |
+| ----------------------------------- | --------------------------------------------- | --------------------------------- |
+| `HELM_PUBLISH_URL`                  | The URL of the Helm repository to publish your Helm package | _gitlab repo on release channel_ `${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/helm/api/release/charts`  |
+| `HELM_REPO_RELEASE_PUBLISH_METHOD`  | HTTP method to use to push the package        | `$HELM_REPO_PUBLISH_METHOD`       |
+| `HELM_REPO_RELEASE_USER`            | Release repo username (for PUT request auth)  | `$HELM_REPO_USER`                 |
+| :lock: `HELM_REPO_RELEASE_PASSWORD` | Release repo password (for PUT request auth)  | `$HELM_REPO_PASSWORD`             |

 ### `helm-test` job


--- a/kicker.json
+++ b/kicker.json
@@ -68,6 +68,24 @@
      "description": "The hostname variable set to helm",
      "default": "hostname",
      "advanced": true
+    },
+    {
+      "name": "HELM_REPO_USER",
+      "description": "Helm repo username",
+      "default": "$CI_REGISTRY_USER",
+      "advanced": true
+    },
+    {
+      "name": "HELM_REPO_PASSWORD",
+      "description": "hELM repo password",
+      "default": "$CI_REGISTRY_PASSWORD",
+      "advanced": true
+    },
+    {
+      "name": "HELM_REPO_PUBLISH_METHOD",
+      "description": "HTTP method to use to push the package",
+      "default": "POST",
+      "advanced": true
    }
  ],
  "features": [
@@ -156,8 +174,38 @@
        {
          "name": "HELM_SEMREL_RELEASE_DISABLED",
          "description": "Disable semantic-release integration",
+          "type": "boolean",
+          "advanced": true
+        },
+        {
+          "name": "HELM_PUBLISH_SNAPSHOT_URL",
+          "description": The URL of the Helm repository to publish your Helm package as a snapshot,
+          "default": "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/helm/api/snapshot/charts",
+          "advanced": true
+        },
+        {
+          "name": "HELM_REPO_SNAPSHOT_USER",
+          "description": "Snapshot repo username",
+          "default": "$HELM_REPO_USER",
+          "secret": true,
+          "advanced": true
+        },
+        {
+          "name": "HELM_REPO_SNAPSHOT_PASSWORD",
+          "description": "Snapshot repo password",
+          "default": "$HELM_REPO_PASSWORD",
+          "secret": true,
+          "advanced": true
+        },
+        {
+          "name": "HELM_REPO_SNAPSHOT_PUBLISH_METHOD",
+          "description": "HTTP method to use to push the snapshot package",
+          "default": "$HELM_REPO_PUBLISH_METHOD",
          "advanced": true
        }
+        
+        
+        
      ]
    },
    {
@@ -166,20 +214,30 @@
      "description": "Publish your Helm chart",
      "variables": [
        {
-          "name": "HELM_PUBLISH_ARGS",
-          "description": "Arguments used by the Helm publish job",
+          "name": "HELM_PUBLISH_URL",
+          "description": "The URL of the Helm repository to publish your Helm package",
+          "default": "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/helm/api/release/charts",
          "advanced": true
        },
        {
-          "name": "HELM_PUBLISH_DIR",
-          "description": "The folder in which the job will publish the chart",
-          "default": ".",
+          "name": "HELM_REPO_RELEASE_USER",
+          "description": "Release repo username",
+          "default": "$HELM_REPO_USER",
+          "secret": true,
          "advanced": true
        },
        {
-          "name": "HELM_PUBLISH_URL",
-          "description": "The URL of the the chart to publish",
-          "mandatory": true
+          "name": "HELM_REPO_RELEASE_PASSWORD",
+          "description": "Release repo password",
+          "default": "$HELM_REPO_PASSWORD",
+          "secret": true,
+          "advanced": true
+        },
+        {
+          "name": "HELM_REPO_RELEASE_PUBLISH_METHOD",
+          "description": "HTTP method to use to push the release package",
+          "default": "$HELM_REPO_PUBLISH_METHOD",
+          "advanced": true
        }
      ]
    },

--- a/templates/gitlab-ci-helm.yml
+++ b/templates/gitlab-ci-helm.yml
@@ -27,9 +27,9 @@ variables:

  HELM_CHART_DIR: "."
  HELM_PACKAGE_ARGS: "package --dependency-update"
-#  HELM_PUBLISH_ARGS: ""
-  HELM_PUBLISH_DIR: "."
-#  HELM_PUBLISH_URL: ""
+  HELM_PUBLISH_URL: "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/helm/api/release/charts"
+  HELM_PUBLISH_SNAPSHOT_URL: "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/helm/api/snapshot/charts"
+  HELM_REPO_PUBLISH_METHOD: "POST"

  HELM_REPOS: "stable@https://charts.helm.sh/stable bitnami@https://charts.bitnami.com/bitnami"

@@ -436,6 +436,15 @@ stages:
    # shellcheck disable=SC2086
    helm ${TRACE+--debug} $helm_opts $helm_namespace_opt $HELM_TEST_ARGS $appname
  }
+  
+  function maybe_install_curl() {
+    if ! command -v curl > /dev/null
+    then
+      log_info "--- installing curl (required to publish Helm charts)..."
+      apk add --no-cache curl
+    fi
+  }
+


  function get_latest_template_version() {
@@ -654,6 +663,23 @@ helm-package:
        helm_version_opts="--app-version ${SEMREL_INFO_NEXT_VERSION} --version ${SEMREL_INFO_NEXT_VERSION}"
      fi
    - helm $HELM_PACKAGE_ARGS ${TRACE+--debug} $helm_version_opts $HELM_CHART_DIR --destination helm_packages
+    - |    
+      package=$(ls -1 ./helm_packages/*.tgz 2>/dev/null || echo "")
+      if [ -n "$HELM_PUBLISH_SNAPSHOT_URL" ] && [ -n "${package}" ]
+      then
+        package_file=$(basename ${package})
+        log_info "publishing helm chart ${package_file} to snapshot url: ${HELM_PUBLISH_SNAPSHOT_URL}"
+        username="${HELM_REPO_SNAPSHOT_USER:-${HELM_REPO_USER:-$CI_REGISTRY_USER}}"
+        password="${HELM_REPO_SNAPSHOT_PASSWORD:-${HELM_REPO_PASSWORD:-$CI_REGISTRY_PASSWORD}}"
+        method="${HELM_REPO_SNAPSHOT_PUBLISH_METHOD:-$HELM_REPO_PUBLISH_METHOD}"
+        if [[ "$method" == "POST" ]]
+        then
+          maybe_install_curl
+          curl --fail --request POST --form "chart=@${package}" --user "$username:$password" $HELM_PUBLISH_SNAPSHOT_URL
+        else
+          wget -v --method=PUT --user="$username" --password="$password" --body-file="${package}" "$HELM_PUBLISH_SNAPSHOT_URL/${package_file}" -O -
+        fi
+      fi
  rules:
    - if: $CI_MERGE_REQUEST_ID
      when: never
@@ -671,14 +697,32 @@ helm-publish:
  extends: .helm-base
  stage: publish
  script:
-    - helm ${TRACE+--debug} $(get_helm_config_opt) repo index $HELM_PUBLISH_DIR --url $HELM_PUBLISH_URL $HELM_PUBLISH_ARGS
-  artifacts:
-    expire_in: 1 week
-    paths:
-      - $HELM_PUBLISH_DIR
+    - |    
+      package=$(ls -1 ./helm_packages/*.tgz 2>/dev/null || echo "")
+      if [ -n "$HELM_PUBLISH_URL" ] && [ -n "${package}" ]
+      then
+        package_file=$(basename ${package})
+        log_info "publishing helm chart ${package_file} to release url: ${HELM_PUBLISH_URL}"
+        username="${HELM_REPO_RELEASE_USER:-${HELM_REPO_USER:-$CI_REGISTRY_USER}}"
+        password="${HELM_REPO_RELEASE_PASSWORD:-${HELM_REPO_PASSWORD:-$CI_REGISTRY_PASSWORD}}"
+        method="${HELM_REPO_RELEASE_PUBLISH_METHOD:-$HELM_REPO_PUBLISH_METHOD}"
+        if [[ "$method" == "POST" ]]
+        then
+          maybe_install_curl
+          curl --fail --request POST --form "chart=@${package}" --user "$username:$password" $HELM_PUBLISH_URL
+        else
+          wget -v --method=PUT --user="$username" --password="$password" --body-file="${package}" "$HELM_PUBLISH_URL/${package_file}" -O -
+        fi
+      else
+        log_error "No Chart to deploy! url is: $HELM_PUBLISH_URL, and package found is: ${package}"
+      fi
  rules:
    - if: $CI_MERGE_REQUEST_ID || $HELM_PUBLISH_URL == null || $CI_COMMIT_REF_NAME !~ $PROD_REF
      when: never
+    - if: $AUTODEPLOY_TO_PROD
+    # else: manual + blocking
+    - if: $CI_COMMIT_REF_NAME # this 'if' is useless but only prevents GitLab warning :(
+      when: manual
    # /!\ variables can't be used in rules:exists
    - exists:
        - "**/Chart.yaml"