# GitLab CI template for Go

This project implements a GitLab CI/CD template to build, test and analyse your [Go](https://golang.org/) projects.

## Usage

This template can be used both as a [CI/CD component](https://docs.gitlab.com/ee/ci/components/#use-a-component) 
or using the legacy [`include:project`](https://docs.gitlab.com/ee/ci/yaml/index.html#includeproject) syntax.

### Use as a CI/CD component

Add the following to your `.gitlab-ci.yml`:

```yaml
include:
  # 1: include the component
  - component: $CI_SERVER_FQDN/to-be-continuous/golang/gitlab-ci-golang@4.11.1
    # 2: set/override component inputs
    inputs:
      image: "registry.hub.docker.com/library/golang:buster" # ⚠ this is only an example
```

### Use as a CI/CD template (legacy)

Add the following to your `.gitlab-ci.yml`:

```yaml
include:
  # 1: include the template
  - project: 'to-be-continuous/golang'
    ref: '4.11.1'
    file: '/templates/gitlab-ci-golang.yml'

variables:
  # 2: set/override template variables
  GO_IMAGE: "registry.hub.docker.com/library/golang:buster" # ⚠ this is only an example
```

## Global configuration

The Go template uses some global configuration used throughout all jobs.

| Input / Variable | Description                                                                                                | Default value   |
|------------------|------------------------------------------------------------------------------------------------------------|-----------------|
| `image` / `GO_IMAGE` | The Docker image used to run Go for `go-build` <br/>:warning: **set the version required by your project** | `registry.hub.docker.com/library/golang:bookworm` |
| `test-image` / `GO_TEST_IMAGE` | The Docker image used to run Go for `go-test` <br/>:warning: **set the version required by your project**  | _none_          |
| `project-dir` / `GO_PROJECT_DIR` | Go project root directory                                                                                  | `.`             |
| `goproxy` / `GOPROXY` | URL of Go module proxy                                                                                     | _none_          |

## Jobs

### go generate job

The Go template supports code generation with [go generate](https://go.dev/blog/generate).
It is disable by default and can be enabled by setting the `GO_GENERATE_MODULES` variable.

| Input / Variable | Description                                                                                                | Default value   |
|------------------|------------------------------------------------------------------------------------------------------------|-----------------|
| `generate-modules` / `GO_GENERATE_MODULES` | Space separated list of Go code generator modules (ex: `stringer mockery`) | _none_ (disabled) |

#### Capture generated files as job artifacts

Using [go generate](https://go.dev/blog/generate) actually generates source files, that have to be captured and promoted all the way down to the build & test jobs.

In its default configuration, the template captures the following:

* any folder named `mock/` wherever in the file tree (entire content),
* any folder named `mocks/` wherever in the file tree (entire content),
* any file matching `*mock*.go` pattern wherever in the file tree.

If this default doesn't suit your needs, you'll have to override the artifact path patterns (YAML).
An example of this is given in the next chapter.

#### Example

```yaml
variables:
  # list all required generate modules (including mockery)
  GO_GENERATE_MODULES: >
    github.com/vektra/mockery/v2@v2.38.0 
    github.com/deepmap/oapi-codegen/v2/cmd/oapi-codegen@latest 
    mockery

# override the artifact path patterns
go-generate:
  artifacts:
    paths:
      # list all the matchers to capture generated code
      - "**/*mockery.go"
      - "myapi/client/"
```

### build & test jobs

You can specify if you want the template to build an `application` or `modules` with the `GO_BUILD_MODE` variable. It may have the following values: 

 * `application` will make the build output the binaries (use `-o` build option, won't work if there is no `main.go` file)
 * `modules` won't output the binaries (no use of the `-o` option)
 * `auto` the template will rely on the presence of a `main.go` file to detect if it should output the binaries.

The build target platform is the one defined by the docker image but it can be overriden using the `GO_TARGET_OS` and `GO_TARGET_ARCH` variables.

```yaml
variables:
  GO_TARGET_OS: "windows"
  GO_TARGET_ARCH: "amd64"
```

Build and tests can be done in separate jobs.
If `GO_TEST_IMAGE` is not set (default), the `go-build-test` job will run build and tests at once.
If `GO_TEST_IMAGE` is set, separate `go-build` and `go-test` jobs will be run in the `build` phase in parallel.

Separating `build` and `test` jobs can be useful to use different images (and hence different tools) or if you want to build muli-platform binaries.

Here is a `.gitlab-ci.yml` example that triggers a build on 3 target platforms using the [parallel matrix jobs](https://docs.gitlab.com/ee/ci/yaml/#parallel-matrix-jobs) pattern:

```yaml
variables:
  GO_IMAGE: "registry.hub.docker.com/library/golang:1.17-buster"
  GO_TEST_IMAGE: "registry.hub.docker.com/library/golang:1.17-buster"

go-build:
  parallel:
    matrix:
      - GO_TARGET_OS: "windows"
        GO_TARGET_ARCH: "amd64"
      - GO_TARGET_OS: "linux"
        GO_TARGET_ARCH: "amd64"
      - GO_TARGET_OS: "linux"
        GO_TARGET_ARCH: "arm"
```

These jobs use the following variable:

| Input / Variable | Description                                                                                                             | Default value                                 |
|-------------------------|-------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------|
| `build-mode` / `GO_BUILD_MODE` | The template build mode (accepted values are `application`, `modules` and `auto`)                                       | `auto`                                        |
| `build-flags` / `GO_BUILD_FLAGS` | Flags used by the [go build command](https://pkg.go.dev/cmd/go#hdr-Compile_packages_and_dependencies)                   | `-mod=readonly`                               |
| `build-linker-flags` / `GO_BUILD_LINKER_FLAGS` | Linker flags used by the [go build command](https://pkg.go.dev/cmd/go#hdr-Compile_packages_and_dependencies) `-ldflags` | `-s -w`                                       |
| `build-packages` / `GO_BUILD_PACKAGES` | Packages to build with the [go build command](https://pkg.go.dev/cmd/go#hdr-Compile_packages_and_dependencies)          | `./...`                                       |
| `test-flags` / `GO_TEST_FLAGS` | Flags used by the [go test command](https://pkg.go.dev/cmd/go#hdr-Test_packages)                                        | `-mod=readonly -v -race`                      |
| `test-packages` / `GO_TEST_PACKAGES` | Packages to test with the [go test command](https://pkg.go.dev/cmd/go#hdr-Test_packages)                                | `./...`                                       |
| `list-args` / `GO_LIST_ARGS` | Arguments used by the list command                                                                                      | `list -u -m -mod=readonly -json all`          |
| `target-os` / `GO_TARGET_OS` | The `GOOS` target [see available values](https://gist.github.com/asukakenji/f15ba7e588ac42795f421b48b8aede63)           | _none_ (fallback to go docker image `GOOS`)   |
| `target-arch` / `GO_TARGET_ARCH` | The `GOARCH` target [see available values](https://gist.github.com/asukakenji/f15ba7e588ac42795f421b48b8aede63)         | _none_ (fallback to go docker image `GOARCH`) |
| `cobertura-flags` / `GO_COBERTURA_FLAGS` | The `GOFLAGS` to use with `gocover-cobertura` if needed                                                                 | _none_                                        |

In addition to a textual report in the console, the test jobs produce the following reports, kept for one day:

| Report                                              | Format                                                                             | Usage                                                                                                                 |
|-----------------------------------------------------|------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------|
| `$GO_PROJECT_DIR/reports/go-test.native.txt`        | native Go test report (text)                                                       | N/A                                                                                                                   |
| `$GO_PROJECT_DIR/reports/go-test.native.json`       | native Go test report (json)                                                       | [SonarQube integration](https://docs.sonarqube.org/latest/analysis/test-coverage/test-execution-parameters/#header-8) |
| `$GO_PROJECT_DIR/reports/go-test.xunit.xml`         | [xUnit](https://en.wikipedia.org/wiki/XUnit) test report(s)                        | [GitLab integration](https://docs.gitlab.com/ee/ci/yaml/artifacts_reports.html#artifactsreportsjunit)                 |
| `$GO_PROJECT_DIR/reports/go-coverage.native.out`    | native Go coverage                                                                 | N/A                                                                                                                   |
| `$GO_PROJECT_DIR/reports/go-coverage.cobertura.xml` | [Cobertura XML](https://gcovr.com/en/stable/output/cobertura.html) coverage report | [GitLab integration](https://docs.gitlab.com/ee/ci/yaml/artifacts_reports.html#artifactsreportscoverage_report)       |

### `go-ci-lint` job

This job enables a manual [GolangCI-Lint](https://github.com/golangci/golangci-lint) analysis.

It is bound to the `build` stage, and uses the following variables:

| Input / Variable | Description                                                                                              | Default value                          |
|-----------------------|----------------------------------------------------------------------------------------------------------|----------------------------------------|
| `ci-lint-image` / `GO_CI_LINT_IMAGE` | The Docker image used to run `golangci-lint`                                                             | `registry.hub.docker.com/golangci/golangci-lint:latest-alpine` |
| `ci-lint-args` / `GO_CI_LINT_ARGS` | `golangci-lint` [command line arguments](https://github.com/golangci/golangci-lint#command-line-options) | `-E gosec,goimports ./...`             |
| `ci-lint-disabled` / `GO_CI_LINT_DISABLED` | Set to `true` to disable this job                                                                        | _none_ (enabled)                        |

In addition to a textual report in the console, this job produces the following reports, kept for one day:

| Report                                                | Format                                                   | Usage                                                                                                       |
|-------------------------------------------------------|----------------------------------------------------------|-------------------------------------------------------------------------------------------------------------|
| `$GO_PROJECT_DIR/reports/go-ci-lint.codeclimate.json` | [Code Climate](https://docs.codeclimate.com/docs/pylint) | [GitLab integration](https://docs.gitlab.com/ee/ci/yaml/artifacts_reports.html#artifactsreportscodequality) |
| `$GO_PROJECT_DIR/reports/go-ci-lint.checkstyle.xml`   | Checkstyle                                               | [SonarQube integration](https://docs.sonarsource.com/sonarqube-server/latest/analyzing-source-code/importing-external-issues/external-analyzer-reports/)                        |

### `go-semgrep` job

This job performs a [Semgrep](https://semgrep.dev/docs/) analysis.

It is bound to the `test` stage, and uses the following variables:

| Input / Variable | Description | Default Value |
| ---------------- | ----------- | ------------- |
| `semgrep-disabled` / `GO_SEMGREP_DISABLED` | Set to `true` to disable this job | _none_ |
| `semgrep-image` / `GO_SEMGREP_IMAGE` | The Docker image used to run [Semgrep](https://semgrep.dev/docs/) | `registry.hub.docker.com/semgrep/semgrep:latest` |
| `semgrep-args` / `GO_SEMGREP_ARGS` | Semgrep [scan options](https://semgrep.dev/docs/cli-reference#semgrep-scan-command-options) | `--metrics off --disable-version-check` |
| `semgrep-rules` / `GO_SEMGREP_RULES` | Space-separated list of [Semgrep rules](https://semgrep.dev/docs/running-rules).<br/>Can be both local YAML files or remote rules from the [Segmrep Registry](https://semgrep.dev/explore) (denoted by the `p/` prefix). | `p/golang p/gosec` |
| `semgrep-download-rules-enabled` / `GO_SEMGREP_DOWNLOAD_RULES_ENABLED` | Download Semgrep remote rules | `true` |

> :information_source: Semgrep may [collect some metrics](https://semgrep.dev/docs/metrics), especially when using rules from the Semgrep Registry.
> To protect your privacy and let you run Semgrep in air-gap environments, this template disables all Semgrep metrics by default:
>
> * rules from the Semgrep registry are pre-downloaded and passed to Semgrep as local rule files (can be disabled by setting `semgrep-download-rules-enabled` / `GO_SEMGREP_DOWNLOAD_RULES_ENABLED` to `false`),
> * the `--metrics` option is set to `off`,
> * the `--disable-version-check` option is set.

In addition to a textual report in the console, this job produces the following reports, kept for one week:

| Report | Format | Usage |
| ------ | ------ | ----- |
| `$GO_PROJECT_DIR/reports/golang-semgrep.gitlab.json` | [GitLab's SAST format](https://semgrep.dev/docs/cli-reference#semgrep-scan-command-options) | [GitLab integration](https://docs.gitlab.com/ee/ci/yaml/artifacts_reports.html#artifactsreportssast) |
| `$GO_PROJECT_DIR/reports/golang-semgrep.native.json` | [Semgrep's JSON format](https://semgrep.dev/docs/cli-reference#semgrep-scan-command-options) | [DefectDojo integration](https://docs.defectdojo.com/en/connecting_your_tools/parsers/file/semgrep/)<br/>_This report is generated only if DefectDojo template is detected_ |

### `go-mod-outdated` job

This job enables a manual [Go-mod-outdated](https://github.com/psampaz/go-mod-outdated) analysis.

It is bound to the `test` stage, and uses the following variables:

| Input / Variable | Description                                                                                   | Default value     |
|------------------------|-----------------------------------------------------------------------------------------------|-------------------|
| `mod-outdated-args` / `GO_MOD_OUTDATED_ARGS` | `god-mod-outdated` [command line arguments](https://github.com/psampaz/go-mod-outdated#usage) | `-update -direct` |

Checking outdated modules can be a long operation and therefore the job is configured to be ran **manually** by default (overridable).

## SonarQube analysis

If you're using the SonarQube template to analyse your Go code, here is a sample `sonar-project.properties` file:

```properties
# see: https://docs.sonarqube.org/latest/analyzing-source-code/test-coverage/test-execution-parameters/#go
# set your source directory(ies) here (relative to the sonar-project.properties file)
sonar.sources=.
# exclude unwanted directories and files from being analysed
sonar.exclusions=bin/**,**/*_test.go,**/vendor/**

# set your tests directory(ies) here (relative to the sonar-project.properties file)
sonar.tests=.
sonar.test.inclusions=**/*_test.go
sonar.test.exclusions=**/vendor/**

# tests report: JSON native format
sonar.go.tests.reportPaths=reports/go-test.native.json
# coverage report: native format
sonar.go.coverage.reportPaths=reports/go-coverage.native.out
# golanci-lint: checkstyle report (if enabled)
sonar.go.golangci-lint.reportPaths=reports/go-ci-lint.checkstyle.xml
```

More info:

* [Go language support](https://docs.sonarqube.org/latest/analyzing-source-code/test-coverage/test-execution-parameters/#go)
* [test coverage & execution parameters](https://docs.sonarqube.org/latest/analysis/coverage/)
* [third-party issues](https://docs.sonarsource.com/sonarqube-server/latest/analyzing-source-code/importing-external-issues/external-analyzer-reports/)

:warning: an [unsolved issue](https://jira.sonarsource.com/browse/SONARSLANG-450) may prevent SonarQube Go plugin from
importing your test reports.

### `go-sbom` job

This job generates a [SBOM](https://cyclonedx.org/) file listing installed packages using [@cyclonedx/cyclonedx-gomod](https://github.com/CycloneDX/cyclonedx-gomod).

It is bound to the `test` stage, and uses the following variables:

| Input / Variable | Description                            | Default value     |
| --------------------- | -------------------------------------- | ----------------- |
| `sbom-disabled` / `GO_SBOM_DISABLED` | Set to `true` to disable this job | _none_ |
| `sbom-image` / `GO_SBOM_IMAGE` | Image of cyclonedx-gomod used for SBOM analysis | `registry.hub.docker.com/cyclonedx/cyclonedx-gomod:latest` |
| `sbom-opts` / `GO_SBOM_OPTS` | [@cyclonedx/cyclonedx-gomod options](https://github.com/CycloneDX/cyclonedx-gomod#usage) used for SBOM analysis | `-main .` |

:warning: if you don't have your main class located at the root of your `GO_PROJECT_DIR`, then you will need to override the `-main` option in `GO_SBOM_OPTS` and define your real main class location.

Example:

```yaml
variables:
  GO_SBOM_OPTS: "-main cmd/my_app"
```

### `go-govulncheck` job

This job enables Vulnerability Management with [Govulncheck](https://go.dev/blog/vuln).

It is bound to the `test` stage, and uses the following variables:

| Input / Variable | Description                            | Default value     |
| --------------------- | -------------------------------------- | ----------------- |
| `vulncheck-disabled` / `GO_VULNCHECK_DISABLED` | Set to `true` to disable this job | _none_ 
| `vulncheck-args` / `GO_VULNCHECK_ARGS` | `govulncheck` [command line arguments](https://pkg.go.dev/golang.org/x/vuln/cmd/govulncheck#hdr-Flags) | `./...` |