Tracking Probe Service
This project builds a Docker image able to send GitLab CI job information to any Elasticsearch server.
It is aimed at being used in GitLab CI as a service container in order to track to be continuous usage and compute statistics.
Which information is tracked ?
The Tracking Probe Service, whenever added as a service container of a GitLab CI job, will send the following JSON payload to one or several Elasticsearch servers:
{
"@type": "ci-job",
"@timestamp": "2020-10-19T07:34:08Z",
"template": {
"name": "maven",
"version": "1.0.0"
},
"ci": {
"user": {
"id": 6097,
"login": "pismy"
},
"project": {
"id": 23568,
"path": "https://gitlab.com/to-be-continuous/samples/maven-library"
},
"build": {
"ref": "master"
},
"job": {
"id": 8585338,
"name": "mvn-build",
"stage": "package-publish",
"url": "https://gitlab.com/to-be-continuous/samples/maven-library/-/jobs/8585338"
},
"pipeline": {
"id": 1835260,
"url": "https://gitlab.com/to-be-continuous/samples/maven-library/-/pipelines/1835260"
},
"runner": {
"id": 44949,
"description": "shared-runners-manager-4.gitlab.com",
"tags": "gce, east-c, shared, docker, linux, ruby, mysql, postgres, mongo, git-annex",
"version": "13.12.0-rc1"
}
}
}Each of those information are retrieved from GitLab CI predefined variables.
From this, you can build any valuable statistics, visualization or so.
How is it used in GitLab CI ?
The Docker image can be used as a service container in any GitLab CI file as follows:
.some-base-job:
services:
- name: "$CI_REGISTRY/to-be-continuous/tools/tracking:master"
command: ["--service", "some-template", "1.0.0"]
some-job-build:
stage: build
extends: .some-base-job
script:
- echo "build"
some-job-test:
stage: test
extends: .some-base-job
script:
- echo "test"The 2 arguments passed to the service container are the template name and version that will be sent with the JSON payload (the only 2 information that can't be retrieved from GitLab CI predefined variables).
💡 this is configured in every to be continuous template.
How to configure the Elasticsearch servers to send to ?
The configuration is defined in JSON and is built with the Docker image.
You shall define it as the TRACKING_CONFIGURATION CI/CD variable of this project as follows:
{
"clients": [
{
"url":"https://elasticsearch-host",
"authentication": {
"username":"tbc-tracking",
"password":"mYp@55w0rd"
},
"timeout":5,
"indexPrefix":"tbc-",
"esMajorVersion":7,
"skipSslVerification":true
}
]
}💡 Notice that you may configure more than one Elasticsearch server.
Then manually start a pipeline on the master branch: this will (re)generate a new Docker image with your
configuration that will now be used by every template job.