Newer
Older
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
# Tracking Probe Service
This project builds a Docker image able to send GitLab CI job information to any Elasticsearch server.
It is aimed at being used in GitLab CI as a [service container](https://docs.gitlab.com/ee/ci/services/)
in order to track _to be continuous_ usage and compute statistics.
## Which information is tracked ?
The _Tracking Probe Service_, whenever added as a [service container](https://docs.gitlab.com/ee/ci/services/)
of a GitLab CI job, will send the following JSON payload to one or several Elasticsearch servers:
```json
{
"@type": "ci-job",
"@timestamp": "2020-10-19T07:34:08Z",
"template": {
"name": "maven",
"version": "1.0.0"
},
"ci": {
"user": {
"id": 6097,
"login": "pismy"
},
"project": {
"id": 23568,
"path": "https://gitlab.com/to-be-continuous/samples/maven-library"
},
"build": {
"ref": "master"
},
"job": {
"id": 8585338,
"name": "mvn-build",
"stage": "package-publish",
"url": "https://gitlab.com/to-be-continuous/samples/maven-library/-/jobs/8585338"
},
"pipeline": {
"id": 1835260,
"url": "https://gitlab.com/to-be-continuous/samples/maven-library/-/pipelines/1835260"
},
"runner": {
"id": 44949,
"description": "shared-runners-manager-4.gitlab.com",
"tags": "gce, east-c, shared, docker, linux, ruby, mysql, postgres, mongo, git-annex",
"version": "13.12.0-rc1"
}
}
}
```
Each of those information are retrieved from [GitLab CI predefined variables](https://docs.gitlab.com/ee/ci/variables/predefined_variables.html).
From this, you can build any valuable statistics, visualization or so.
## How is it used in GitLab CI ?
The Docker image can be used as a [service container](https://docs.gitlab.com/ee/ci/services/)
in any GitLab CI file as follows:
```yaml
.some-base-job:
services:
- name: "$CI_REGISTRY/to-be-continuous/tools/tracking:master"
command: ["--service", "some-template", "1.0.0"]
some-job-build:
stage: build
extends: .some-base-job
script:
- echo "build"
some-job-test:
stage: test
extends: .some-base-job
script:
- echo "test"
```
The 2 arguments passed to the service container are the template **name** and **version** that will be sent with
the JSON payload (the only 2 information that can't be retrieved from [GitLab CI predefined variables](https://docs.gitlab.com/ee/ci/variables/predefined_variables.html)).
:bulb: this is configured in every _to be continuous_ template.
## How to configure the Elasticsearch servers to send to ?
The configuration is defined in JSON and is **built with the Docker image**.
You shall define it as the `TRACKING_CONFIGURATION` CI/CD variable of this project as follows:
```JSON
{
"clients": [
{
"url":"https://elasticsearch-host",
"authentication": {
"username":"tbc-tracking",
"password":"mYp@55w0rd"
},
"timeout":5,
"indexPrefix":"tbc-",
"esMajorVersion":7,
"skipSslVerification":true
}
]
}
```
:bulb: Notice that you may configure **more than one** Elasticsearch server.
Then manually start a pipeline on the `master` branch: this will (re)generate a new Docker image with your
configuration that will now be used by every template job.