Skip to content
Snippets Groups Projects
Commit 0a249370 authored by František Řezníček's avatar František Řezníček
Browse files

refactor: added docs + CMG_OUT_METRIC_DIR -> CMG_OUT_METRICS_DIR

parent 2857ac04
No related branches found
No related tags found
No related merge requests found
Pipeline #93031 passed
...@@ -6,6 +6,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ...@@ -6,6 +6,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased] ## [Unreleased]
## [1.0.1] - 2021-06-30
### Changed
- change default output metric directory (singular->plural)
### Added
- Documentation added
## [1.0.0] - 2021-06-29 ## [1.0.0] - 2021-06-29
### Added ### Added
- Initial release - Initial release
...@@ -18,15 +18,29 @@ Although every custom metrics may have its dedicated prometheus exporter, [Prome ...@@ -18,15 +18,29 @@ Although every custom metrics may have its dedicated prometheus exporter, [Prome
### How to run custom-metrics-generator ### How to run custom-metrics-generator
```sh ```sh
docker run -it --rm --privileged \ docker run -it --privileged \
-v /var/run/docker.sock:/var/run/docker.sock -v /var/run/docker.sock:/var/run/docker.sock
-v /etc/node-exporter/node-role.prom:/etc/node-exporter/node-role.prom:ro -v /etc/node-exporter/node-role.prom:/etc/node-exporter/node-role.prom:ro
-v /opt/puppetlabs/puppet/cache/state/last_run_summary.yaml:/var/lib/puppet/state/last_run_summary.yaml:ro \ -v /opt/puppetlabs/puppet/cache/state/last_run_summary.yaml:/var/lib/puppet/state/last_run_summary.yaml:ro \
-v /var/run/node-exporter:/opt/custom-metrics-generator/out-metric-dir \ -v /var/run/node-exporter:/opt/custom-metrics-generator/out-metrics-dir \
-e 'CMG_EXEC_MODULES=smartmon dockermon puppetmon noderolemon' \ -e 'CMG_EXEC_MODULES=smartmon dockermon puppetmon noderolemon' \
registry.gitlab.ics.muni.cz:443/cloud/custom-metrics-generator:latest registry.gitlab.ics.muni.cz:443/cloud/custom-metrics-generator:latest
``` ```
### Debugging ### Debugging
TODO * list containers and exec to custom-metrics-generator
* `docker ps`
* get custom-metrics-generator logs
* `docker logs custom-metrics-generator`
* exec to custom-metrics-generator
* `docker exec -it custom-metrics-generator bash`
* list and inspect crontab rules
* `crontab -l`
* list and inspect exported metrics
* `ls -la /opt/custom-metrics-generator/out-metrics-dir`
* list runner execution env. dump (for instance puppetmon), seek env. variable [STAGE_NAME value to identify what happened](https://gitlab.ics.muni.cz/cloud/custom-metrics-generator/-/blob/e37df3ea3c1eed794cbfb3b129321ded26ba5d2f/src/metric-generator-exec.sh#L94)
* `more /tmp/cmg-puppetmon-trace.log`
* try to run the job manually in trace mode (for instance puppetmon)
* `bash -x /opt/custom-metrics-generator/src/metric-generator-exec.sh puppetmon /opt/custom-metrics-generator/out-metrics-dir/puppetmon.prom /opt/custom-metrics-generator/out-metrics-dir/custom-metrics-generator_puppetmon.prom`
```
...@@ -8,7 +8,7 @@ set -eo pipefail ...@@ -8,7 +8,7 @@ set -eo pipefail
# constants # constants
CMG_BASE_DIR=$(dirname $(readlink -f $0)) CMG_BASE_DIR=$(dirname $(readlink -f $0))
CMG_MAX_JITTER_DELAY="${CMG_MAX_JITTER_DELAY:-"3"}" CMG_MAX_JITTER_DELAY="${CMG_MAX_JITTER_DELAY:-"3"}"
CMG_OUT_METRIC_DIR="${CMG_OUT_METRIC_DIR:-"${CMG_BASE_DIR}/out-metric-dir"}" CMG_OUT_METRICS_DIR="${CMG_OUT_METRICS_DIR:-"${CMG_BASE_DIR}/out-metrics-dir"}"
CMG_STDOUT_LOG="${CMG_STDOUT_LOG:-"${CMG_BASE_DIR}/custom-metrics-generator.stdout.log"}" CMG_STDOUT_LOG="${CMG_STDOUT_LOG:-"${CMG_BASE_DIR}/custom-metrics-generator.stdout.log"}"
CMG_STDERR_LOG="${CMG_STDERR_LOG:-"${CMG_BASE_DIR}/custom-metrics-generator.stderr.log"}" CMG_STDERR_LOG="${CMG_STDERR_LOG:-"${CMG_BASE_DIR}/custom-metrics-generator.stderr.log"}"
CMG_EXEC_MODULES="${CMG_EXEC_MODULES:-"smartmon dockermon puppetmon noderolemon"}" CMG_EXEC_MODULES="${CMG_EXEC_MODULES:-"smartmon dockermon puppetmon noderolemon"}"
...@@ -23,15 +23,15 @@ rotate_file ${CMG_STDOUT_LOG} & ...@@ -23,15 +23,15 @@ rotate_file ${CMG_STDOUT_LOG} &
rotate_file ${CMG_STDERR_LOG} & rotate_file ${CMG_STDERR_LOG} &
# create out directory (if not present) # create out directory (if not present)
test -d "${CMG_OUT_METRIC_DIR}" || \ test -d "${CMG_OUT_METRICS_DIR}" || \
mkdir -p "${CMG_OUT_METRIC_DIR}" mkdir -p "${CMG_OUT_METRICS_DIR}"
i_fixed_sleep=0 i_fixed_sleep=0
for i_module in ${CMG_EXEC_MODULES}; do for i_module in ${CMG_EXEC_MODULES}; do
touch "${CMG_OUT_METRIC_DIR}/${i_module}.prom" \ touch "${CMG_OUT_METRICS_DIR}/${i_module}.prom" \
"${CMG_OUT_METRIC_DIR}/custom-metrics-generator_${i_module}.prom" > /dev/null "${CMG_OUT_METRICS_DIR}/custom-metrics-generator_${i_module}.prom" > /dev/null
echo "* * * * * ${CMG_BASE_DIR}/src/delay-jitter-exec.sh ${i_fixed_sleep} ${CMG_MAX_JITTER_DELAY} \ echo "* * * * * ${CMG_BASE_DIR}/src/delay-jitter-exec.sh ${i_fixed_sleep} ${CMG_MAX_JITTER_DELAY} \
${CMG_BASE_DIR}/src/metric-generator-exec.sh ${i_module} ${CMG_OUT_METRIC_DIR}/${i_module}.prom ${CMG_OUT_METRIC_DIR}/custom-metrics-generator_${i_module}.prom" ${CMG_BASE_DIR}/src/metric-generator-exec.sh ${i_module} ${CMG_OUT_METRICS_DIR}/${i_module}.prom ${CMG_OUT_METRICS_DIR}/custom-metrics-generator_${i_module}.prom"
i_fixed_sleep=$(( ${i_fixed_sleep} + 5 )) i_fixed_sleep=$(( ${i_fixed_sleep} + 5 ))
done > /var/spool/cron/root done > /var/spool/cron/root
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment