From 05180b61a088b3865ef64cb36cabe1a0becca525 Mon Sep 17 00:00:00 2001 From: Gabriela Cervantes Date: Wed, 30 Aug 2023 16:31:49 +0000 Subject: [PATCH] metrics: Add README for kata metrics report This PR adds the README for kata metrics report. Fixes #7802 Signed-off-by: Gabriela Cervantes (cherry picked from commit c0ed5ea0ad71066b259cddccab378ff262324b91) --- tests/metrics/report/README.md | 64 ++++++++++++++++++++++++++++++++++ 1 file changed, 64 insertions(+) create mode 100644 tests/metrics/report/README.md diff --git a/tests/metrics/report/README.md b/tests/metrics/report/README.md new file mode 100644 index 0000000000..da586953d1 --- /dev/null +++ b/tests/metrics/report/README.md @@ -0,0 +1,64 @@ +# Kata Containers metrics report generator + +The files within this directory can be used to generate a "metrics report" for Kata Containers. The +primary workflow consists of two stages: +1) Run the provided report metrics data gathering scripts on the system(s) you wish to analyze. +2) Run the provided report generation script to analyze the data and generate a report file. + +## Data gathering + +Data gathering is provided by the `grabdata.sh` script. When run, this script executes a set of +tests from the `tests/metrics` directory. The JSON results files will be placed into the +`tests/metrics/results` directory. Once the results are generated, create a suitably named +subdirectory of `tests/metrics/results`, and move the JSON files into it. Repeat this process if +you want to compare multiple sets of results. Note, the report generation scripts process all +subdirectories of `tests/metrics/results` when generating the report. + +> **Note:** By default, the `grabdata.sh` script tries to launch some moderately large containers +> (i.e. 8Gbyte RAM) and may fail to produce some results on a memory constrained system. + +You can restrict the subset of tests run by `grabdata.sh` via its commandline parameters: + +| Option | Description | +| ------ | ------------------------| +| -a | Run all tests (default) | +| -d | Run the density tests | +| -h | Print this help | +| -s | Run the storage tests | +| -t | Run the time tests | + +## Report generation + +Report generation is provided by the `makereport.sh` script. By default this script processes all +subdirectories of the `tests/metrics/results` directory to generate the report. To run in the +default mode, execute the following: + +``` +$ ./makereport.sh +``` + +The report generation tool uses [`Rmarkdown`](https://github.com/rstudio/rmarkdown), [R](https://www.r-project.org/about.html) and +[Pandoc](https://pandoc.org/) to produce a PDF report. To avoid the need for all users to set up a +working environment with all the necessary tooling, the `makereport.sh` script utilises a +`Dockerfile` with the environment pre-defined in order to produce the report. Thus, you need to +have Docker installed on your system in order to run the report generation. The resulting +`metrics_report.pdf` is generated into the `output` subdirectory of the `report` directory. + +## Debugging and development + +To aid in script development and debugging, the `makereport.sh` script offers a debug facility via +the `-d` command line option. Using this option will place you into a `bash` shell within the +running `Dockerfile` image used to generate the report, whilst also mapping your host side `R` +scripts from the `report_dockerfile` subdirectory into the container, thus facilitating a "live" +edit/reload/run development cycle. From there you can examine the Docker image environment, and +execute the generation scripts. E.g., to test the `scaling.R` script, you can execute: + +``` +$ makereport.sh -d +# R +> source('/inputdir/Env.R') +> source('/scripts/lifecycle-time.R') +``` + +You can then edit the `report_dockerfile/lifecycle-time.R` file on the host, and re-run +the `source('/scripts/lifecycle-time.R')` command inside the still running `R` container.