3.5 KiB
LinuxKit Testing and CI
Testing
The test suite uses rtf
To install this you should use make bin/rtf && make install.
Running the tests
To run the test suite:
cd test
rtf -v run -x
This will run the tests and put the results in a the _results directory!
Run control is handled using labels and with pattern matching. To run add a label you may use:
rtf -v -l slow run -x
You can list the tests which will be run using:
rtf list
Some tests may be marked as SKIP and LABELS column will typically provide and indication as to why a test may be skipped.
To run tests that match the pattern linuxkit.build you would use the following command:
rtf -v run -x linuxkit.build
Writing new tests
To add a new test, you should first decide which group it should be added to.
Check the test/cases folder for a list of existing groups.
Groups can be nested and there examples of this in the existing set of test cases.
If you feel that a new group is warranted you can create one by mkdir 000_name where
000 is the desired order and name is the name you would like to use
You must copy an existing group.sh in to this folder and adjust as required or you may use the
example
To write your test, create a folder within the group using the 000_name format as described above.
You should then copy an existing test.sh in to this directory and amend it,
or start from an example
If your test can only be run when certain conditions are met, you should consider adding a label to
avoid it being run by default and document the use of the label in tests/README.md
Continuous Integration
Note: This will hopefully change significantly soon
The LinuxKit CI system uses DatakitCI The configuration can be found here
The test logs are available via the Details link but also via this website. The raw test logs are also stored on GitHub where each branch contains the results for a particular run.
PR Testing
Each PR is tested on disposable VM's spawned in Google Cloud Platform This machine has no privileges or credentials to talk to GCP or other cloud platforms.
TODO: Add instructions on how to build a base image for LinuxKit CI in GCP.
LinuxKit CI runs make ci-pr in the VM.
This target runs the tests using rtf and the results directory is scp'ed back to the controller.
The test results will be stored in DataKit for additional access
Additionally, the ./artifacts folder is scp'ed back to the controller.
If the tests passed the next step is to check if the kernel config test image runs on GCP.
The ./artifacts/test.img.tar.gz file is used to create a GCP image by the Python scripts
that are part of LiunxKit CI. It runs this image and greps for the success message in the logs.
Branch and Tag testing
Branches and Tags are tested on a dedicated machine that runs in GCP.
LinuxKit CI runs make ci or make ci-tag in the VM
This target runs the tests using rtf and the results directory is SCP'd back to the controller.
The test results will be stored in DataKit.
If the tests pass, the GCP test is run in the same manner as described for PR tests.
Finally, CI will run make test-ltp which will run the Linux Testing Project tests.