83bfe2cfda
Remove E2E_CLUSTER_NAME, E2E_NETWORK, and KUBE_GCE_INSTANCE_PREFIX from jobs that don't need them, because they are in their own project, and add defaults Fail on leaked resources by default Kill E2E_SET_CLUSTER_API_VERSION Rely on defaults for E2E_UP, E2E_TEST, E2E_DOWN Only PUBLISH_GREEN_VERSION for kubernetes-e2e-gce Directly export ZONE vars rather than chaining and setting locals Collapse E2E_CLUSTER_NAME, E2E_NETWORK, and KUBE_GCE_INSTANCE_PREFIX into E2E_NAME (always take the shortest of the three, to avoid name length problems Factor out soak vars Remove incorrect feature enablement ENV vars export GINKGO_TEST_ARGS directly, rather than looping through export at the bottom Directly export E2E Control Variables Export KUBEMARK ENV vars directly export KUBERNETES_PROVIDER directly Export JENKINS_PUBLISHED_VERSION directly, and don't use it where unnecessary; remove old CURRENT_RELEASE_PUBLISHED_VERSION export PROJECT and NETWORK_PROVIDER directly export node ENV vars directly export other shared cluster ENV vars directly Directly export CLOUDSDK ENV vars export GCE and AWS ENVs directly Various cleanup, including refactoring upgrades Fixup: no PERFORMANCE env for AWS Address comment Fixes for comments up to cebf501 Address comments through 81f39b8 Fix: default E2E_UP, E2E_TEST, E2E_DOWN Use JENKINS_FORCE_GET_TARS for upgrade jobs; fixes through 140ea2b Fix ubernetes-lite config Address rebased 2 comments Remove tab
Jenkins
Jenkins is a pluggable continuous integration system. The Google team is running a Jenkins server on a private GCE instance for the Kubernetes project in order to run longer integration tests, continuously, on different providers. Currently, we are running tests on GCE, GKE, and AWS.
General flow
The flow of the Google Jenkins server:
- Under the
kubernetes-buildjob: Every 2 minutes, Jenkins polls for a batch of new commits, after which it runs thebuild.shscript (in this directory) on the latest tip. This results in build assets getting pushed to GCS and thelatest.txtfile in thecibucket being updated. - On trigger, and every half hour (which effectively means all the time, unless
we're failing cluster creation), e2e variants run, on the latest build assets
in GCS:
kubernetes-e2e-gce: Standard GCE e2e.kubernetes-e2e-gke: GKE provider e2e, with head k8s client and GKE creating clusters at its default version.kubernetes-e2e-gke-ci: GKE provider e2e, with head k8s client and GKE creating clusters at the head k8s version.kubernetes-e2e-aws: AWS provider e2e.
- Each job will not run concurrently with itself, so, for instance,
Jenkins executor will only ever run one
kubernetes-buildjob. However, it may run the jobs in parallel, i.e.kubernetes-buildmay be run at the same time askubernetes-e2e-gce. For this reason, you may see your changes pushed to our GCS bucket rapidly, but they may take some time to fully work through Jenkins. Or you may get lucky and catch the train in 5 minutes. - There are many jobs not listed here, including upgrade tests, soak tests, and tests for previous releases.
Scripts
The scripts in this directory are directly used by Jenkins, either by curl from githubusercontent (if we don't have a git checkout handy) or by executing it from the git checkout. Since Jenkins is an entity outside this repository, it's tricky to keep documentation for it up to date quickly. However, the scripts themselves attempt to provide color for the configuration(s) that each script runs in.
Job Builder
New jobs should be specified as YAML files to be processed by Jenkins Job
Builder. The YAML files
live in job-configs and its subfolders. Jenkins runs Jenkins Job Builder
in a Docker container defined in job-builder-image, and triggers it using
update-jobs.sh. Jenkins Job Builder uses a config file called
jenkins_jobs.ini
which contains the location and credentials of the Jenkins server.