diff --git a/hack/jenkins/README.md b/hack/jenkins/README.md index 92b1a14d632..39e01cbeccd 100644 --- a/hack/jenkins/README.md +++ b/hack/jenkins/README.md @@ -4,16 +4,23 @@ integration system. The Google team is running a Jenkins server on a private GCE instance for the Kubernetes project in order to run longer integration tests, continuously, on different providers. Currently, we -(Google) are only running Jenkins on our own providers (GCE and GKE) -in different flavors. +are running tests on GCE, GKE, and AWS. ## General flow The flow of the Google Jenkins server: -* Under the `kubernetes-build` job: Every 5 minutes, Jenkins polls for a batch of new commits, after which it runs the `build.sh` script (in this directory) on the latest tip. This results in build assets getting pushed to GCS and the `latest.txt` file in the `ci` bucket being updated. That job then triggers `kubernetes-e2e-*`. -* On trigger, and every half hour (which effectively means all the time, unless we're failing cluster creation), e2e variants run, on the latest build assets in GCS: - * `kubernetes-e2e-gce`: Standard GCE e2e - * `kubernetes-e2e-gke`: GKE provider e2e, with head k8s client and GKE creating clusters at its default version - * `kubernetes-e2e-gke-ci`: GKE provider e2e, with head k8s client and GKE creating clusters at the head k8s version +* Under the `kubernetes-build` job: Every 2 minutes, Jenkins polls for a batch + of new commits, after which it runs the `build.sh` script (in this directory) + on the latest tip. This results in build assets getting pushed to GCS and the + `latest.txt` file in the `ci` bucket being updated. +* On trigger, and every half hour (which effectively means all the time, unless + we're failing cluster creation), e2e variants run, on the latest build assets + in GCS: + * `kubernetes-e2e-gce`: Standard GCE e2e. + * `kubernetes-e2e-gke`: GKE provider e2e, with head k8s client and GKE + creating clusters at its default version. + * `kubernetes-e2e-gke-ci`: GKE provider e2e, with head k8s client and GKE + creating clusters at the head k8s version. + * `kubernetes-e2e-aws`: AWS provider e2e. * Each job will not run concurrently with itself, so, for instance, Jenkins executor will only ever run one `kubernetes-build` job. However, it may run the jobs in parallel, @@ -22,6 +29,8 @@ The flow of the Google Jenkins server: pushed to our GCS bucket rapidly, but they may take some time to fully work through Jenkins. Or you may get lucky and catch the train in 5 minutes. +* There are many jobs not listed here, including upgrade tests, soak tests, and + tests for previous releases. ## Scripts @@ -32,5 +41,14 @@ outside this repository, it's tricky to keep documentation for it up to date quickly. However, the scripts themselves attempt to provide color for the configuration(s) that each script runs in. +## Job Builder + +New jobs should be specified as YAML files to be processed by [Jenkins Job +Builder](http://docs.openstack.org/infra/jenkins-job-builder/). The YAML files +live in `job-configs` and its subfolders. Jenkins runs Jenkins Job Builder +in a Docker container defined in `job-builder-image`, and triggers it using +`update-jobs.sh`. Jenkins Job Builder uses a config file called +[jenkins_jobs.ini](http://docs.openstack.org/infra/jenkins-job-builder/execution.html) +which contains the location and credentials of the Jenkins server. [![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/hack/jenkins/README.md?pixel)]() diff --git a/hack/jenkins/job-builder-image/Dockerfile b/hack/jenkins/job-builder-image/Dockerfile new file mode 100644 index 00000000000..3e116dfe7e2 --- /dev/null +++ b/hack/jenkins/job-builder-image/Dockerfile @@ -0,0 +1,33 @@ +# This docker image runs Jenkins Job Builder (JJB) for automatic job reconciliation. + +FROM ubuntu:14.04 +MAINTAINER Joe Finney + +RUN mkdir /build +WORKDIR /build + +# Dependencies for JJB +RUN apt-get update && apt-get install -y \ + wget \ + git \ + python-dev \ + python-pip \ + libyaml-dev \ + python-yaml +RUN pip install PyYAML python-jenkins +# Required since JJB supports python 2.6, which doesn't have ordereddict built-in. We have 2.7. +RUN wget https://pypi.python.org/packages/source/o/ordereddict/ordereddict-1.1.tar.gz \ + && tar -xvf ordereddict-1.1.tar.gz \ + && cd ordereddict-1.1 \ + && python setup.py install + +RUN git clone https://git.openstack.org/openstack-infra/jenkins-job-builder \ + && cd jenkins-job-builder \ + && python setup.py install + +# JJB configuration lives in /etc/jenkins_jobs/jenkins_jobs.ini +RUN mkdir -p /etc/jenkins_jobs + +WORKDIR / +RUN git clone https://github.com/kubernetes/kubernetes.git +WORKDIR kubernetes diff --git a/hack/jenkins/job-builder-image/Makefile b/hack/jenkins/job-builder-image/Makefile new file mode 100644 index 00000000000..d0c0ba1c736 --- /dev/null +++ b/hack/jenkins/job-builder-image/Makefile @@ -0,0 +1,5 @@ +TAG = 1 + +all: + docker build -t gcr.io/google_containers/kubekins-job-builder:$(TAG) . + gcloud docker push gcr.io/google_containers/kubekins-job-builder:$(TAG) diff --git a/hack/jenkins/job-configs/kubernetes-update-jenkins-jobs.yaml b/hack/jenkins/job-configs/kubernetes-update-jenkins-jobs.yaml new file mode 100644 index 00000000000..86bac456a33 --- /dev/null +++ b/hack/jenkins/job-configs/kubernetes-update-jenkins-jobs.yaml @@ -0,0 +1,9 @@ +- job: + name: kubernetes-update-jenkins-jobs + description: "Update Jenkins jobs" + + triggers: + - timed: "H/15 * * * *" + + builders: + - shell: "curl -fsS https://raw.githubusercontent.com/kubernetes/kubernetes/master/hack/jenkins/update-jobs.sh | /bin/bash -" diff --git a/hack/jenkins/update-jobs.sh b/hack/jenkins/update-jobs.sh new file mode 100755 index 00000000000..a14a41ede90 --- /dev/null +++ b/hack/jenkins/update-jobs.sh @@ -0,0 +1,45 @@ +#!/bin/bash + +# Copyright 2015 The Kubernetes Authors All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# Update all Jenkins jobs in a folder. If no folder is provided in $1, +# defaults to hack/jenkins/job-configs. + +if [[ $# -eq 1 ]]; then + config_dir=$1 +else + config_dir="hack/jenkins/job-configs" +fi + +# Run the container if it isn't present. +if ! docker inspect job-builder &> /dev/null; then + # jenkins_jobs.ini contains administrative credentials for Jenkins. + # Store it in the workspace of the Jenkins job that calls this script. + if [[ -e jenkins_jobs.ini ]]; then + docker run -idt \ + --net host \ + --name job-builder \ + --restart always \ + gcr.io/google_containers/kubekins-job-builder:1 + docker cp jenkins_jobs.ini job-builder:/etc/jenkins_jobs + else + echo "jenkins_jobs.ini not found in workspace" >&2 + exit 1 + fi +fi + +docker exec job-builder git checkout master +docker exec job-builder git pull +docker exec job-builder jenkins-jobs test ${config_dir}