mirror of
https://github.com/k3s-io/kubernetes.git
synced 2026-01-13 11:25:19 +00:00
51817850ba
This adds a very basic Zeppelin image that works with the existing Spark example. As can be seen from the documentation, it has a couple of warts: * It requires kubectl port-forward (which is unstable across long periods of time, at least for me, on this app, bug incoming). See * I needed to roll my own container (none of the existing containers exactly matched needs, or even built anymore against modern Zeppelin master, and the rest of the example is Spark 1.5). The image itself is *huge*. One of the further refinements we need to look at is how to possibly strip the Maven build for this container down to just the interpreters we care about, because the deps here are frankly ridiculous. This might be a case where, if possible, we might want to open an upstream request to build things dynamically, then use something like probably the cut the image down considerably. (This might already be possible, need to poke at whether you can late-bind interpreters later.)
46 lines
2.0 KiB
Makefile
46 lines
2.0 KiB
Makefile
all: push
|
|
push: push-spark push-zeppelin
|
|
.PHONY: push push-spark push-zeppelin spark zeppelin
|
|
|
|
# To bump the Spark version, bump the version in base/Dockerfile, bump
|
|
# the version in zeppelin/Dockerfile, bump this tag and reset to
|
|
# v1. You should also double check the native Hadoop libs at that
|
|
# point (we grab the 2.6.1 libs, which are appropriate for
|
|
# 1.5.1-with-2.6). Note that you'll need to re-test Zeppelin (and it
|
|
# may not have caught up to newest Spark).
|
|
TAG = 1.5.1_v2
|
|
|
|
# To bump the Zeppelin version, bump the version in
|
|
# zeppelin/Dockerfile and bump this tag and reset to v1.
|
|
ZEPPELIN_TAG = v0.5.5_v1
|
|
|
|
spark:
|
|
docker build -t gcr.io/google_containers/spark-base base
|
|
docker tag gcr.io/google_containers/spark-base gcr.io/google_containers/spark-base:$(TAG)
|
|
docker build -t gcr.io/google_containers/spark-worker worker
|
|
docker tag gcr.io/google_containers/spark-worker gcr.io/google_containers/spark-worker:$(TAG)
|
|
docker build -t gcr.io/google_containers/spark-master master
|
|
docker tag gcr.io/google_containers/spark-master gcr.io/google_containers/spark-master:$(TAG)
|
|
docker build -t gcr.io/google_containers/spark-driver driver
|
|
docker tag gcr.io/google_containers/spark-driver gcr.io/google_containers/spark-driver:$(TAG)
|
|
|
|
zeppelin:
|
|
docker build -t gcr.io/google_containers/zeppelin zeppelin
|
|
docker tag -f gcr.io/google_containers/zeppelin gcr.io/google_containers/zeppelin:$(ZEPPELIN_TAG)
|
|
|
|
push-spark: spark
|
|
gcloud docker push gcr.io/google_containers/spark-base
|
|
gcloud docker push gcr.io/google_containers/spark-base:$(TAG)
|
|
gcloud docker push gcr.io/google_containers/spark-worker
|
|
gcloud docker push gcr.io/google_containers/spark-worker:$(TAG)
|
|
gcloud docker push gcr.io/google_containers/spark-master
|
|
gcloud docker push gcr.io/google_containers/spark-master:$(TAG)
|
|
gcloud docker push gcr.io/google_containers/spark-driver
|
|
gcloud docker push gcr.io/google_containers/spark-driver:$(TAG)
|
|
|
|
push-zeppelin: zeppelin
|
|
gcloud docker push gcr.io/google_containers/zeppelin
|
|
gcloud docker push gcr.io/google_containers/zeppelin:$(ZEPPELIN_TAG)
|
|
|
|
clean:
|