Spark: Update to current example standards, add GCS connector

* Pod -> ReplicationController, which also forced me to hack around
hostname issue on the master. (Spark master sees the incoming slave
request to spark-master and assumes it's not meant for it, since it's
name is spark-master-controller-abcdef.)

* Remove service env dependencies (depend on DNS instead).

* JSON -> YAML.

* Add GCS connector.

* Make example do something actually useful: A familiar example to
anyone at Google, implement wordcount of all of Shakespeare's works.

* Fix a minor service connection issue in the gluster example.
This commit is contained in:
Zach Loafman
2015-10-26 13:42:02 -07:00
parent c0fba82da4
commit 777d3a6ec0
25 changed files with 325 additions and 231 deletions

View File

@@ -3,7 +3,7 @@ apiVersion: v1
metadata:
name: spark-master-controller
labels:
component: spark-master-controller
component: spark-master
spec:
replicas: 1
selector:
@@ -22,7 +22,7 @@ spec:
- mountPath: /mnt/glusterfs
name: glusterfsvol
resources:
limits:
requests:
cpu: 100m
volumes:
- name: glusterfsvol

View File

@@ -9,4 +9,4 @@ spec:
- port: 7077
targetPort: 7077
selector:
component: spark-master-controller
component: spark-master

View File

@@ -12,7 +12,7 @@ spec:
metadata:
labels:
component: spark-worker
uses: spark-master-controller
uses: spark-master
spec:
containers:
- name: spark-worker
@@ -23,7 +23,7 @@ spec:
- mountPath: /mnt/glusterfs
name: glusterfsvol
resources:
limits:
requests:
cpu: 100m
volumes:
- name: glusterfsvol