Launch and manage schedulers on Mesos
Install go 1.4 (or higher) http://golang.org/doc/install
Install godep https://github.com/tools/godep
Clone and build the project
# git clone https://github.com/stealthly/triceratops.git
# cd triceratops
# godep restore ./...
# go build
Triceratops requires Mesos, Marathon and Cassandra to work. Cassandra though is not quite required during startup because Triceratops supports bootstrapping.
In case you have Cassandra running somewhere you may run the Triceratops server like this:
# ./triceratops server --marathon http://master:8080 --cassandra cassandra.service
In case you wish Triceratops to bootstrap you should configure a bootstrap stack which will be run before actually starting the server. A sample bootstrap stack is available in bootstrap.stack file. Please note that bootstrap stacks do not allow inheritance (should not have from field). To run Triceratops bootstrapping with some stack do this:
# ./triceratops server --marathon http://master:8080 --cassandra {dse-mesos.cassandraConnect} --bootstrap bootstrap.stack
Available flags:
--marathon - [http://127.0.0.1:8080] - Marathon address [ip]:[port].
--api - [0.0.0.0:4200] - Triceratops server bind address.
--bootstrap - Stack file to bootstrap with.
--cassandra - [127.0.0.1] - Cassandra cluster IP addresses, comma separated.
--keyspace - [triceratops] - Cassandra keyspace.
--connect.retries - [10] - Number of retries to connect either to Marathon or Cassandra.
--connect.backoff - [10s] - Backoff between connection attempts to either Marathon or Cassandra.
--debug - [false] - Flag for debug mode.
marathon.json file contains a Marathon template that is able to run Triceratops server and bootstrap it. You will probably want to change it a little bit before using (at least before we support inferring Mesos Master which should be now set explicitly).
Use the following command to put this json template to Marathon:
# curl -X PUT -d@marathon.json -H "Content-Type: application/json" http://master:8080/v2/apps/triceratops
Stack is a YAML file that contains information about Applications that should be deployed on Mesos. Every stack has a unique name that allows to distinguish stacks among each other. Stack can also extend other stack to reduce amount of configurations being set. Every stack also has one or more applications which can depend on other applications in this stack.
Stack fields description:
name - [string] - unique stack name. This will be used to run the stack or when extending it.
from - [string] - parent stack name to inherit configurations from. Not allowed in bootstrap stacks.
applications - [map[string]Application] - applications to be deployed on Mesos. Map keys are application names to allow overriding configurations for specific applications and values are the actual application configurations.
Application fields description:
type - [string] - application type which is used to determine a TaskRunner to run the actual application tasks.
id - [string] - unique application ID that will be used as in Marathon.
version - [string] - optional field to specify an application version.
cpu - [double] - amount of CPUs for application scheduler.
mem - [double] - amount of memory for application scheduler.
ports - [array[int]] - ports to accept for application scheduler. Can be left empty to accept any offered port.
user - [string] - Mesos user that will be used to run the application. Defaults to current system user if not set.
healthcheck - [string] - URL to perform healthchecks. Optional but highly recommended.
launch_command - [string] - launch command for application scheduler. The scheduler flags should not be set here.
artifact_urls - [array[string]] - artifacts to be downloaded before running the application scheduler.
additional_artifacts - [array[string]] - additional artifacts to be downloaded before running the application scheduler. This can be used to avoid overriding the artifact list in child stacks. All additional artifacts will be appended to artifact urls list.
scheduler - [map[string]string] - scheduler configurations. Everything specified in these configurations will be appended to launch_command in form --k v.
tasks - [array[map[string]string]] - array of task configurations. Array length defines the number of separate tasks launched for this application. It is up to TaskRunner to decide how to use information contained in each task configuration.
dependencies - [array[string]] - application dependencies. The application in stack won't be run until all its dependencies are satisfied. E.g. applications without dependencies will be launched first, then others with resolved dependencies.
Stacks support inheritance (the only exception are stacks used for bootstrapping as we cannot fetch parent stacks from Cassandra during bootstrapping). This way you can define a stack with some common applications/behavior (not necessarily complete) and extend it in other stacks.
Consider a following example:
name: default.kafka-mesos
applications:
kafka-mesos:
type: "kafka-mesos-0.9.x"
id: kafka-mesos
version: 0.9.2
cpu: 0.5
mem: 512
launch_command: "/usr/bin/java -jar kafka-mesos-0.9.2.0.jar scheduler"
artifact_urls:
- "https://github.com/mesos/kafka/releases/download/v0.9.2.0/kafka-mesos-0.9.2.0.jar"
- "http://apache.volia.net/kafka/0.8.2.2/kafka_2.10-0.8.2.2.tgz"
healthcheck: /health
scheduler:
api: http://$HOST:$PORT0
master: zk://zookeeper:2181/mesos
zk: zookeeper:2181
debug: true
name: dev.kafka-mesos
from: default.kafka-mesos
applications:
kafka-mesos:
cpu: 1
mem: 1024
tasks:
- id: 0..3
constraints: "hostname=unique"
The dev.kafka-mesos stack will inherit everything contained in default.kafka-mesos but will also override cpu and mem and add tasks. It is also possible for some other stack to extend dev.kafka-mesos - there are no limitations on how many inheritance layers you have.
All stacks should be kept in Triceratops before being run. To add a stack define a stack file and run the following command:
# ./triceratops add --file path/to/stackfile
You can also pass an --api flag to specify the address of Triceratops server. By default it assumes Triceratops is running on 127.0.0.1:4200.
Available flags:
--api - [http://127.0.0.1:4200] - Triceratops server address.
--file - [Stackfile] - Stackfile with application configurations.
--debug - [false] - Flag for debug mode.
To remove a stack run the following command:
# ./triceratops remove dev.kafka-mesos
If the given stack does not have any dependent stacks it will be removed. Otherwise you will get an error message with a list of dependent stacks that should be removed first. Alternatively you may pass a --force flag to remove a stack and all its children. Be careful though as it doesn't ask for confirmation.
You can also pass an --api flag to specify the address of Triceratops server. By default it assumes Triceratops is running on 127.0.0.1:4200.
Available flags:
--api - [http://127.0.0.1:4200] - Triceratops server address.
--force - [false] - Flag to force delete the stack with all children.
--debug - [false] - Flag for debug mode.
To list all available stacks kept in Triceratops run the following command:
# ./triceratops list
You can also pass an --api flag to specify the address of Triceratops server. By default it assumes Triceratops is running on 127.0.0.1:4200.
Available flags:
--api - [http://127.0.0.1:4200] - Triceratops server address.
--debug - [false] - Flag for debug mode.
To show stack contents run the following command:
# ./triceratops show dev.kafka-mesos
You can also pass an --api flag to specify the address of Triceratops server. By default it assumes Triceratops is running on 127.0.0.1:4200.
Available flags:
--api - [http://127.0.0.1:4200] - Triceratops server address.
--debug - [false] - Flag for debug mode.
To run a stack run the following command:
# ./triceratops run dev.kafka-mesos
This call will block until the stack successfully deploys or an error occurs.
You can also pass an --api flag to specify the address of Triceratops server. By default it assumes Triceratops is running on 127.0.0.1:4200.
Available flags:
--api - [http://127.0.0.1:4200] - Triceratops server address.
--debug - [false] - Flag for debug mode.
Minimal stack examples are located in stacks directory in this repository.