Spark job on Mesos not working when submitting job via mesos dispatcher -
i have had spark running on standalone cluster manager, , decided try out mesos.
when using client mode, works working perfectly. if want run in cluster mode, realized had start "mesos cluster dispatcher". tried starting regular script:
./start-mesos-dispatcher mesos://ip:5050
as using directions following link
http://www.simosh.com/article/dbcjefbf-start-spark-via-mesos.html
but cluster mode did not work in both cases. in both of them, saw in mesos ui driver submitted, after several seconds gets killed.
after tried submit job mesos in client mode, using dispatcher again, , saw submitting process stucks in following step:
i0509 11:09:44.550375 3339 sched.cpp:326] new master detected @ master@127.0.1.1:7078 i0509 11:09:44.555305 3339 sched.cpp:336] no credentials provided. attempting register without authentication
in spark logs, saw dispatcher correctly started, , see framework registered in mesos ui.
are there settings missing?
Comments
Post a Comment