storm源码工作流程（三）Supervisor监听zookeeper 启动worker

clipboard27.png

由上诉代码可以知道无论任务是否存在都会有一个Assignment 对象

storm源码工作流程（三）Supervisor监听zookeeper 启动worker_第2张图片

clipboard29.png

对 Assignment 对象进行一些设置框出代码是对集群信息进行更改将任务信息更新到zookeep上

clipboard30.png

继续点击

storm源码工作流程（三）Supervisor监听zookeeper 启动worker_第3张图片

clipboard31.png

path 是zookeeper的路径然后根据对象类型更新zookeep信息

接下里通过 watch 机制会回调 supervisor 中的main方法

storm源码工作流程（三）Supervisor监听zookeeper 启动worker_第4张图片

clipboard32.png

main方法在调用run方法

storm源码工作流程（三）Supervisor监听zookeeper 启动worker_第5张图片

clipboard33.png

通过 mkSupervisor 获得一个 SuperviosrManger对象点击进入mkSupervisor 方法

clipboard34.png

点击进入类 SyncProcessEvent

public void run(Map localAssignments, Set downloadFailedTopologyIds) {
    LOG.debug("Syncing processes, interval seconds:" + TimeUtils.time_delta(lastTime));
    lastTime = TimeUtils.current_time_secs();
    try { /**
         * Step 1: get assigned tasks from localstat Map
         */
        //从本地获取任务信息，请问本地的任务信息从哪里来的，由SyncSupervisorEvent类获取zookeeper的信息写入到本地。 if (localAssignments == null) {
            localAssignments = new HashMap<>();
        }
        LOG.debug("Assigned tasks: " + localAssignments); /**
         * Step 2: get local WorkerStats from local_dir/worker/ids/heartbeat Map
         * 获取当前机器上所属的worker的状态信息
         */
        Map localWorkerStats;
        try {
            localWorkerStats = getLocalWorkerStats(conf, localState, localAssignments);
        } catch (Exception e) {
            LOG.error("Failed to get Local worker stats");
            throw e;
        }
        LOG.debug("Allocated: " + localWorkerStats); /**
         * Step 3: kill Invalid Workers and remove killed worker from localWorkerStats
         * 杀死一些无效的worker并移除本地状态
         */
        Map taskCleaupTimeoutMap;
        Set keepPorts = null;
        try {
            taskCleaupTimeoutMap = (Map) localState.get(Common.LS_TASK_CLEANUP_TIMEOUT);
            keepPorts = killUselessWorkers(localWorkerStats, localAssignments, taskCleaupTimeoutMap);
            localState.put(Common.LS_TASK_CLEANUP_TIMEOUT, taskCleaupTimeoutMap);
        } catch (IOException e) {
            LOG.error("Failed to kill workers", e);
        } // check new workers
        checkNewWorkers(conf); // check which topology need update
        checkNeedUpdateTopologys(localWorkerStats, localAssignments); // start new workers
        //启动新的任务
        startNewWorkers(keepPorts, localAssignments, downloadFailedTopologyIds); } catch (Exception e) {
        LOG.error("Failed Sync Process", e);
        // throw e
    }}

其中startNewWorkers 需要会创建一个新的worker 点击进入