Twister编译及安装 [多节点方式]

注：该流程也适用单节点模式, 设置ip及工作节点的时候设置本节点即可.

1. Twister设置

1.1 设置Twister路径，编辑~/.bashrc [注: 最好是这个文件，本人也试过在.bash_profile, 但之后运行就出错了，不知道为什么！]

export TWISTER_HOME =/home/username/TWISTER

1.2 编辑Twster/bin/twister.properties

daemon_port=12500
pubsub_broker=ActiveMQ
data_dir=/home/lucktroy/twister-0.9/data
app_dir=/home/lucktroy/twister-0.9/apps
workers_per_daemon=8
nodes_file=/home/lucktroy/twister-0.9/bin/nodes
daemons_per_node=1

1.3 编辑Twsiter/bin/nodes, 添加工作节点

node02
node03

1.4 编辑Twister/bin/nb.properties [注：单节点模式]

#本人用的是node02做broker
broker_host = 10.10.10.2
broker_port = 3045
comm_type = niotcp

1.5 编辑Twister/bin/amq.properties [注：多节点模式]

uri=failover\:(tcp\://node03\:61616)

2. 下载ActiveMQ 5.4.2

把$ACTIVEMQ_HOME/activemq-all-X.X.X.jar 复制到 $TWISTER_HOME/lib/

3. 使用ant编译

[lucktroy@node01 twister-0.9]$ ant
Buildfile: build.xml

clean:
   [delete] Deleting directory /home/lucktroy/twister-0.9/build

init:
    [mkdir] Created dir: /home/lucktroy/twister-0.9/build

compile:
    [javac] Compiling 107 source files to /home/lucktroy/twister-0.9/build

dist:
    [mkdir] Created dir: /home/lucktroy/twister-0.9/dist/lib
      [jar] Building jar: /home/lucktroy/twister-0.9/dist/lib/Twister-0.9.jar
     [copy] Copying 1 file to /home/lucktroy/twister-0.9/lib

BUILD SUCCESSFUL
Total time: 4 seconds

4. 运行ActiveMQ, 这个时候终端一直会被该进程占用，因而下一步时需要再开启一个终端. [注: 高端用户可以用shell命令screen开启这个activemq,这样不必一直占用终端了！]

$ bin/activemq console
INFO: Using default configuration
(you can configure options in one of these file: /etc/default/activemq /home/jbz                                                                     hang/.activemqrc)

INFO: Invoke the following command to create a configuration file
bin/activemq setup [ /etc/default/activemq | /home/lucktroy/.activemqrc ]

INFO: Using java '/usr/bin/java'
INFO: Starting in foreground, this is just for debugging purposes (stop process                                                                      by pressing CTRL+C)
Java Runtime: Sun Microsystems Inc. 1.6.0_24 /usr/lib/jvm/java-1.6.0-openjdk-1.6                                                                     .0.0.x86_64/jre
  Heap sizes: current=251264k  free=248639k  max=251264k
    JVM args: -Xms256M -Xmx256M -Dorg.apache.activemq.UseDedicatedTaskRunner=tru                                                                     e -Djava.util.logging.config.file=logging.properties -Dcom.sun.management.jmxrem                                                                     ote -Dactivemq.classpath=/home/lucktroy/apache-activemq-5.4.2/conf; -Dactivemq.ho                                                                     me=/home/lucktroy/apache-activemq-5.4.2 -Dactivemq.base=/home/lucktroy/apache-acti                                                                     vemq-5.4.2
ACTIVEMQ_HOME: /home/lucktroy/apache-activemq-5.4.2
ACTIVEMQ_BASE: /home/lucktroy/apache-activemq-5.4.2
Loading message broker from: xbean:activemq.xml
 INFO | Refreshing org.apache.activemq.xbean.XBeanBrokerFactory$1@5d6d2633: star                                                                     tup date [Fri Apr 05 10:48:34 CDT 2013]; root of context hierarchy
 WARN | destroyApplicationContextOnStop parameter is deprecated, please use shut                                                                     down hooks instead
 INFO | PListStore:/home/lucktroy/apache-activemq-5.4.2/data/localhost/tmp_storag                                                                     e started
 INFO | Using Persistence Adapter: KahaDBPersistenceAdapter[/home/lucktroy/apache                                                                     -activemq-5.4.2/data/kahadb]
 INFO | KahaDB is version 3
 INFO | Recovering from the journal ...
 INFO | Recovery replayed 1 operations from the journal in 0.012 seconds.
 INFO | ActiveMQ 5.4.2 JMS Message Broker (localhost) is starting
 INFO | For help or more information please see: http://activemq.apache.org/
 INFO | Listening for connections at: tcp://node03.cluster:61616
 INFO | Connector openwire Started
 INFO | ActiveMQ JMS Message Broker (localhost, ID:node03.cluster-55179-13651769                                                                     16124-0:1) started
 INFO | jetty-7.1.6.v20100715
 INFO | ActiveMQ WebConsole initialized.
 INFO | Initializing Spring FrameworkServlet 'dispatcher'
 INFO | ActiveMQ Console at http://0.0.0.0:8161/admin
 INFO | Initializing Spring root WebApplicationContext
 INFO | camel-osgi.jar/camel-spring-osgi.jar not detected in classpath
 INFO | Apache Camel 2.4.0 (CamelContext: camel) is starting
 INFO | JMX enabled. Using ManagedManagementStrategy.
 INFO | Found 4 packages with 15 @Converter classes to load
 INFO | Loaded 146 type converters in 0.509 seconds
 INFO | Connector vm://localhost Started
 INFO | Route: route1 started and consuming from: Endpoint[activemq://example.A]
 INFO | Started 1 routes
 INFO | Apache Camel 2.4.0 (CamelContext: camel) started in 1.275 seconds
 INFO | Camel Console at http://0.0.0.0:8161/camel
 INFO | ActiveMQ Web Demos at http://0.0.0.0:8161/demo
 INFO | RESTful file access application at http://0.0.0.0:8161/fileserver
 INFO | Started [email protected]:8161

5. 运行Twister

$ ./start_twister.sh
node02
/home/lucktroy/.bashrc: line 10: /act/Modules/3.2.6/init/bash: No such file or directory
node03
Apr 5, 2013 10:48:55 AM org.apache.activemq.transport.failover.FailoverTransport doReconnect
INFO: Successfully connected to tcp://node03:61616
0    [main] INFO  cgl.imr.worker.DaemonWorker  - Daemon no: 0 started with 8 workers.
/home/lucktroy/.bashrc: line 10: /act/Modules/3.2.6/init/bash: No such file or directory
Apr 5, 2013 10:48:56 AM org.apache.activemq.transport.failover.FailoverTransport doReconnect
INFO: Successfully connected to tcp://node03:61616
2    [main] INFO  cgl.imr.worker.DaemonWorker  - Daemon no: 1 started with 8 workers.

6. 退出Twister

$ ./stop_twister.sh
node02
0    [main] DEBUG cgl.imr.script.StopDaemons  - Stopping daemon on port number: 12500
7    [main] DEBUG cgl.imr.script.StopDaemons  - Finished sending the term signal to daemons in host: node02
node03
0    [main] DEBUG cgl.imr.script.StopDaemons  - Stopping daemon on port number: 12501
6    [main] DEBUG cgl.imr.script.StopDaemons  - Finished sending the term signal to daemons in host: node03

7. 退出ActiveMQ, 进入步骤4的终端, Ctrl+C即可退出，也可以使用暴力kill~~

^C INFO | ActiveMQ Message Broker (localhost, ID:node03.cluster-55179-1365176916
 INFO | ActiveMQ Message Broker (localhost, ID:node03.cluster-55179-136517691612
 INFO | Connector openwire Stopped
 INFO | Connector vm://localhost Stopped
 INFO | PListStore:/home/jbzhang/apache-activemq-5.4.2/data/localhost/tmp_storag
 INFO | Stopping async queue tasks
 INFO | Stopping async topic tasks
 INFO | Stopped KahaDB
 INFO | ActiveMQ JMS Message Broker (localhost, ID:node03.cluster-55179-13651769
 INFO | Connector openwire Stopped
 INFO | ActiveMQ JMS Message Broker (localhost, ID:node03.cluster-55179-13651769

错误分析：

1. 运行中遇到如下错误：

$ Error getting System Properties: 
java.lang.StringIndexOutOfBoundsException: String index out of range: 
-1 
Exception in thread "main" java.lang.NullPointerException 
        at cgl.narada.util.SystemInit.init(SystemInit.java:133) 
        at cgl.narada.node.BrokerNode.(BrokerNode.java:146) 
        at cgl.narada.node.BrokerNode.main(BrokerNode.java:926) 

方案：添加 unset module 到你的 ~/.bash_profile, 如下：

echo "unset module">>~/.bash_profile

参考: 

[1] http://www.iterativemapreduce.org/

[2] https://groups.google.com/forum/#!msg/vscse-big-data-for-science-2010/JUNJbsTp1yE/whZOERAC77QJ