Kafka源代码环境搭建

软件下载

gradle-3.1-all.zip
ideaIU-2016.2.4.exe
JetbrainsCrack-2.6.2.jar
kafka-0.10.0.1-src.tgz
scala-2.10.6.zip
scala-intellij-bin-2016.2.1.zip
zookeeper-3.4.9.tar.gz

安装、配置

• 解压gradle-3.1-all.zip，把%gradle_home%\bin路径加入到path中

• 安装 ideaIU-2016.2.4.exe，在idea64.exe.vmoptions添加-javaagent:D:\ProgramFiles\JetBrains\JetbrainsCrack-2.6.2.jar，输入任意激活码激活

• 打开idea的plugins，离线安装插件scala-intellij-bin-2016.2.1.zip

• 解压zookeeper-3.4.9.tar.gz，复制zookeeper-3.4.9/conf/zoo_sample.cfg到zookeeper-3.4.9/conf/zoo.cfg，修改为：dataDir=E:\work\zookeeper-3.4.9\data

• 解压 scala-2.10.6.zip，把%scala_home%\bin加入到path中

• 解压kafka-0.10.0.1-src.tgz，修改build.gradle

   import org.ajoberstar.grgit.Grgit
   ScalaCompileOptions.metaClass.daemonServer = true  
   ScalaCompileOptions.metaClass.fork = true  
   ScalaCompileOptions.metaClass.useAnt = false  
   ScalaCompileOptions.metaClass.useCompileDaemon = false  
   buildscript {
  

• 执行gradle idea,完成后再idea中打开此功能

• kafka-0.10.0.1-src\config\server.properties 中修改
  log.dirs=E:\work\kafka-0.10.0.1-src-two\data

• 修改
  def main(args: Array[String]): Unit = {
  try {
  var log4jConfPath = "E:\work\kafka-0.10.0.1-src-two\kafka-0.10.0.1-src\config\log4j.properties";
  PropertyConfigurator.configure(log4jConfPath);
  val serverProps = getPropsFromArgs(args)
  val kafkaServerStartable = KafkaServerStartable.fromProps(serverProps)

启动

• 运行zookeeper-3.4.9\bin\zkServer.cmd
• idea中，配置Run configurations-> application -> new , main class 选择 kafka.Kafka, program arguments 配置 config/server.properties,work dir:E:\work\kafka-0.10.0.1-src-two\kafka-0.10.0.1-src,use classpath of module:core_main; jre:1.8
• idea，运行 run->kafka

测试

• 创建topic，启动消费者
  ./bin/kafka-topics.sh --zookeeper localhost:2181 --create -topic test1 --partitions 3 --replication-factor 1
  ./bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic test1

• 启动生产者，发送消息
  ./bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test1

测试