logstash数据采集差8小时问题及解决(原创)

作者：star

1. 软件版本

软件名称	版本号
elasticsearch	2.1.1
logstash	2.3.1

1. 问题解决办法
   解决方案1：

input {
    jdbc {
      jdbc_connection_string => "jdbc:oracle:thin:@10.35.4.97:1521:OMS1"
      jdbc_driver_library => "/home/yarn/logstash-2.3.1/lib/ojdbc14.jar"
      jdbc_driver_class => "Java::oracle.jdbc.driver.OracleDriver"
      jdbc_user => "ypt_jk"
      jdbc_password => "ypt_jk123"
      schedule => "11 18 25 11 *"
      statement => "SELECT '省调' dkflag,id, '' as loglevel, CONTENT AS datastr,'oms' AS sources,'自动化' AS type  ,'人工录入' as generatetype,'' AS host, to_char(RECORDDATE,'YYYY-MM-DD HH24:MI:SS') AS modifydate,RECORDUSER as RECORDUSER,recordyy AS remark FROM AUTO.T_AUTORUNRECORD"
    }
}
filter {
    date {
        match => [ "modifydate", "yyyy-MM-dd HH:mm:ss'GMT+8'"]
        target => "modifydate"
    }
}

output {
    elasticsearch {
        document_id => "%{id}"
        hosts => ["10.35.8.23:9200"]
        index => "test_log"
    }
    stdout {
        codec => json_lines
    }
}

解决方案2：

input {
    jdbc {
      jdbc_connection_string => "jdbc:oracle:thin:@10.35.4.97:1521:OMS1"
      jdbc_driver_library => "/home/yarn/logstash-2.3.1/lib/ojdbc14.jar"
      jdbc_driver_class => "Java::oracle.jdbc.driver.OracleDriver"
      jdbc_user => "ypt_jk"
      jdbc_password => "ypt_jk123"
      schedule => "54 10 25 11 *"
      statement => "SELECT '省调' dkflag,'' as loglevel,CONTENT   AS datastr,'oms' AS sources,'监控' AS type,'系统生成' as generatetype,'' AS host,to_char(OPERATETIME,'YYYY-MM-DD HH24:MI:SS') AS modifydate,RECORDPERSON as RECORDUSER,'' AS remark FROM DISPATCH.T_CONTROL_LOG_CONTENT"
    }
}
filter {
    date {
        match => [ "modifydate", "yyyy-MM-dd HH:mm:ss"]
        target => "modifydate"
        locale => "en"
        timezone => "+00:00"
        }

}

output {
    elasticsearch {
        hosts => ["10.35.8.23:9200"]
        index => "log_t_control_log_content"
        document_type => "oms_log"
    }
    stdout {
        codec => json_lines
    }
}

经测试：两种方案都可以很好的解决此问题

方案三：修改logstash源码

vi logstash-2.3.1\vendor\bundle\jruby\1.9\gems\logstash-core-event-2.3.1-java\lib\logstash\timestamp.rb

UTC = org.joda.time.DateTimeZone.forID(“UTC”)  
修改为：  
UTC = org.joda.time.DateTimeZone.getDefault()  

作者：雨中星辰0
链接：https://www.jianshu.com/p/4bb5470da4a9
来源：简书
简书著作权归作者所有，任何形式的转载都请联系作者获得授权并注明出处。