今天只是在计算留存率的代码中添加了一个条件,结果报错如下:
Log Type: stderr
Log Upload Time: 6-Dec-2019 13:18:45
Log Length: 12549
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/data6/yarn/nm-local-dir/usercache/spark/filecache/0/3565450/__spark_libs__8210665535340896795.zip/slf4j-log4j12-1.7.16.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/app/hadoop-2.6.2/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
[Stage 0:> (0 + 0) / 847]
[Stage 0:> (0 + 8) / 847]
[Stage 0:> (1 + 11) / 847]
[Stage 0:=> (27 + 8) / 847]
[Stage 0:=====> (80 + 8) / 847]
[Stage 0:=========> (152 + 8) / 847]
[Stage 0:=============> (212 + 8) / 847]
[Stage 0:==================> (280 + 9) / 847]
[Stage 0:========================> (378 + 8) / 847]
[Stage 0:=============================> (457 + 8) / 847]
[Stage 0:==================================> (528 + 8) / 847]
[Stage 0:=======================================> (623 + 10) / 847]
[Stage 0:===============================================> (735 + 8) / 847]
[Stage 0:=====================================================> (817 + 8) / 847]
[Stage 1:> (0 + 0) / 822]
[Stage 1:> (0 + 8) / 822]
[Stage 1:> (0 + 20) / 822]
[Stage 1:> (4 + 18) / 822]
[Stage 1:> (6 + 16) / 822]
[Stage 1:> (6 + 20) / 822]
[Stage 1:> (9 + 24) / 822]
[Stage 1:> (9 + 25) / 822]
[Stage 1:> (13 + 24) / 822]
[Stage 1:=> (17 + 24) / 822]
[Stage 1:=> (20 + 24) / 822]
[Stage 1:=> (23 + 24) / 822]
[Stage 1:=> (27 + 24) / 822]
[Stage 1:==> (31 + 24) / 822]
[Stage 1:==> (36 + 24) / 822]
[Stage 1:==> (41 + 20) / 822]
[Stage 1:===> (53 + 20) / 822]
[Stage 1:====> (63 + 20) / 822]
[Stage 1:====> (65 + 20) / 822]
[Stage 1:=====> (76 + 20) / 822]
[Stage 1:=====> (88 + 16) / 822]
[Stage 1:======> (96 + 8) / 822]
[Stage 1:======> (103 + 8) / 822]
[Stage 1:======> (104 + 12) / 822]
[Stage 1:=======> (111 + 12) / 822]
[Stage 1:=======> (113 + 12) / 822]
[Stage 1:=======> (119 + 12) / 822]
[Stage 1:========> (127 + 12) / 822]
[Stage 1:=========> (139 + 12) / 822]
[Stage 1:=========> (143 + 8) / 822]
[Stage 1:=========> (146 + 8) / 822]
[Stage 1:=========> (147 + 8) / 822]
[Stage 1:==========> (150 + 8) / 822]
[Stage 1:==========> (154 + 8) / 822]
[Stage 1:==========> (162 + 20) / 822]
[Stage 1:==========> (166 + 56) / 822]
[Stage 1:===========> (174 + 56) / 822]
[Stage 1:===========> (181 + 56) / 822]
[Stage 1:============> (186 + 57) / 822]
[Stage 1:============> (191 + 64) / 822]
[Stage 1:============> (195 + 64) / 822]
[Stage 1:=============> (200 + 64) / 822]
[Stage 1:=============> (204 + 64) / 822]
[Stage 1:=============> (212 + 61) / 822]
[Stage 1:=============> (213 + 60) / 822]
[Stage 1:==============> (214 + 60) / 822]
[Stage 1:==============> (214 + 64) / 822]
[Stage 1:==============> (214 + 68) / 822]
[Stage 1:==============> (214 + 76) / 822]
[Stage 1:==============> (223 + 80) / 822]
[Stage 1:===============> (240 + 84) / 822]
[Stage 1:=================> (264 + 84) / 822]
[Stage 1:==================> (287 + 81) / 822]
[Stage 1:====================> (317 + 79) / 822]
[Stage 1:=======================> (363 + 50) / 822]
[Stage 1:=========================> (389 + 40) / 822]
[Stage 1:==========================> (399 + 33) / 822]
[Stage 1:==========================> (405 + 31) / 822]
[Stage 1:===========================> (411 + 25) / 822]
[Stage 1:===========================> (412 + 24) / 822]
[Stage 1:===========================> (414 + 24) / 822]
[Stage 1:===========================> (415 + 24) / 822]
[Stage 1:===========================> (424 + 24) / 822]
[Stage 1:============================> (431 + 24) / 822]
[Stage 1:============================> (432 + 24) / 822]
[Stage 1:============================> (433 + 24) / 822]
[Stage 1:=============================> (452 + 24) / 822]
[Stage 1:==============================> (461 + 29) / 822]
[Stage 1:===============================> (476 + 29) / 822]
[Stage 1:================================> (495 + 29) / 822]
[Stage 1:=================================> (515 + 33) / 822]
[Stage 1:===================================> (539 + 30) / 822]
[Stage 1:====================================> (558 + 23) / 822]
[Stage 1:=====================================> (572 + 20) / 822]
[Stage 1:======================================> (584 + 16) / 822]
[Stage 1:=======================================> (594 + 9) / 822]
[Stage 1:=======================================> (594 + 13) / 822]
[Stage 1:=======================================> (596 + 11) / 822]
[Stage 1:========================================> (599 + 8) / 822]
[Stage 1:========================================> (603 + 4) / 822]
[Stage 1:======================================> (603 + 160) / 822]
[Stage 1:========================================> (623 + 148) / 822]
[Stage 1:===============================================> (716 + 57) / 822]
[Stage 1:================================================> (739 + 34) / 822]
[Stage 1:=================================================> (747 + 26) / 822]
[Stage 1:=================================================> (748 + 25) / 822]
[Stage 1:=================================================> (749 + 24) / 822]
[Stage 1:=================================================> (753 + 20) / 822]
[Stage 1:=================================================> (757 + 20) / 822]
[Stage 1:==================================================> (768 + 54) / 822]
[Stage 1:=====================================================>(812 + 10) / 822]
[Stage 1:======================================================>(818 + 4) / 822]
[Stage 5:=====================================================> (794 + 4) / 822]
19/12/06 13:17:33 ERROR ApplicationMaster: User class threw exception: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Unknown column 'NaN' in 'field list'
com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Unknown column 'NaN' in 'field list'
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:404)
at com.mysql.jdbc.Util.getInstance(Util.java:387)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:942)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3966)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3902)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2526)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2673)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2545)
at com.mysql.jdbc.StatementImpl.executeUpdateInternal(StatementImpl.java:1540)
at com.mysql.jdbc.StatementImpl.executeLargeUpdate(StatementImpl.java:2595)
at com.mysql.jdbc.StatementImpl.executeUpdate(StatementImpl.java:1468)
at com.moretv.bi.retention.MedusaDayMemberExpireRetentionRate$.insertSQL(MedusaDayMemberExpireRetentionRate.scala:153)
at com.moretv.bi.retention.MedusaDayMemberExpireRetentionRate$$anonfun$execute$1$$anonfun$apply$1.apply$mcII$sp(MedusaDayMemberExpireRetentionRate.scala:131)
at com.moretv.bi.retention.MedusaDayMemberExpireRetentionRate$$anonfun$execute$1$$anonfun$apply$1.apply(MedusaDayMemberExpireRetentionRate.scala:81)
at com.moretv.bi.retention.MedusaDayMemberExpireRetentionRate$$anonfun$execute$1$$anonfun$apply$1.apply(MedusaDayMemberExpireRetentionRate.scala:81)
at scala.collection.immutable.Range.foreach(Range.scala:166)
at com.moretv.bi.retention.MedusaDayMemberExpireRetentionRate$$anonfun$execute$1.apply(MedusaDayMemberExpireRetentionRate.scala:81)
at com.moretv.bi.retention.MedusaDayMemberExpireRetentionRate$$anonfun$execute$1.apply(MedusaDayMemberExpireRetentionRate.scala:47)
at scala.collection.immutable.Range.foreach(Range.scala:166)
at com.moretv.bi.retention.MedusaDayMemberExpireRetentionRate$.execute(MedusaDayMemberExpireRetentionRate.scala:47)
at com.moretv.bi.util.baseclasee.ModuleClass$.executor(ModuleClass.scala:38)
at com.moretv.bi.retention.MedusaDayMemberExpireRetentionRate$.main(MedusaDayMemberExpireRetentionRate.scala:28)
at com.moretv.bi.retention.MedusaDayMemberExpireRetentionRate.main(MedusaDayMemberExpireRetentionRate.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:635)
可能是在生成数仓的事实表数据时,时间跑的太短了,重新跑一个月的事实表之后,再重新执行该任务发现无报错了。