梦飞天

第57课：SparkSQL案例实战学习笔记

第57课：SparkSQL案例实战学习笔记
本期内容：
1.SparkSQL基础案例实战
2.SparkSQL商业类型的案例

进入Spark官网的sql-programming-guide：
http://spark.apache.org/docs/latest/sql-programming-guide.html#getting-started
可以看到
The entry point into all functionality in Spark SQL is the SQLContext class, or one of its descendants. To create a basic SQLContext, all you need is a SparkContext.
val sc: SparkContext // An existing SparkContext.
val sqlContext = new org.apache.spark.sql.SQLContext(sc)
// this is used to implicitly convert an RDD to a DataFrame.
import sqlContext.implicits._
sqlContext以SparkContext为参数，具有SparkContext的功能，又扩展了SparkContext的功能。

运行SparkSQL前需要做一个配置，即在${SPARK_HOME}/conf目录下，新建hive-site.xml文件，并加入如下内容：

注意：不是把hive的hive-site.xml拷贝过来，而是需要新建hive-site.xml再做这一个配置即可。因为要用SparkSQL操作Hive的话，是把Hive当作数据仓库，数据仓库就需要有元数据和数据本身，要想访问真正的数据，就需要访问元数据。所以只需要配置hive.metastore.uris就可以了。
为什么要配置hive.metastore.uris？
=>因为底层是Hive作数据仓库做存储引擎，SparkSQL是计算引擎，SparkSQL要访问元数据就需要做这个配置。
另外要访问hive的元数据mysql，需要把mysql-connector-java-5.1.35-bin.jar放入Spark的lib目录下（不配置也可？）。
这个配置是每台机都要配置吗？
=>不需要，只要在Hive机上配置即可。
Spark启动时不会到Hive的文件夹读取hive-site.xml。只跟Hive中的元数据有关系，跟Hive本身没关系。因为Hive只是一个数据仓库，不是一个计算引擎，计算引擎是SparkSQL。

1）启动HDFS：${HADOOP_HOME}/sbin/start-dfs.sh
2）启动Spark：${SPARK_HOME}/sbin/start-all.sh
3）启动metastore服务：
hive --service metastore > metastore.log 2>& 1&
4）进入spark目录下的bin目录，启动spark-shell
./spark-shell --master spark://slq1:7077
5）Spark-shell启动后生成一个hiveContext:
val hiveContext = new org.apache.spark.sql.hive.HiveContext(sc)
执行结果：
scala> val hiveContext= new org.apache.spark.sql.hive.HiveContext(sc)
16/03/27 01:45:59 INFO hive.HiveContext: Initializing execution hive, version 1.2.1
16/03/27 01:45:59 INFO client.ClientWrapper: Inspected Hadoop version: 2.6.0
16/03/27 01:45:59 INFO client.ClientWrapper: Loaded org.apache.hadoop.hive.shims.Hadoop23Shims for Hadoop version 2.6.0
16/03/27 01:46:01 INFO hive.metastore: Mestastore configuration hive.metastore.warehouse.dir changed from file:/tmp/spark-b5ed27dd-b732-41f6-bf34-82f3f8fdaa02/metastore to file:/tmp/spark-99274b47-7d87-4fbf-9815-8c913bec38a9/metastore
16/03/27 01:46:01 INFO hive.metastore: Mestastore configuration javax.jdo.option.ConnectionURL changed from jdbc:derby:;databaseName=/tmp/spark-b5ed27dd-b732-41f6-bf34-82f3f8fdaa02/metastore;create=true to jdbc:derby:;databaseName=/tmp/spark-99274b47-7d87-4fbf-9815-8c913bec38a9/metastore;create=true
16/03/27 01:46:01 INFO metastore.HiveMetaStore: 0: Shutting down the object store...
16/03/27 01:46:01 INFO HiveMetaStore.audit: ugi=richard ip=unknown-ip-addr cmd=Shutting down the object store...
16/03/27 01:46:01 INFO metastore.HiveMetaStore: 0: Metastore shutdown complete.
16/03/27 01:46:01 INFO HiveMetaStore.audit: ugi=richard ip=unknown-ip-addr cmd=Metastore shutdown complete.
16/03/27 01:46:01 INFO metastore.HiveMetaStore: 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
16/03/27 01:46:01 INFO metastore.ObjectStore: ObjectStore, initialize called
16/03/27 01:46:01 INFO DataNucleus.Persistence: Property hive.metastore.integral.jdo.pushdown unknown - will be ignored
16/03/27 01:46:01 INFO DataNucleus.Persistence: Property datanucleus.cache.level2 unknown - will be ignored
16/03/27 01:46:02 WARN DataNucleus.Connection: BoneCP specified but not present in CLASSPATH (or one of dependencies)
16/03/27 01:46:02 WARN DataNucleus.Connection: BoneCP specified but not present in CLASSPATH (or one of dependencies)
16/03/27 01:46:09 INFO metastore.ObjectStore: Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order"
16/03/27 01:46:12 INFO DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table.
16/03/27 01:46:12 INFO DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table.
16/03/27 01:46:26 INFO DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table.
16/03/27 01:46:26 INFO DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table.
16/03/27 01:46:30 INFO metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY
16/03/27 01:46:30 INFO metastore.ObjectStore: Initialized ObjectStore
16/03/27 01:46:30 WARN metastore.ObjectStore: Failed to get database default, returning NoSuchObjectException
16/03/27 01:46:32 INFO metastore.HiveMetaStore: Added admin role in metastore
16/03/27 01:46:32 INFO metastore.HiveMetaStore: Added public role in metastore
16/03/27 01:46:33 INFO metastore.HiveMetaStore: No user is added in admin role, since config is empty
16/03/27 01:46:33 INFO session.SessionState: Created local directory: /tmp/1711ab6e-62f0-496d-8f04-de91c2808678_resources
16/03/27 01:46:34 INFO session.SessionState: Created HDFS directory: /tmp/hive/richard/1711ab6e-62f0-496d-8f04-de91c2808678
16/03/27 01:46:34 INFO session.SessionState: Created local directory: /tmp/richard/1711ab6e-62f0-496d-8f04-de91c2808678
16/03/27 01:46:34 INFO session.SessionState: Created HDFS directory: /tmp/hive/richard/1711ab6e-62f0-496d-8f04-de91c2808678/_tmp_space.db
16/03/27 01:46:35 INFO hive.HiveContext: default warehouse location is /user/hive/warehouse
16/03/27 01:46:35 INFO hive.HiveContext: Initializing HiveMetastoreConnection version 1.2.1 using Spark classes.
16/03/27 01:46:35 INFO client.ClientWrapper: Inspected Hadoop version: 2.6.0
16/03/27 01:46:35 INFO client.ClientWrapper: Loaded org.apache.hadoop.hive.shims.Hadoop23Shims for Hadoop version 2.6.0
16/03/27 01:46:41 INFO hive.metastore: Trying to connect to metastore with URI thrift://slq1:9083
16/03/27 01:46:42 INFO hive.metastore: Connected to metastore.
16/03/27 01:46:43 INFO session.SessionState: Created local directory: /tmp/5f541be5-2b00-43a4-92f8-80ebd5e43d13_resources
16/03/27 01:46:43 INFO session.SessionState: Created HDFS directory: /tmp/hive/richard/5f541be5-2b00-43a4-92f8-80ebd5e43d13
16/03/27 01:46:43 INFO session.SessionState: Created local directory: /tmp/richard/5f541be5-2b00-43a4-92f8-80ebd5e43d13
16/03/27 01:46:43 INFO session.SessionState: Created HDFS directory: /tmp/hive/richard/5f541be5-2b00-43a4-92f8-80ebd5e43d13/_tmp_space.db
hiveContext: org.apache.spark.sql.hive.HiveContext = org.apache.spark.sql.hive.HiveContext@3b14d63b

6)这样就可以写sql了：
hiveContext.sql(“use hive”)
执行结果：
scala> hiveContext.sql("use hive")
16/03/27 01:56:24 INFO parse.ParseDriver: Parsing command: use hive
16/03/27 01:56:33 INFO parse.ParseDriver: Parse Completed
16/03/27 01:56:41 INFO log.PerfLogger:
16/03/27 01:56:41 INFO log.PerfLogger:
16/03/27 01:56:41 INFO log.PerfLogger:
16/03/27 01:56:42 INFO log.PerfLogger:
16/03/27 01:56:42 INFO parse.ParseDriver: Parsing command: use hive
16/03/27 01:56:50 INFO parse.ParseDriver: Parse Completed
16/03/27 01:56:50 INFO log.PerfLogger:
16/03/27 01:56:50 INFO log.PerfLogger:
16/03/27 01:56:51 INFO ql.Driver: Semantic Analysis Completed
16/03/27 01:56:51 INFO log.PerfLogger:
16/03/27 01:56:51 INFO ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null)
16/03/27 01:56:51 INFO log.PerfLogger:
16/03/27 01:56:51 INFO ql.Driver: Concurrency mode is disabled, not creating a lock manager
16/03/27 01:56:51 INFO log.PerfLogger:
16/03/27 01:56:51 INFO ql.Driver: Starting command(queryId=richard_20160327015642_7563ed54-0636-47bd-8e0d-c3c9c84d7bd6): use hive
16/03/27 01:56:52 INFO log.PerfLogger:
16/03/27 01:56:52 INFO log.PerfLogger:
16/03/27 01:56:52 INFO log.PerfLogger:
16/03/27 01:56:52 INFO ql.Driver: Starting task [Stage-0:DDL] in serial mode
16/03/27 01:56:52 INFO log.PerfLogger:
16/03/27 01:56:52 INFO log.PerfLogger:
16/03/27 01:56:52 INFO ql.Driver: OK
16/03/27 01:56:52 INFO log.PerfLogger:
16/03/27 01:56:52 INFO log.PerfLogger:
16/03/27 01:56:52 INFO log.PerfLogger:
res0: org.apache.spark.sql.DataFrame = [result: string]

hiveContext.sql(“show tables”).collect.foreach(println)
执行结果：
scala> hiveContext.sql("show tables").collect.foreach(println)
[person,false]

hiveContext.sql(“select count(*) from person”).collect.foreach(println)
执行结果：
scala> hiveContext.sql("select count(*) from person").collect.foreach(println)
16/03/27 02:09:42 INFO parse.ParseDriver: Parsing command: select count(*) from person
16/03/27 02:09:43 INFO parse.ParseDriver: Parse Completed
16/03/27 02:10:19 INFO storage.MemoryStore: Block broadcast_0 stored as values in memory (estimated size 553.5 KB, free 553.5 KB)
16/03/27 02:10:20 INFO storage.MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 41.8 KB, free 595.3 KB)
16/03/27 02:10:20 INFO storage.BlockManagerInfo: Added broadcast_0_piece0 in memory on localhost:43229 (size: 41.8 KB, free: 517.4 MB)
16/03/27 02:10:20 INFO spark.SparkContext: Created broadcast 0 from collect at :30
16/03/27 02:10:29 INFO mapred.FileInputFormat: Total input paths to process : 1
16/03/27 02:10:32 INFO spark.SparkContext: Starting job: collect at :30
16/03/27 02:10:32 INFO scheduler.DAGScheduler: Registering RDD 6 (collect at :30)
16/03/27 02:10:32 INFO scheduler.DAGScheduler: Got job 0 (collect at :30) with 1 output partitions
16/03/27 02:10:32 INFO scheduler.DAGScheduler: Final stage: ResultStage 1 (collect at :30)
16/03/27 02:10:32 INFO scheduler.DAGScheduler: Parents of final stage: List(ShuffleMapStage 0)
16/03/27 02:10:32 INFO scheduler.DAGScheduler: Missing parents: List(ShuffleMapStage 0)
16/03/27 02:10:32 INFO scheduler.DAGScheduler: Submitting ShuffleMapStage 0 (MapPartitionsRDD[6] at collect at :30), which has no missing parents
16/03/27 02:10:33 INFO storage.MemoryStore: Block broadcast_1 stored as values in memory (estimated size 13.1 KB, free 608.3 KB)
16/03/27 02:10:33 INFO storage.MemoryStore: Block broadcast_1_piece0 stored as bytes in memory (estimated size 6.6 KB, free 614.9 KB)
16/03/27 02:10:33 INFO storage.BlockManagerInfo: Added broadcast_1_piece0 in memory on localhost:43229 (size: 6.6 KB, free: 517.4 MB)
16/03/27 02:10:33 INFO spark.SparkContext: Created broadcast 1 from broadcast at DAGScheduler.scala:1006
16/03/27 02:10:33 INFO scheduler.DAGScheduler: Submitting 1 missing tasks from ShuffleMapStage 0 (MapPartitionsRDD[6] at collect at :30)
16/03/27 02:10:34 INFO scheduler.TaskSchedulerImpl: Adding task set 0.0 with 1 tasks
16/03/27 02:10:35 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 0.0 (TID 0, localhost, partition 0,ANY, 2151 bytes)
16/03/27 02:10:35 INFO executor.Executor: Running task 0.0 in stage 0.0 (TID 0)
16/03/27 02:10:36 INFO rdd.HadoopRDD: Input split: hdfs://slq1:9000/user/hive/warehouse/hive.db/person/000000_0:0+11
16/03/27 02:10:37 INFO Configuration.deprecation: mapred.tip.id is deprecated. Instead, use mapreduce.task.id
16/03/27 02:10:37 INFO Configuration.deprecation: mapred.task.id is deprecated. Instead, use mapreduce.task.attempt.id
16/03/27 02:10:37 INFO Configuration.deprecation: mapred.task.is.map is deprecated. Instead, use mapreduce.task.ismap
16/03/27 02:10:37 INFO Configuration.deprecation: mapred.task.partition is deprecated. Instead, use mapreduce.task.partition
16/03/27 02:10:37 INFO Configuration.deprecation: mapred.job.id is deprecated. Instead, use mapreduce.job.id
16/03/27 02:10:45 INFO codegen.GenerateMutableProjection: Code generated in 5525.488798 ms
16/03/27 02:10:46 INFO codegen.GenerateUnsafeProjection: Code generated in 497.07644 ms
16/03/27 02:10:47 INFO codegen.GenerateMutableProjection: Code generated in 326.10197 ms
16/03/27 02:10:47 INFO codegen.GenerateUnsafeRowJoiner: Code generated in 185.831859 ms
16/03/27 02:10:47 INFO codegen.GenerateUnsafeProjection: Code generated in 199.900204 ms
16/03/27 02:10:49 INFO executor.Executor: Finished task 0.0 in stage 0.0 (TID 0). 2500 bytes result sent to driver
16/03/27 02:10:49 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 0.0 (TID 0) in 14563 ms on localhost (1/1)
16/03/27 02:10:49 INFO scheduler.TaskSchedulerImpl: Removed TaskSet 0.0, whose tasks have all completed, from pool
16/03/27 02:10:49 INFO scheduler.DAGScheduler: ShuffleMapStage 0 (collect at :30) finished in 15.180 s
16/03/27 02:10:49 INFO scheduler.DAGScheduler: looking for newly runnable stages
16/03/27 02:10:49 INFO scheduler.DAGScheduler: running: Set()
16/03/27 02:10:49 INFO scheduler.DAGScheduler: waiting: Set(ResultStage 1)
16/03/27 02:10:49 INFO scheduler.DAGScheduler: failed: Set()
16/03/27 02:10:49 INFO scheduler.DAGScheduler: Submitting ResultStage 1 (MapPartitionsRDD[9] at collect at :30), which has no missing parents
16/03/27 02:10:49 INFO storage.MemoryStore: Block broadcast_2 stored as values in memory (estimated size 12.2 KB, free 627.1 KB)
16/03/27 02:10:49 INFO storage.MemoryStore: Block broadcast_2_piece0 stored as bytes in memory (estimated size 6.1 KB, free 633.3 KB)
16/03/27 02:10:49 INFO storage.BlockManagerInfo: Added broadcast_2_piece0 in memory on localhost:43229 (size: 6.1 KB, free: 517.4 MB)
16/03/27 02:10:49 INFO spark.SparkContext: Created broadcast 2 from broadcast at DAGScheduler.scala:1006
16/03/27 02:10:50 INFO scheduler.DAGScheduler: Submitting 1 missing tasks from ResultStage 1 (MapPartitionsRDD[9] at collect at :30)
16/03/27 02:10:50 INFO scheduler.TaskSchedulerImpl: Adding task set 1.0 with 1 tasks
16/03/27 02:10:50 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 1.0 (TID 1, localhost, partition 0,NODE_LOCAL, 1999 bytes)
16/03/27 02:10:50 INFO executor.Executor: Running task 0.0 in stage 1.0 (TID 1)
16/03/27 02:10:50 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 02:10:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 95 ms
16/03/27 02:10:50 INFO codegen.GenerateMutableProjection: Code generated in 176.590518 ms
16/03/27 02:10:52 INFO codegen.GenerateMutableProjection: Code generated in 209.671601 ms
16/03/27 02:10:52 INFO executor.Executor: Finished task 0.0 in stage 1.0 (TID 1). 1830 bytes result sent to driver
16/03/27 02:10:52 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 1.0 (TID 1) in 2267 ms on localhost (1/1)
16/03/27 02:10:52 INFO scheduler.TaskSchedulerImpl: Removed TaskSet 1.0, whose tasks have all completed, from pool
16/03/27 02:10:52 INFO scheduler.DAGScheduler: ResultStage 1 (collect at :30) finished in 2.272 s
16/03/27 02:10:52 INFO scheduler.DAGScheduler: Job 0 finished: collect at :30, took 20.351784 s
[1]

从运行的log可以看出是Spark的引擎对Hive进行查询。

此时访问Mater机的4040端口即可看到刚才执行的JOB的信息：

点击Job名：collect at :30即可查看JOB的信息：

可以看到菜单上有SQL、SQL1、SQL2栏，这里点击SQL2栏可以看到刚才运行的SQL的信息：

对比SparkSQL和Hive的运行时间可以看出SparkSQL从Hive数据仓库中查询数据的速度比直接用Hive查询快几十倍。

下面讲解sqlContext，在spark-shell中输入sqlContext可以看到显示：
org.apache.spark.sql.SQLContext = org.apache.spark.sql.hive.HiveContext@78762e87
说明可以创建多个HiveContext并行查询。

下面实验Spark官网的例子（http://spark.apache.org/docs/latest/sql-programming-guide.html）
DataFrame Operations
DataFrames provide a domain-specific language for structured data manipulation in Scala, Java, Python and R.
Here we include some basic examples of structured data processing using DataFrames:
Scala
Java
Python
R
val sc: SparkContext
// An existing SparkContext.
val sqlContext = new org.apache.spark.sql.SQLContext(sc)
// Create the DataFrame
val df = sqlContext.read.json("examples/src/main/resources/people.json")
// Show the content of the DataFrame
df.show()
// age name// null Michael// 30 Andy// 19 Justin
// Print the schema in a tree format
df.printSchema()
// root// |-- age: long (nullable = true)// |-- name: string (nullable = true)
// Select only the "name" column
df.select("name").show()
// name// Michael// Andy// Justin
// Select everybody, but increment the age by 1
df.select(df("name"), df("age") + 1).show()
// name (age + 1)// Michael null// Andy 31// Justin 20
// Select people older than 21
df.filter(df("age") > 21).show()
// age name// 30 Andy
// Count people by age
df.groupBy("age").count().show()
// age count// null 1// 19 1// 30 1
下面是实验结果：

scala> val df = sqlContext.read.json("/user/data/SparkResources/people.json")
16/03/27 03:17:21 INFO json.JSONRelation: Listing hdfs://slq1:9000/user/data/SparkResources/people.json on driver
16/03/27 03:17:22 INFO storage.MemoryStore: Block broadcast_8 stored as values in memory (estimated size 211.8 KB, free 772.8 KB)
16/03/27 03:17:22 INFO storage.MemoryStore: Block broadcast_8_piece0 stored as bytes in memory (estimated size 19.8 KB, free 792.6 KB)
16/03/27 03:17:22 INFO storage.BlockManagerInfo: Added broadcast_8_piece0 in memory on localhost:43229 (size: 19.8 KB, free: 517.3 MB)
16/03/27 03:17:22 INFO spark.SparkContext: Created broadcast 8 from json at :25
16/03/27 03:17:22 INFO mapred.FileInputFormat: Total input paths to process : 1
16/03/27 03:17:22 INFO spark.SparkContext: Starting job: json at :25
16/03/27 03:17:22 INFO scheduler.DAGScheduler: Got job 3 (json at :25) with 1 output partitions
16/03/27 03:17:22 INFO scheduler.DAGScheduler: Final stage: ResultStage 4 (json at :25)
16/03/27 03:17:22 INFO scheduler.DAGScheduler: Parents of final stage: List()
16/03/27 03:17:22 INFO scheduler.DAGScheduler: Missing parents: List()
16/03/27 03:17:22 INFO scheduler.DAGScheduler: Submitting ResultStage 4 (MapPartitionsRDD[23] at json at :25), which has no missing parents
16/03/27 03:17:22 INFO storage.MemoryStore: Block broadcast_9 stored as values in memory (estimated size 4.3 KB, free 796.9 KB)
16/03/27 03:17:22 INFO storage.MemoryStore: Block broadcast_9_piece0 stored as bytes in memory (estimated size 2.4 KB, free 799.3 KB)
16/03/27 03:17:22 INFO storage.BlockManagerInfo: Added broadcast_9_piece0 in memory on localhost:43229 (size: 2.4 KB, free: 517.3 MB)
16/03/27 03:17:22 INFO spark.SparkContext: Created broadcast 9 from broadcast at DAGScheduler.scala:1006
16/03/27 03:17:22 INFO scheduler.DAGScheduler: Submitting 1 missing tasks from ResultStage 4 (MapPartitionsRDD[23] at json at :25)
16/03/27 03:17:22 INFO scheduler.TaskSchedulerImpl: Adding task set 4.0 with 1 tasks
16/03/27 03:17:22 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 4.0 (TID 4, localhost, partition 0,ANY, 2155 bytes)
16/03/27 03:17:23 INFO executor.Executor: Running task 0.0 in stage 4.0 (TID 4)
16/03/27 03:17:23 INFO rdd.HadoopRDD: Input split: hdfs://slq1:9000/user/data/SparkResources/people.json:0+73
16/03/27 03:17:23 INFO executor.Executor: Finished task 0.0 in stage 4.0 (TID 4). 2845 bytes result sent to driver
16/03/27 03:17:23 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 4.0 (TID 4) in 311 ms on localhost (1/1)
16/03/27 03:17:23 INFO scheduler.TaskSchedulerImpl: Removed TaskSet 4.0, whose tasks have all completed, from pool
16/03/27 03:17:23 INFO scheduler.DAGScheduler: ResultStage 4 (json at :25) finished in 0.316 s
16/03/27 03:17:23 INFO scheduler.DAGScheduler: Job 3 finished: json at :25, took 0.436831 s
df: org.apache.spark.sql.DataFrame = [age: bigint, name: string]

scala> df.show()
16/03/27 03:17:42 INFO storage.MemoryStore: Block broadcast_10 stored as values in memory (estimated size 211.3 KB, free 1010.7 KB)
16/03/27 03:17:42 INFO storage.MemoryStore: Block broadcast_10_piece0 stored as bytes in memory (estimated size 19.7 KB, free 1030.4 KB)
16/03/27 03:17:42 INFO storage.BlockManagerInfo: Added broadcast_10_piece0 in memory on localhost:43229 (size: 19.7 KB, free: 517.3 MB)
16/03/27 03:17:42 INFO spark.SparkContext: Created broadcast 10 from show at :28
16/03/27 03:17:43 INFO storage.MemoryStore: Block broadcast_11 stored as values in memory (estimated size 211.8 KB, free 1242.2 KB)
16/03/27 03:17:43 INFO storage.MemoryStore: Block broadcast_11_piece0 stored as bytes in memory (estimated size 19.8 KB, free 1262.0 KB)
16/03/27 03:17:43 INFO storage.BlockManagerInfo: Added broadcast_11_piece0 in memory on localhost:43229 (size: 19.8 KB, free: 517.3 MB)
16/03/27 03:17:43 INFO spark.SparkContext: Created broadcast 11 from show at :28
16/03/27 03:17:44 INFO mapred.FileInputFormat: Total input paths to process : 1
16/03/27 03:17:44 INFO spark.SparkContext: Starting job: show at :28
16/03/27 03:17:44 INFO scheduler.DAGScheduler: Got job 4 (show at :28) with 1 output partitions
16/03/27 03:17:44 INFO scheduler.DAGScheduler: Final stage: ResultStage 5 (show at :28)
16/03/27 03:17:44 INFO scheduler.DAGScheduler: Parents of final stage: List()
16/03/27 03:17:44 INFO scheduler.DAGScheduler: Missing parents: List()
16/03/27 03:17:44 INFO scheduler.DAGScheduler: Submitting ResultStage 5 (MapPartitionsRDD[29] at show at :28), which has no missing parents
16/03/27 03:17:44 INFO storage.MemoryStore: Block broadcast_12 stored as values in memory (estimated size 5.7 KB, free 1267.6 KB)
16/03/27 03:17:44 INFO storage.MemoryStore: Block broadcast_12_piece0 stored as bytes in memory (estimated size 3.2 KB, free 1270.9 KB)
16/03/27 03:17:44 INFO storage.BlockManagerInfo: Added broadcast_12_piece0 in memory on localhost:43229 (size: 3.2 KB, free: 517.3 MB)
16/03/27 03:17:44 INFO spark.SparkContext: Created broadcast 12 from broadcast at DAGScheduler.scala:1006
16/03/27 03:17:44 INFO scheduler.DAGScheduler: Submitting 1 missing tasks from ResultStage 5 (MapPartitionsRDD[29] at show at :28)
16/03/27 03:17:44 INFO scheduler.TaskSchedulerImpl: Adding task set 5.0 with 1 tasks
16/03/27 03:17:44 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 5.0 (TID 5, localhost, partition 0,ANY, 2155 bytes)
16/03/27 03:17:44 INFO executor.Executor: Running task 0.0 in stage 5.0 (TID 5)
16/03/27 03:17:44 INFO rdd.HadoopRDD: Input split: hdfs://slq1:9000/user/data/SparkResources/people.json:0+73
16/03/27 03:17:45 INFO executor.Executor: Finished task 0.0 in stage 5.0 (TID 5). 2508 bytes result sent to driver
16/03/27 03:17:45 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 5.0 (TID 5) in 464 ms on localhost (1/1)
16/03/27 03:17:45 INFO scheduler.TaskSchedulerImpl: Removed TaskSet 5.0, whose tasks have all completed, from pool
16/03/27 03:17:45 INFO scheduler.DAGScheduler: ResultStage 5 (show at :28) finished in 0.469 s
16/03/27 03:17:45 INFO scheduler.DAGScheduler: Job 4 finished: show at :28, took 0.663429 s
+----+-------+
| age| name|
+----+-------+
|null|Michael|
| 30| Andy|
| 19| Justin|
+----+-------+

scala> df.printSchema
root
|-- age: long (nullable = true)
|-- name: string (nullable = true)

scala> df.printSchema()
root
|-- age: long (nullable = true)
|-- name: string (nullable = true)

scala> df.select("name").show
16/03/27 03:18:34 INFO storage.MemoryStore: Block broadcast_13 stored as values in memory (estimated size 211.3 KB, free 1482.2 KB)
16/03/27 03:18:34 INFO storage.MemoryStore: Block broadcast_13_piece0 stored as bytes in memory (estimated size 19.7 KB, free 1501.9 KB)
16/03/27 03:18:34 INFO storage.BlockManagerInfo: Added broadcast_13_piece0 in memory on localhost:43229 (size: 19.7 KB, free: 517.3 MB)
16/03/27 03:18:34 INFO spark.SparkContext: Created broadcast 13 from show at :28
16/03/27 03:18:34 INFO storage.MemoryStore: Block broadcast_14 stored as values in memory (estimated size 211.8 KB, free 1713.7 KB)
16/03/27 03:18:35 INFO storage.MemoryStore: Block broadcast_14_piece0 stored as bytes in memory (estimated size 19.8 KB, free 1733.5 KB)
16/03/27 03:18:35 INFO storage.BlockManagerInfo: Added broadcast_14_piece0 in memory on localhost:43229 (size: 19.8 KB, free: 517.2 MB)
16/03/27 03:18:35 INFO spark.SparkContext: Created broadcast 14 from show at :28
16/03/27 03:18:35 INFO mapred.FileInputFormat: Total input paths to process : 1
16/03/27 03:18:35 INFO spark.SparkContext: Starting job: show at :28
16/03/27 03:18:35 INFO scheduler.DAGScheduler: Got job 5 (show at :28) with 1 output partitions
16/03/27 03:18:35 INFO scheduler.DAGScheduler: Final stage: ResultStage 6 (show at :28)
16/03/27 03:18:35 INFO scheduler.DAGScheduler: Parents of final stage: List()
16/03/27 03:18:35 INFO scheduler.DAGScheduler: Missing parents: List()
16/03/27 03:18:35 INFO scheduler.DAGScheduler: Submitting ResultStage 6 (MapPartitionsRDD[35] at show at :28), which has no missing parents
16/03/27 03:18:35 INFO storage.MemoryStore: Block broadcast_15 stored as values in memory (estimated size 5.7 KB, free 1739.2 KB)
16/03/27 03:18:35 INFO storage.MemoryStore: Block broadcast_15_piece0 stored as bytes in memory (estimated size 3.3 KB, free 1742.5 KB)
16/03/27 03:18:35 INFO storage.BlockManagerInfo: Added broadcast_15_piece0 in memory on localhost:43229 (size: 3.3 KB, free: 517.2 MB)
16/03/27 03:18:35 INFO spark.SparkContext: Created broadcast 15 from broadcast at DAGScheduler.scala:1006
16/03/27 03:18:35 INFO scheduler.DAGScheduler: Submitting 1 missing tasks from ResultStage 6 (MapPartitionsRDD[35] at show at :28)
16/03/27 03:18:35 INFO scheduler.TaskSchedulerImpl: Adding task set 6.0 with 1 tasks
16/03/27 03:18:35 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 6.0 (TID 6, localhost, partition 0,ANY, 2155 bytes)
16/03/27 03:18:35 INFO executor.Executor: Running task 0.0 in stage 6.0 (TID 6)
16/03/27 03:18:35 INFO rdd.HadoopRDD: Input split: hdfs://slq1:9000/user/data/SparkResources/people.json:0+73
16/03/27 03:18:36 INFO codegen.GenerateUnsafeProjection: Code generated in 238.622957 ms
16/03/27 03:18:36 INFO codegen.GenerateSafeProjection: Code generated in 314.33143 ms
16/03/27 03:18:37 INFO executor.Executor: Finished task 0.0 in stage 6.0 (TID 6). 2415 bytes result sent to driver
16/03/27 03:18:37 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 6.0 (TID 6) in 1211 ms on localhost (1/1)
16/03/27 03:18:37 INFO scheduler.TaskSchedulerImpl: Removed TaskSet 6.0, whose tasks have all completed, from pool
16/03/27 03:18:37 INFO scheduler.DAGScheduler: ResultStage 6 (show at :28) finished in 1.214 s
16/03/27 03:18:37 INFO scheduler.DAGScheduler: Job 5 finished: show at :28, took 1.341020 s
+-------+
| name|
+-------+
|Michael|
| Andy|
| Justin|
+-------+

scala> df.select(df("name"),df("age") + 1).show()
16/03/27 03:20:07 INFO storage.MemoryStore: Block broadcast_16 stored as values in memory (estimated size 211.3 KB, free 1953.8 KB)
16/03/27 03:20:08 INFO storage.MemoryStore: Block broadcast_16_piece0 stored as bytes in memory (estimated size 19.7 KB, free 1973.5 KB)
16/03/27 03:20:08 INFO storage.BlockManagerInfo: Added broadcast_16_piece0 in memory on localhost:43229 (size: 19.7 KB, free: 517.2 MB)
16/03/27 03:20:08 INFO spark.SparkContext: Created broadcast 16 from show at :28
16/03/27 03:20:08 INFO storage.MemoryStore: Block broadcast_17 stored as values in memory (estimated size 211.8 KB, free 2.1 MB)
16/03/27 03:20:08 INFO storage.MemoryStore: Block broadcast_17_piece0 stored as bytes in memory (estimated size 19.8 KB, free 2.2 MB)
16/03/27 03:20:08 INFO storage.BlockManagerInfo: Added broadcast_17_piece0 in memory on localhost:43229 (size: 19.8 KB, free: 517.2 MB)
16/03/27 03:20:08 INFO spark.SparkContext: Created broadcast 17 from show at :28
16/03/27 03:20:09 INFO mapred.FileInputFormat: Total input paths to process : 1
16/03/27 03:20:09 INFO spark.SparkContext: Starting job: show at :28
16/03/27 03:20:09 INFO scheduler.DAGScheduler: Got job 6 (show at :28) with 1 output partitions
16/03/27 03:20:09 INFO scheduler.DAGScheduler: Final stage: ResultStage 7 (show at :28)
16/03/27 03:20:09 INFO scheduler.DAGScheduler: Parents of final stage: List()
16/03/27 03:20:09 INFO scheduler.DAGScheduler: Missing parents: List()
16/03/27 03:20:09 INFO scheduler.DAGScheduler: Submitting ResultStage 7 (MapPartitionsRDD[42] at show at :28), which has no missing parents
16/03/27 03:20:10 INFO storage.MemoryStore: Block broadcast_18 stored as values in memory (estimated size 7.8 KB, free 2.2 MB)
16/03/27 03:20:10 INFO storage.MemoryStore: Block broadcast_18_piece0 stored as bytes in memory (estimated size 4.2 KB, free 2.2 MB)
16/03/27 03:20:10 INFO storage.BlockManagerInfo: Added broadcast_18_piece0 in memory on localhost:43229 (size: 4.2 KB, free: 517.2 MB)
16/03/27 03:20:10 INFO spark.SparkContext: Created broadcast 18 from broadcast at DAGScheduler.scala:1006
16/03/27 03:20:10 INFO scheduler.DAGScheduler: Submitting 1 missing tasks from ResultStage 7 (MapPartitionsRDD[42] at show at :28)
16/03/27 03:20:10 INFO scheduler.TaskSchedulerImpl: Adding task set 7.0 with 1 tasks
16/03/27 03:20:10 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 7.0 (TID 7, localhost, partition 0,ANY, 2155 bytes)
16/03/27 03:20:10 INFO executor.Executor: Running task 0.0 in stage 7.0 (TID 7)
16/03/27 03:20:10 INFO rdd.HadoopRDD: Input split: hdfs://slq1:9000/user/data/SparkResources/people.json:0+73
16/03/27 03:20:10 INFO codegen.GenerateUnsafeProjection: Code generated in 331.131861 ms
16/03/27 03:20:11 INFO codegen.GenerateUnsafeProjection: Code generated in 245.055735 ms
16/03/27 03:20:11 INFO codegen.GenerateSafeProjection: Code generated in 203.81864 ms
16/03/27 03:20:11 INFO executor.Executor: Finished task 0.0 in stage 7.0 (TID 7). 2653 bytes result sent to driver
16/03/27 03:20:11 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 7.0 (TID 7) in 1438 ms on localhost (1/1)
16/03/27 03:20:11 INFO scheduler.TaskSchedulerImpl: Removed TaskSet 7.0, whose tasks have all completed, from pool
16/03/27 03:20:11 INFO scheduler.DAGScheduler: ResultStage 7 (show at :28) finished in 1.442 s
16/03/27 03:20:11 INFO scheduler.DAGScheduler: Job 6 finished: show at :28, took 1.643127 s
+-------+---------+
| name|(age + 1)|
+-------+---------+
|Michael| null|
| Andy| 31|
| Justin| 20|
+-------+---------+

scala> df.filter(df("age") > 21).show()
16/03/27 03:20:44 INFO storage.MemoryStore: Block broadcast_19 stored as values in memory (estimated size 211.3 KB, free 2.4 MB)
16/03/27 03:20:44 INFO storage.MemoryStore: Block broadcast_19_piece0 stored as bytes in memory (estimated size 19.7 KB, free 2.4 MB)
16/03/27 03:20:44 INFO storage.BlockManagerInfo: Added broadcast_19_piece0 in memory on localhost:43229 (size: 19.7 KB, free: 517.2 MB)
16/03/27 03:20:44 INFO spark.SparkContext: Created broadcast 19 from show at :28
16/03/27 03:20:44 INFO storage.MemoryStore: Block broadcast_20 stored as values in memory (estimated size 211.8 KB, free 2.6 MB)
16/03/27 03:20:44 INFO storage.MemoryStore: Block broadcast_20_piece0 stored as bytes in memory (estimated size 19.8 KB, free 2.6 MB)
16/03/27 03:20:44 INFO storage.BlockManagerInfo: Added broadcast_20_piece0 in memory on localhost:43229 (size: 19.8 KB, free: 517.2 MB)
16/03/27 03:20:44 INFO spark.SparkContext: Created broadcast 20 from show at :28
16/03/27 03:20:45 INFO mapred.FileInputFormat: Total input paths to process : 1
16/03/27 03:20:45 INFO spark.SparkContext: Starting job: show at :28
16/03/27 03:20:45 INFO scheduler.DAGScheduler: Got job 7 (show at :28) with 1 output partitions
16/03/27 03:20:45 INFO scheduler.DAGScheduler: Final stage: ResultStage 8 (show at :28)
16/03/27 03:20:45 INFO scheduler.DAGScheduler: Parents of final stage: List()
16/03/27 03:20:45 INFO scheduler.DAGScheduler: Missing parents: List()
16/03/27 03:20:45 INFO scheduler.DAGScheduler: Submitting ResultStage 8 (MapPartitionsRDD[49] at show at :28), which has no missing parents
16/03/27 03:20:45 INFO storage.MemoryStore: Block broadcast_21 stored as values in memory (estimated size 7.7 KB, free 2.6 MB)
16/03/27 03:20:45 INFO storage.MemoryStore: Block broadcast_21_piece0 stored as bytes in memory (estimated size 4.1 KB, free 2.6 MB)
16/03/27 03:20:45 INFO storage.BlockManagerInfo: Added broadcast_21_piece0 in memory on localhost:43229 (size: 4.1 KB, free: 517.2 MB)
16/03/27 03:20:45 INFO spark.SparkContext: Created broadcast 21 from broadcast at DAGScheduler.scala:1006
16/03/27 03:20:45 INFO scheduler.DAGScheduler: Submitting 1 missing tasks from ResultStage 8 (MapPartitionsRDD[49] at show at :28)
16/03/27 03:20:45 INFO scheduler.TaskSchedulerImpl: Adding task set 8.0 with 1 tasks
16/03/27 03:20:45 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 8.0 (TID 8, localhost, partition 0,ANY, 2155 bytes)
16/03/27 03:20:45 INFO executor.Executor: Running task 0.0 in stage 8.0 (TID 8)
16/03/27 03:20:45 INFO rdd.HadoopRDD: Input split: hdfs://slq1:9000/user/data/SparkResources/people.json:0+73
16/03/27 03:20:46 INFO codegen.GeneratePredicate: Code generated in 220.458295 ms
16/03/27 03:20:46 INFO executor.Executor: Finished task 0.0 in stage 8.0 (TID 8). 2636 bytes result sent to driver
16/03/27 03:20:46 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 8.0 (TID 8) in 664 ms on localhost (1/1)
16/03/27 03:20:46 INFO scheduler.TaskSchedulerImpl: Removed TaskSet 8.0, whose tasks have all completed, from pool
16/03/27 03:20:46 INFO scheduler.DAGScheduler: ResultStage 8 (show at :28) finished in 0.670 s
16/03/27 03:20:46 INFO scheduler.DAGScheduler: Job 7 finished: show at :28, took 0.794079 s
+---+----+
|age|name|
+---+----+
| 30|Andy|
+---+----+

scala> df.groupBy("age").count().show()
16/03/27 03:21:13 INFO storage.MemoryStore: Block broadcast_22 stored as values in memory (estimated size 211.3 KB, free 2.8 MB)
16/03/27 03:21:14 INFO storage.MemoryStore: Block broadcast_22_piece0 stored as bytes in memory (estimated size 19.7 KB, free 2.9 MB)
16/03/27 03:21:14 INFO storage.BlockManagerInfo: Added broadcast_22_piece0 in memory on localhost:43229 (size: 19.7 KB, free: 517.1 MB)
16/03/27 03:21:14 INFO spark.SparkContext: Created broadcast 22 from show at :28
16/03/27 03:21:14 INFO storage.MemoryStore: Block broadcast_23 stored as values in memory (estimated size 211.8 KB, free 3.1 MB)
16/03/27 03:21:14 INFO storage.MemoryStore: Block broadcast_23_piece0 stored as bytes in memory (estimated size 19.8 KB, free 3.1 MB)
16/03/27 03:21:14 INFO storage.BlockManagerInfo: Added broadcast_23_piece0 in memory on localhost:43229 (size: 19.8 KB, free: 517.1 MB)
16/03/27 03:21:14 INFO spark.SparkContext: Created broadcast 23 from show at :28
16/03/27 03:21:15 INFO mapred.FileInputFormat: Total input paths to process : 1
16/03/27 03:21:15 INFO spark.SparkContext: Starting job: show at :28
16/03/27 03:21:15 INFO scheduler.DAGScheduler: Registering RDD 55 (show at :28)
16/03/27 03:21:15 INFO scheduler.DAGScheduler: Got job 8 (show at :28) with 1 output partitions
16/03/27 03:21:15 INFO scheduler.DAGScheduler: Final stage: ResultStage 10 (show at :28)
16/03/27 03:21:15 INFO scheduler.DAGScheduler: Parents of final stage: List(ShuffleMapStage 9)
16/03/27 03:21:15 INFO scheduler.DAGScheduler: Missing parents: List(ShuffleMapStage 9)
16/03/27 03:21:15 INFO scheduler.DAGScheduler: Submitting ShuffleMapStage 9 (MapPartitionsRDD[55] at show at :28), which has no missing parents
16/03/27 03:21:16 INFO storage.MemoryStore: Block broadcast_24 stored as values in memory (estimated size 10.7 KB, free 3.1 MB)
16/03/27 03:21:16 INFO storage.MemoryStore: Block broadcast_24_piece0 stored as bytes in memory (estimated size 5.5 KB, free 3.1 MB)
16/03/27 03:21:16 INFO storage.BlockManagerInfo: Added broadcast_24_piece0 in memory on localhost:43229 (size: 5.5 KB, free: 517.1 MB)
16/03/27 03:21:16 INFO spark.SparkContext: Created broadcast 24 from broadcast at DAGScheduler.scala:1006
16/03/27 03:21:16 INFO scheduler.DAGScheduler: Submitting 1 missing tasks from ShuffleMapStage 9 (MapPartitionsRDD[55] at show at :28)
16/03/27 03:21:16 INFO scheduler.TaskSchedulerImpl: Adding task set 9.0 with 1 tasks
16/03/27 03:21:16 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 9.0 (TID 9, localhost, partition 0,ANY, 2144 bytes)
16/03/27 03:21:16 INFO executor.Executor: Running task 0.0 in stage 9.0 (TID 9)
16/03/27 03:21:16 INFO rdd.HadoopRDD: Input split: hdfs://slq1:9000/user/data/SparkResources/people.json:0+73
16/03/27 03:21:16 INFO codegen.GenerateUnsafeRowJoiner: Code generated in 210.413957 ms
16/03/27 03:21:17 INFO codegen.GenerateMutableProjection: Code generated in 240.118575 ms
16/03/27 03:21:17 INFO storage.BlockManagerInfo: Removed broadcast_3_piece0 on localhost:43229 in memory (size: 19.8 KB, free: 517.1 MB)
16/03/27 03:21:17 INFO spark.ContextCleaner: Cleaned accumulator 12
16/03/27 03:21:17 INFO storage.BlockManagerInfo: Removed broadcast_4_piece0 on localhost:43229 in memory (size: 2.4 KB, free: 517.1 MB)
16/03/27 03:21:17 INFO storage.BlockManagerInfo: Removed broadcast_5_piece0 on localhost:43229 in memory (size: 19.7 KB, free: 517.1 MB)
16/03/27 03:21:17 INFO storage.BlockManagerInfo: Removed broadcast_6_piece0 on localhost:43229 in memory (size: 19.8 KB, free: 517.2 MB)
16/03/27 03:21:17 INFO spark.ContextCleaner: Cleaned accumulator 13
16/03/27 03:21:17 INFO storage.BlockManagerInfo: Removed broadcast_7_piece0 on localhost:43229 in memory (size: 3.2 KB, free: 517.2 MB)
16/03/27 03:21:17 INFO storage.BlockManagerInfo: Removed broadcast_8_piece0 on localhost:43229 in memory (size: 19.8 KB, free: 517.2 MB)
16/03/27 03:21:17 INFO spark.ContextCleaner: Cleaned accumulator 14
16/03/27 03:21:17 INFO storage.BlockManagerInfo: Removed broadcast_9_piece0 on localhost:43229 in memory (size: 2.4 KB, free: 517.2 MB)
16/03/27 03:21:17 INFO storage.BlockManagerInfo: Removed broadcast_10_piece0 on localhost:43229 in memory (size: 19.7 KB, free: 517.2 MB)
16/03/27 03:21:18 INFO storage.BlockManagerInfo: Removed broadcast_11_piece0 on localhost:43229 in memory (size: 19.8 KB, free: 517.2 MB)
16/03/27 03:21:18 INFO spark.ContextCleaner: Cleaned accumulator 15
16/03/27 03:21:18 INFO storage.BlockManagerInfo: Removed broadcast_12_piece0 on localhost:43229 in memory (size: 3.2 KB, free: 517.2 MB)
16/03/27 03:21:18 INFO storage.BlockManagerInfo: Removed broadcast_13_piece0 on localhost:43229 in memory (size: 19.7 KB, free: 517.3 MB)
16/03/27 03:21:18 INFO storage.BlockManagerInfo: Removed broadcast_14_piece0 on localhost:43229 in memory (size: 19.8 KB, free: 517.3 MB)
16/03/27 03:21:18 INFO spark.ContextCleaner: Cleaned accumulator 16
16/03/27 03:21:18 INFO storage.BlockManagerInfo: Removed broadcast_15_piece0 on localhost:43229 in memory (size: 3.3 KB, free: 517.3 MB)
16/03/27 03:21:18 INFO storage.BlockManagerInfo: Removed broadcast_16_piece0 on localhost:43229 in memory (size: 19.7 KB, free: 517.3 MB)
16/03/27 03:21:18 INFO storage.BlockManagerInfo: Removed broadcast_17_piece0 on localhost:43229 in memory (size: 19.8 KB, free: 517.3 MB)
16/03/27 03:21:18 INFO spark.ContextCleaner: Cleaned accumulator 17
16/03/27 03:21:18 INFO spark.ContextCleaner: Cleaned accumulator 18
16/03/27 03:21:18 INFO storage.BlockManagerInfo: Removed broadcast_18_piece0 on localhost:43229 in memory (size: 4.2 KB, free: 517.3 MB)
16/03/27 03:21:18 INFO storage.BlockManagerInfo: Removed broadcast_19_piece0 on localhost:43229 in memory (size: 19.7 KB, free: 517.3 MB)
16/03/27 03:21:18 INFO storage.BlockManagerInfo: Removed broadcast_20_piece0 on localhost:43229 in memory (size: 19.8 KB, free: 517.4 MB)
16/03/27 03:21:18 INFO spark.ContextCleaner: Cleaned accumulator 19
16/03/27 03:21:18 INFO spark.ContextCleaner: Cleaned accumulator 20
16/03/27 03:21:18 INFO spark.ContextCleaner: Cleaned accumulator 21
16/03/27 03:21:18 INFO storage.BlockManagerInfo: Removed broadcast_21_piece0 on localhost:43229 in memory (size: 4.1 KB, free: 517.4 MB)
16/03/27 03:21:18 INFO storage.BlockManagerInfo: Removed broadcast_22_piece0 on localhost:43229 in memory (size: 19.7 KB, free: 517.4 MB)
16/03/27 03:21:20 INFO executor.Executor: Finished task 0.0 in stage 9.0 (TID 9). 2702 bytes result sent to driver
16/03/27 03:21:20 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 9.0 (TID 9) in 4092 ms on localhost (1/1)
16/03/27 03:21:20 INFO scheduler.TaskSchedulerImpl: Removed TaskSet 9.0, whose tasks have all completed, from pool
16/03/27 03:21:20 INFO scheduler.DAGScheduler: ShuffleMapStage 9 (show at :28) finished in 4.101 s
16/03/27 03:21:20 INFO scheduler.DAGScheduler: looking for newly runnable stages
16/03/27 03:21:20 INFO scheduler.DAGScheduler: running: Set()
16/03/27 03:21:20 INFO scheduler.DAGScheduler: waiting: Set(ResultStage 10)
16/03/27 03:21:20 INFO scheduler.DAGScheduler: failed: Set()
16/03/27 03:21:20 INFO scheduler.DAGScheduler: Submitting ResultStage 10 (MapPartitionsRDD[59] at show at :28), which has no missing parents
16/03/27 03:21:20 INFO storage.MemoryStore: Block broadcast_25 stored as values in memory (estimated size 12.8 KB, free 260.6 KB)
16/03/27 03:21:20 INFO storage.MemoryStore: Block broadcast_25_piece0 stored as bytes in memory (estimated size 6.8 KB, free 267.4 KB)
16/03/27 03:21:20 INFO storage.BlockManagerInfo: Added broadcast_25_piece0 in memory on localhost:43229 (size: 6.8 KB, free: 517.4 MB)
16/03/27 03:21:20 INFO spark.SparkContext: Created broadcast 25 from broadcast at DAGScheduler.scala:1006
16/03/27 03:21:20 INFO scheduler.DAGScheduler: Submitting 1 missing tasks from ResultStage 10 (MapPartitionsRDD[59] at show at :28)
16/03/27 03:21:20 INFO scheduler.TaskSchedulerImpl: Adding task set 10.0 with 1 tasks
16/03/27 03:21:20 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 10.0 (TID 10, localhost, partition 0,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:20 INFO executor.Executor: Running task 0.0 in stage 10.0 (TID 10)
16/03/27 03:21:20 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:20 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:20 INFO codegen.GenerateSafeProjection: Code generated in 185.34935 ms
16/03/27 03:21:20 INFO executor.Executor: Finished task 0.0 in stage 10.0 (TID 10). 1609 bytes result sent to driver
16/03/27 03:21:20 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 10.0 (TID 10) in 454 ms on localhost (1/1)
16/03/27 03:21:20 INFO scheduler.TaskSchedulerImpl: Removed TaskSet 10.0, whose tasks have all completed, from pool
16/03/27 03:21:20 INFO scheduler.DAGScheduler: ResultStage 10 (show at :28) finished in 0.459 s
16/03/27 03:21:20 INFO scheduler.DAGScheduler: Job 8 finished: show at :28, took 5.076179 s
16/03/27 03:21:21 INFO spark.SparkContext: Starting job: show at :28
16/03/27 03:21:21 INFO spark.MapOutputTrackerMaster: Size of output statuses for shuffle 1 is 150 bytes
16/03/27 03:21:21 INFO scheduler.DAGScheduler: Got job 9 (show at :28) with 199 output partitions
16/03/27 03:21:21 INFO scheduler.DAGScheduler: Final stage: ResultStage 12 (show at :28)
16/03/27 03:21:21 INFO scheduler.DAGScheduler: Parents of final stage: List(ShuffleMapStage 11)
16/03/27 03:21:21 INFO scheduler.DAGScheduler: Missing parents: List()
16/03/27 03:21:21 INFO scheduler.DAGScheduler: Submitting ResultStage 12 (MapPartitionsRDD[59] at show at :28), which has no missing parents
16/03/27 03:21:21 INFO storage.MemoryStore: Block broadcast_26 stored as values in memory (estimated size 12.8 KB, free 280.2 KB)
16/03/27 03:21:21 INFO storage.MemoryStore: Block broadcast_26_piece0 stored as bytes in memory (estimated size 6.8 KB, free 287.0 KB)
16/03/27 03:21:21 INFO storage.BlockManagerInfo: Added broadcast_26_piece0 in memory on localhost:43229 (size: 6.8 KB, free: 517.4 MB)
16/03/27 03:21:21 INFO spark.SparkContext: Created broadcast 26 from broadcast at DAGScheduler.scala:1006
16/03/27 03:21:21 INFO scheduler.DAGScheduler: Submitting 199 missing tasks from ResultStage 12 (MapPartitionsRDD[59] at show at :28)
16/03/27 03:21:21 INFO scheduler.TaskSchedulerImpl: Adding task set 12.0 with 199 tasks
16/03/27 03:21:22 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 12.0 (TID 11, localhost, partition 1,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:22 INFO executor.Executor: Running task 0.0 in stage 12.0 (TID 11)
16/03/27 03:21:22 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:22 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:22 INFO executor.Executor: Finished task 0.0 in stage 12.0 (TID 11). 1609 bytes result sent to driver
16/03/27 03:21:22 INFO scheduler.TaskSetManager: Starting task 1.0 in stage 12.0 (TID 12, localhost, partition 2,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:22 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 12.0 (TID 11) in 137 ms on localhost (1/199)
16/03/27 03:21:22 INFO executor.Executor: Running task 1.0 in stage 12.0 (TID 12)
16/03/27 03:21:22 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:22 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
16/03/27 03:21:22 INFO executor.Executor: Finished task 1.0 in stage 12.0 (TID 12). 1609 bytes result sent to driver
16/03/27 03:21:22 INFO scheduler.TaskSetManager: Starting task 2.0 in stage 12.0 (TID 13, localhost, partition 3,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:22 INFO scheduler.TaskSetManager: Finished task 1.0 in stage 12.0 (TID 12) in 230 ms on localhost (2/199)
16/03/27 03:21:22 INFO executor.Executor: Running task 2.0 in stage 12.0 (TID 13)
16/03/27 03:21:22 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:22 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:22 INFO executor.Executor: Finished task 2.0 in stage 12.0 (TID 13). 1609 bytes result sent to driver
16/03/27 03:21:22 INFO scheduler.TaskSetManager: Starting task 3.0 in stage 12.0 (TID 14, localhost, partition 4,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:22 INFO scheduler.TaskSetManager: Finished task 2.0 in stage 12.0 (TID 13) in 172 ms on localhost (3/199)
16/03/27 03:21:22 INFO executor.Executor: Running task 3.0 in stage 12.0 (TID 14)
16/03/27 03:21:22 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:22 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:22 INFO executor.Executor: Finished task 3.0 in stage 12.0 (TID 14). 1609 bytes result sent to driver
16/03/27 03:21:22 INFO scheduler.TaskSetManager: Starting task 4.0 in stage 12.0 (TID 15, localhost, partition 5,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:22 INFO scheduler.TaskSetManager: Finished task 3.0 in stage 12.0 (TID 14) in 90 ms on localhost (4/199)
16/03/27 03:21:22 INFO executor.Executor: Running task 4.0 in stage 12.0 (TID 15)
16/03/27 03:21:22 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:22 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:22 INFO executor.Executor: Finished task 4.0 in stage 12.0 (TID 15). 1609 bytes result sent to driver
16/03/27 03:21:22 INFO scheduler.TaskSetManager: Starting task 5.0 in stage 12.0 (TID 16, localhost, partition 6,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:22 INFO scheduler.TaskSetManager: Finished task 4.0 in stage 12.0 (TID 15) in 143 ms on localhost (5/199)
16/03/27 03:21:22 INFO executor.Executor: Running task 5.0 in stage 12.0 (TID 16)
16/03/27 03:21:22 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:22 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:22 INFO executor.Executor: Finished task 5.0 in stage 12.0 (TID 16). 1609 bytes result sent to driver
16/03/27 03:21:22 INFO scheduler.TaskSetManager: Starting task 6.0 in stage 12.0 (TID 17, localhost, partition 7,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:22 INFO scheduler.TaskSetManager: Finished task 5.0 in stage 12.0 (TID 16) in 182 ms on localhost (6/199)
16/03/27 03:21:22 INFO executor.Executor: Running task 6.0 in stage 12.0 (TID 17)
16/03/27 03:21:23 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:23 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:23 INFO executor.Executor: Finished task 6.0 in stage 12.0 (TID 17). 1609 bytes result sent to driver
16/03/27 03:21:23 INFO scheduler.TaskSetManager: Starting task 7.0 in stage 12.0 (TID 18, localhost, partition 8,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:23 INFO scheduler.TaskSetManager: Finished task 6.0 in stage 12.0 (TID 17) in 149 ms on localhost (7/199)
16/03/27 03:21:23 INFO executor.Executor: Running task 7.0 in stage 12.0 (TID 18)
16/03/27 03:21:23 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:23 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:23 INFO executor.Executor: Finished task 7.0 in stage 12.0 (TID 18). 1609 bytes result sent to driver
16/03/27 03:21:23 INFO scheduler.TaskSetManager: Starting task 8.0 in stage 12.0 (TID 19, localhost, partition 9,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:23 INFO scheduler.TaskSetManager: Finished task 7.0 in stage 12.0 (TID 18) in 228 ms on localhost (8/199)
16/03/27 03:21:23 INFO executor.Executor: Running task 8.0 in stage 12.0 (TID 19)
16/03/27 03:21:23 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:23 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:23 INFO executor.Executor: Finished task 8.0 in stage 12.0 (TID 19). 1609 bytes result sent to driver
16/03/27 03:21:23 INFO scheduler.TaskSetManager: Starting task 9.0 in stage 12.0 (TID 20, localhost, partition 10,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:23 INFO scheduler.TaskSetManager: Finished task 8.0 in stage 12.0 (TID 19) in 98 ms on localhost (9/199)
16/03/27 03:21:23 INFO executor.Executor: Running task 9.0 in stage 12.0 (TID 20)
16/03/27 03:21:23 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:23 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:23 INFO executor.Executor: Finished task 9.0 in stage 12.0 (TID 20). 1609 bytes result sent to driver
16/03/27 03:21:23 INFO scheduler.TaskSetManager: Starting task 10.0 in stage 12.0 (TID 21, localhost, partition 11,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:23 INFO scheduler.TaskSetManager: Finished task 9.0 in stage 12.0 (TID 20) in 103 ms on localhost (10/199)
16/03/27 03:21:23 INFO executor.Executor: Running task 10.0 in stage 12.0 (TID 21)
16/03/27 03:21:23 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:23 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
16/03/27 03:21:23 INFO executor.Executor: Finished task 10.0 in stage 12.0 (TID 21). 1609 bytes result sent to driver
16/03/27 03:21:23 INFO scheduler.TaskSetManager: Starting task 11.0 in stage 12.0 (TID 22, localhost, partition 12,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:23 INFO scheduler.TaskSetManager: Finished task 10.0 in stage 12.0 (TID 21) in 141 ms on localhost (11/199)
16/03/27 03:21:23 INFO executor.Executor: Running task 11.0 in stage 12.0 (TID 22)
16/03/27 03:21:23 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:23 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:23 INFO executor.Executor: Finished task 11.0 in stage 12.0 (TID 22). 1609 bytes result sent to driver
16/03/27 03:21:23 INFO scheduler.TaskSetManager: Starting task 12.0 in stage 12.0 (TID 23, localhost, partition 13,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:23 INFO scheduler.TaskSetManager: Finished task 11.0 in stage 12.0 (TID 22) in 152 ms on localhost (12/199)
16/03/27 03:21:23 INFO executor.Executor: Running task 12.0 in stage 12.0 (TID 23)
16/03/27 03:21:23 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:23 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
16/03/27 03:21:23 INFO executor.Executor: Finished task 12.0 in stage 12.0 (TID 23). 1609 bytes result sent to driver
16/03/27 03:21:23 INFO scheduler.TaskSetManager: Starting task 13.0 in stage 12.0 (TID 24, localhost, partition 14,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:23 INFO scheduler.TaskSetManager: Finished task 12.0 in stage 12.0 (TID 23) in 103 ms on localhost (13/199)
16/03/27 03:21:23 INFO executor.Executor: Running task 13.0 in stage 12.0 (TID 24)
16/03/27 03:21:24 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:24 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:24 INFO executor.Executor: Finished task 13.0 in stage 12.0 (TID 24). 1609 bytes result sent to driver
16/03/27 03:21:24 INFO scheduler.TaskSetManager: Starting task 14.0 in stage 12.0 (TID 25, localhost, partition 15,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:24 INFO scheduler.TaskSetManager: Finished task 13.0 in stage 12.0 (TID 24) in 181 ms on localhost (14/199)
16/03/27 03:21:24 INFO executor.Executor: Running task 14.0 in stage 12.0 (TID 25)
16/03/27 03:21:24 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:24 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:24 INFO executor.Executor: Finished task 14.0 in stage 12.0 (TID 25). 1609 bytes result sent to driver
16/03/27 03:21:24 INFO scheduler.TaskSetManager: Starting task 15.0 in stage 12.0 (TID 26, localhost, partition 16,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:24 INFO scheduler.TaskSetManager: Finished task 14.0 in stage 12.0 (TID 25) in 146 ms on localhost (15/199)
16/03/27 03:21:24 INFO executor.Executor: Running task 15.0 in stage 12.0 (TID 26)
16/03/27 03:21:24 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:24 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
16/03/27 03:21:24 INFO executor.Executor: Finished task 15.0 in stage 12.0 (TID 26). 1609 bytes result sent to driver
16/03/27 03:21:24 INFO scheduler.TaskSetManager: Starting task 16.0 in stage 12.0 (TID 27, localhost, partition 17,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:24 INFO scheduler.TaskSetManager: Finished task 15.0 in stage 12.0 (TID 26) in 236 ms on localhost (16/199)
16/03/27 03:21:24 INFO executor.Executor: Running task 16.0 in stage 12.0 (TID 27)
16/03/27 03:21:24 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:24 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:24 INFO executor.Executor: Finished task 16.0 in stage 12.0 (TID 27). 1609 bytes result sent to driver
16/03/27 03:21:24 INFO scheduler.TaskSetManager: Starting task 17.0 in stage 12.0 (TID 28, localhost, partition 18,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:24 INFO scheduler.TaskSetManager: Finished task 16.0 in stage 12.0 (TID 27) in 182 ms on localhost (17/199)
16/03/27 03:21:24 INFO executor.Executor: Running task 17.0 in stage 12.0 (TID 28)
16/03/27 03:21:24 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:24 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:24 INFO executor.Executor: Finished task 17.0 in stage 12.0 (TID 28). 1609 bytes result sent to driver
16/03/27 03:21:24 INFO scheduler.TaskSetManager: Starting task 18.0 in stage 12.0 (TID 29, localhost, partition 19,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:24 INFO scheduler.TaskSetManager: Finished task 17.0 in stage 12.0 (TID 28) in 154 ms on localhost (18/199)
16/03/27 03:21:24 INFO executor.Executor: Running task 18.0 in stage 12.0 (TID 29)
16/03/27 03:21:24 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:24 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:24 INFO executor.Executor: Finished task 18.0 in stage 12.0 (TID 29). 1609 bytes result sent to driver
16/03/27 03:21:24 INFO scheduler.TaskSetManager: Starting task 19.0 in stage 12.0 (TID 30, localhost, partition 20,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:24 INFO scheduler.TaskSetManager: Finished task 18.0 in stage 12.0 (TID 29) in 193 ms on localhost (19/199)
16/03/27 03:21:25 INFO executor.Executor: Running task 19.0 in stage 12.0 (TID 30)
16/03/27 03:21:25 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:25 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
16/03/27 03:21:25 INFO executor.Executor: Finished task 19.0 in stage 12.0 (TID 30). 1609 bytes result sent to driver
16/03/27 03:21:25 INFO scheduler.TaskSetManager: Starting task 20.0 in stage 12.0 (TID 31, localhost, partition 21,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:25 INFO scheduler.TaskSetManager: Finished task 19.0 in stage 12.0 (TID 30) in 120 ms on localhost (20/199)
16/03/27 03:21:25 INFO executor.Executor: Running task 20.0 in stage 12.0 (TID 31)
16/03/27 03:21:25 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:25 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:25 INFO executor.Executor: Finished task 20.0 in stage 12.0 (TID 31). 1609 bytes result sent to driver
16/03/27 03:21:25 INFO scheduler.TaskSetManager: Starting task 21.0 in stage 12.0 (TID 32, localhost, partition 22,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:25 INFO scheduler.TaskSetManager: Finished task 20.0 in stage 12.0 (TID 31) in 190 ms on localhost (21/199)
16/03/27 03:21:25 INFO executor.Executor: Running task 21.0 in stage 12.0 (TID 32)
16/03/27 03:21:25 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:25 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:25 INFO executor.Executor: Finished task 21.0 in stage 12.0 (TID 32). 1609 bytes result sent to driver
16/03/27 03:21:25 INFO scheduler.TaskSetManager: Starting task 22.0 in stage 12.0 (TID 33, localhost, partition 23,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:25 INFO scheduler.TaskSetManager: Finished task 21.0 in stage 12.0 (TID 32) in 216 ms on localhost (22/199)
16/03/27 03:21:25 INFO executor.Executor: Running task 22.0 in stage 12.0 (TID 33)
16/03/27 03:21:25 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:25 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:25 INFO executor.Executor: Finished task 22.0 in stage 12.0 (TID 33). 1609 bytes result sent to driver
16/03/27 03:21:25 INFO scheduler.TaskSetManager: Starting task 23.0 in stage 12.0 (TID 34, localhost, partition 24,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:25 INFO scheduler.TaskSetManager: Finished task 22.0 in stage 12.0 (TID 33) in 204 ms on localhost (23/199)
16/03/27 03:21:25 INFO executor.Executor: Running task 23.0 in stage 12.0 (TID 34)
16/03/27 03:21:25 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:25 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 30 ms
16/03/27 03:21:25 INFO executor.Executor: Finished task 23.0 in stage 12.0 (TID 34). 1609 bytes result sent to driver
16/03/27 03:21:25 INFO scheduler.TaskSetManager: Starting task 24.0 in stage 12.0 (TID 35, localhost, partition 25,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:25 INFO scheduler.TaskSetManager: Finished task 23.0 in stage 12.0 (TID 34) in 143 ms on localhost (24/199)
16/03/27 03:21:25 INFO executor.Executor: Running task 24.0 in stage 12.0 (TID 35)
16/03/27 03:21:25 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:25 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:26 INFO executor.Executor: Finished task 24.0 in stage 12.0 (TID 35). 1609 bytes result sent to driver
16/03/27 03:21:26 INFO scheduler.TaskSetManager: Starting task 25.0 in stage 12.0 (TID 36, localhost, partition 26,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:26 INFO scheduler.TaskSetManager: Finished task 24.0 in stage 12.0 (TID 35) in 203 ms on localhost (25/199)
16/03/27 03:21:26 INFO executor.Executor: Running task 25.0 in stage 12.0 (TID 36)
16/03/27 03:21:26 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:26 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:26 INFO executor.Executor: Finished task 25.0 in stage 12.0 (TID 36). 1609 bytes result sent to driver
16/03/27 03:21:26 INFO scheduler.TaskSetManager: Starting task 26.0 in stage 12.0 (TID 37, localhost, partition 27,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:26 INFO scheduler.TaskSetManager: Finished task 25.0 in stage 12.0 (TID 36) in 172 ms on localhost (26/199)
16/03/27 03:21:26 INFO executor.Executor: Running task 26.0 in stage 12.0 (TID 37)
16/03/27 03:21:26 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:26 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:26 INFO executor.Executor: Finished task 26.0 in stage 12.0 (TID 37). 1609 bytes result sent to driver
16/03/27 03:21:26 INFO scheduler.TaskSetManager: Starting task 27.0 in stage 12.0 (TID 38, localhost, partition 28,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:26 INFO scheduler.TaskSetManager: Finished task 26.0 in stage 12.0 (TID 37) in 110 ms on localhost (27/199)
16/03/27 03:21:26 INFO executor.Executor: Running task 27.0 in stage 12.0 (TID 38)
16/03/27 03:21:26 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:26 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms
16/03/27 03:21:26 INFO executor.Executor: Finished task 27.0 in stage 12.0 (TID 38). 1609 bytes result sent to driver
16/03/27 03:21:26 INFO scheduler.TaskSetManager: Starting task 28.0 in stage 12.0 (TID 39, localhost, partition 29,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:26 INFO scheduler.TaskSetManager: Finished task 27.0 in stage 12.0 (TID 38) in 173 ms on localhost (28/199)
16/03/27 03:21:26 INFO executor.Executor: Running task 28.0 in stage 12.0 (TID 39)
16/03/27 03:21:26 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:26 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:26 INFO executor.Executor: Finished task 28.0 in stage 12.0 (TID 39). 1609 bytes result sent to driver
16/03/27 03:21:26 INFO scheduler.TaskSetManager: Starting task 29.0 in stage 12.0 (TID 40, localhost, partition 30,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:26 INFO scheduler.TaskSetManager: Finished task 28.0 in stage 12.0 (TID 39) in 182 ms on localhost (29/199)
16/03/27 03:21:26 INFO executor.Executor: Running task 29.0 in stage 12.0 (TID 40)
16/03/27 03:21:26 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:26 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms
16/03/27 03:21:26 INFO executor.Executor: Finished task 29.0 in stage 12.0 (TID 40). 1609 bytes result sent to driver
16/03/27 03:21:26 INFO scheduler.TaskSetManager: Starting task 30.0 in stage 12.0 (TID 41, localhost, partition 31,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:26 INFO scheduler.TaskSetManager: Finished task 29.0 in stage 12.0 (TID 40) in 162 ms on localhost (30/199)
16/03/27 03:21:26 INFO executor.Executor: Running task 30.0 in stage 12.0 (TID 41)
16/03/27 03:21:26 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:26 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:26 INFO executor.Executor: Finished task 30.0 in stage 12.0 (TID 41). 1609 bytes result sent to driver
16/03/27 03:21:26 INFO scheduler.TaskSetManager: Starting task 31.0 in stage 12.0 (TID 42, localhost, partition 32,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:26 INFO scheduler.TaskSetManager: Finished task 30.0 in stage 12.0 (TID 41) in 118 ms on localhost (31/199)
16/03/27 03:21:27 INFO executor.Executor: Running task 31.0 in stage 12.0 (TID 42)
16/03/27 03:21:27 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:27 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
16/03/27 03:21:27 INFO executor.Executor: Finished task 31.0 in stage 12.0 (TID 42). 1609 bytes result sent to driver
16/03/27 03:21:27 INFO scheduler.TaskSetManager: Starting task 32.0 in stage 12.0 (TID 43, localhost, partition 33,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:27 INFO scheduler.TaskSetManager: Finished task 31.0 in stage 12.0 (TID 42) in 175 ms on localhost (32/199)
16/03/27 03:21:27 INFO executor.Executor: Running task 32.0 in stage 12.0 (TID 43)
16/03/27 03:21:27 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:27 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
16/03/27 03:21:27 INFO executor.Executor: Finished task 32.0 in stage 12.0 (TID 43). 1609 bytes result sent to driver
16/03/27 03:21:27 INFO scheduler.TaskSetManager: Starting task 33.0 in stage 12.0 (TID 44, localhost, partition 34,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:27 INFO scheduler.TaskSetManager: Finished task 32.0 in stage 12.0 (TID 43) in 263 ms on localhost (33/199)
16/03/27 03:21:27 INFO executor.Executor: Running task 33.0 in stage 12.0 (TID 44)
16/03/27 03:21:27 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:27 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:27 INFO executor.Executor: Finished task 33.0 in stage 12.0 (TID 44). 1609 bytes result sent to driver
16/03/27 03:21:27 INFO scheduler.TaskSetManager: Starting task 34.0 in stage 12.0 (TID 45, localhost, partition 35,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:27 INFO scheduler.TaskSetManager: Finished task 33.0 in stage 12.0 (TID 44) in 162 ms on localhost (34/199)
16/03/27 03:21:27 INFO executor.Executor: Running task 34.0 in stage 12.0 (TID 45)
16/03/27 03:21:27 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:27 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:27 INFO executor.Executor: Finished task 34.0 in stage 12.0 (TID 45). 1609 bytes result sent to driver
16/03/27 03:21:27 INFO scheduler.TaskSetManager: Starting task 35.0 in stage 12.0 (TID 46, localhost, partition 36,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:27 INFO scheduler.TaskSetManager: Finished task 34.0 in stage 12.0 (TID 45) in 181 ms on localhost (35/199)
16/03/27 03:21:27 INFO executor.Executor: Running task 35.0 in stage 12.0 (TID 46)
16/03/27 03:21:27 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:27 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 25 ms
16/03/27 03:21:27 INFO executor.Executor: Finished task 35.0 in stage 12.0 (TID 46). 1609 bytes result sent to driver
16/03/27 03:21:27 INFO scheduler.TaskSetManager: Starting task 36.0 in stage 12.0 (TID 47, localhost, partition 37,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:27 INFO scheduler.TaskSetManager: Finished task 35.0 in stage 12.0 (TID 46) in 231 ms on localhost (36/199)
16/03/27 03:21:27 INFO executor.Executor: Running task 36.0 in stage 12.0 (TID 47)
16/03/27 03:21:28 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:28 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:28 INFO executor.Executor: Finished task 36.0 in stage 12.0 (TID 47). 1609 bytes result sent to driver
16/03/27 03:21:28 INFO scheduler.TaskSetManager: Starting task 37.0 in stage 12.0 (TID 48, localhost, partition 38,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:28 INFO scheduler.TaskSetManager: Finished task 36.0 in stage 12.0 (TID 47) in 207 ms on localhost (37/199)
16/03/27 03:21:28 INFO executor.Executor: Running task 37.0 in stage 12.0 (TID 48)
16/03/27 03:21:28 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:28 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
16/03/27 03:21:28 INFO executor.Executor: Finished task 37.0 in stage 12.0 (TID 48). 1609 bytes result sent to driver
16/03/27 03:21:28 INFO scheduler.TaskSetManager: Starting task 38.0 in stage 12.0 (TID 49, localhost, partition 39,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:28 INFO scheduler.TaskSetManager: Finished task 37.0 in stage 12.0 (TID 48) in 135 ms on localhost (38/199)
16/03/27 03:21:28 INFO executor.Executor: Running task 38.0 in stage 12.0 (TID 49)
16/03/27 03:21:28 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:28 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:28 INFO executor.Executor: Finished task 38.0 in stage 12.0 (TID 49). 1609 bytes result sent to driver
16/03/27 03:21:28 INFO scheduler.TaskSetManager: Starting task 39.0 in stage 12.0 (TID 50, localhost, partition 40,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:28 INFO scheduler.TaskSetManager: Finished task 38.0 in stage 12.0 (TID 49) in 135 ms on localhost (39/199)
16/03/27 03:21:28 INFO executor.Executor: Running task 39.0 in stage 12.0 (TID 50)
16/03/27 03:21:28 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:28 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:28 INFO executor.Executor: Finished task 39.0 in stage 12.0 (TID 50). 1609 bytes result sent to driver
16/03/27 03:21:28 INFO scheduler.TaskSetManager: Starting task 40.0 in stage 12.0 (TID 51, localhost, partition 41,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:28 INFO scheduler.TaskSetManager: Finished task 39.0 in stage 12.0 (TID 50) in 124 ms on localhost (40/199)
16/03/27 03:21:28 INFO executor.Executor: Running task 40.0 in stage 12.0 (TID 51)
16/03/27 03:21:28 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:28 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:28 INFO executor.Executor: Finished task 40.0 in stage 12.0 (TID 51). 1609 bytes result sent to driver
16/03/27 03:21:28 INFO scheduler.TaskSetManager: Starting task 41.0 in stage 12.0 (TID 52, localhost, partition 42,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:28 INFO scheduler.TaskSetManager: Finished task 40.0 in stage 12.0 (TID 51) in 166 ms on localhost (41/199)
16/03/27 03:21:28 INFO executor.Executor: Running task 41.0 in stage 12.0 (TID 52)
16/03/27 03:21:28 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:28 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:28 INFO executor.Executor: Finished task 41.0 in stage 12.0 (TID 52). 1609 bytes result sent to driver
16/03/27 03:21:28 INFO scheduler.TaskSetManager: Starting task 42.0 in stage 12.0 (TID 53, localhost, partition 43,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:28 INFO scheduler.TaskSetManager: Finished task 41.0 in stage 12.0 (TID 52) in 226 ms on localhost (42/199)
16/03/27 03:21:28 INFO executor.Executor: Running task 42.0 in stage 12.0 (TID 53)
16/03/27 03:21:28 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:28 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:29 INFO executor.Executor: Finished task 42.0 in stage 12.0 (TID 53). 1609 bytes result sent to driver
16/03/27 03:21:29 INFO scheduler.TaskSetManager: Starting task 43.0 in stage 12.0 (TID 54, localhost, partition 44,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:29 INFO scheduler.TaskSetManager: Finished task 42.0 in stage 12.0 (TID 53) in 136 ms on localhost (43/199)
16/03/27 03:21:29 INFO executor.Executor: Running task 43.0 in stage 12.0 (TID 54)
16/03/27 03:21:29 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:29 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 57 ms
16/03/27 03:21:29 INFO executor.Executor: Finished task 43.0 in stage 12.0 (TID 54). 1609 bytes result sent to driver
16/03/27 03:21:29 INFO scheduler.TaskSetManager: Starting task 44.0 in stage 12.0 (TID 55, localhost, partition 45,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:29 INFO scheduler.TaskSetManager: Finished task 43.0 in stage 12.0 (TID 54) in 110 ms on localhost (44/199)
16/03/27 03:21:29 INFO executor.Executor: Running task 44.0 in stage 12.0 (TID 55)
16/03/27 03:21:29 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:29 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:29 INFO executor.Executor: Finished task 44.0 in stage 12.0 (TID 55). 1609 bytes result sent to driver
16/03/27 03:21:29 INFO scheduler.TaskSetManager: Starting task 45.0 in stage 12.0 (TID 56, localhost, partition 46,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:29 INFO scheduler.TaskSetManager: Finished task 44.0 in stage 12.0 (TID 55) in 94 ms on localhost (45/199)
16/03/27 03:21:29 INFO executor.Executor: Running task 45.0 in stage 12.0 (TID 56)
16/03/27 03:21:29 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:29 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
16/03/27 03:21:29 INFO executor.Executor: Finished task 45.0 in stage 12.0 (TID 56). 1609 bytes result sent to driver
16/03/27 03:21:29 INFO scheduler.TaskSetManager: Starting task 46.0 in stage 12.0 (TID 57, localhost, partition 47,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:29 INFO scheduler.TaskSetManager: Finished task 45.0 in stage 12.0 (TID 56) in 76 ms on localhost (46/199)
16/03/27 03:21:29 INFO executor.Executor: Running task 46.0 in stage 12.0 (TID 57)
16/03/27 03:21:29 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:29 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:29 INFO executor.Executor: Finished task 46.0 in stage 12.0 (TID 57). 1609 bytes result sent to driver
16/03/27 03:21:29 INFO scheduler.TaskSetManager: Starting task 47.0 in stage 12.0 (TID 58, localhost, partition 48,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:29 INFO scheduler.TaskSetManager: Finished task 46.0 in stage 12.0 (TID 57) in 132 ms on localhost (47/199)
16/03/27 03:21:29 INFO executor.Executor: Running task 47.0 in stage 12.0 (TID 58)
16/03/27 03:21:29 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:29 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
16/03/27 03:21:29 INFO executor.Executor: Finished task 47.0 in stage 12.0 (TID 58). 1609 bytes result sent to driver
16/03/27 03:21:29 INFO scheduler.TaskSetManager: Starting task 48.0 in stage 12.0 (TID 59, localhost, partition 49,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:29 INFO scheduler.TaskSetManager: Finished task 47.0 in stage 12.0 (TID 58) in 140 ms on localhost (48/199)
16/03/27 03:21:29 INFO executor.Executor: Running task 48.0 in stage 12.0 (TID 59)
16/03/27 03:21:29 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:29 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
16/03/27 03:21:29 INFO executor.Executor: Finished task 48.0 in stage 12.0 (TID 59). 1609 bytes result sent to driver
16/03/27 03:21:29 INFO scheduler.TaskSetManager: Starting task 49.0 in stage 12.0 (TID 60, localhost, partition 50,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:29 INFO scheduler.TaskSetManager: Finished task 48.0 in stage 12.0 (TID 59) in 82 ms on localhost (49/199)
16/03/27 03:21:29 INFO executor.Executor: Running task 49.0 in stage 12.0 (TID 60)
16/03/27 03:21:29 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:29 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 64 ms
16/03/27 03:21:29 INFO executor.Executor: Finished task 49.0 in stage 12.0 (TID 60). 1609 bytes result sent to driver
16/03/27 03:21:29 INFO scheduler.TaskSetManager: Starting task 50.0 in stage 12.0 (TID 61, localhost, partition 51,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:29 INFO scheduler.TaskSetManager: Finished task 49.0 in stage 12.0 (TID 60) in 117 ms on localhost (50/199)
16/03/27 03:21:29 INFO executor.Executor: Running task 50.0 in stage 12.0 (TID 61)
16/03/27 03:21:29 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:29 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:29 INFO executor.Executor: Finished task 50.0 in stage 12.0 (TID 61). 1609 bytes result sent to driver
16/03/27 03:21:29 INFO scheduler.TaskSetManager: Starting task 51.0 in stage 12.0 (TID 62, localhost, partition 52,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:29 INFO scheduler.TaskSetManager: Finished task 50.0 in stage 12.0 (TID 61) in 158 ms on localhost (51/199)
16/03/27 03:21:29 INFO executor.Executor: Running task 51.0 in stage 12.0 (TID 62)
16/03/27 03:21:29 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:29 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
16/03/27 03:21:30 INFO executor.Executor: Finished task 51.0 in stage 12.0 (TID 62). 1609 bytes result sent to driver
16/03/27 03:21:30 INFO scheduler.TaskSetManager: Starting task 52.0 in stage 12.0 (TID 63, localhost, partition 53,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:30 INFO scheduler.TaskSetManager: Finished task 51.0 in stage 12.0 (TID 62) in 162 ms on localhost (52/199)
16/03/27 03:21:30 INFO executor.Executor: Running task 52.0 in stage 12.0 (TID 63)
16/03/27 03:21:30 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:30 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
16/03/27 03:21:30 INFO executor.Executor: Finished task 52.0 in stage 12.0 (TID 63). 1609 bytes result sent to driver
16/03/27 03:21:30 INFO scheduler.TaskSetManager: Starting task 53.0 in stage 12.0 (TID 64, localhost, partition 54,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:30 INFO scheduler.TaskSetManager: Finished task 52.0 in stage 12.0 (TID 63) in 122 ms on localhost (53/199)
16/03/27 03:21:30 INFO executor.Executor: Running task 53.0 in stage 12.0 (TID 64)
16/03/27 03:21:30 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:30 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:30 INFO executor.Executor: Finished task 53.0 in stage 12.0 (TID 64). 1609 bytes result sent to driver
16/03/27 03:21:30 INFO scheduler.TaskSetManager: Starting task 54.0 in stage 12.0 (TID 65, localhost, partition 55,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:30 INFO scheduler.TaskSetManager: Finished task 53.0 in stage 12.0 (TID 64) in 145 ms on localhost (54/199)
16/03/27 03:21:30 INFO executor.Executor: Running task 54.0 in stage 12.0 (TID 65)
16/03/27 03:21:30 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:30 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 41 ms
16/03/27 03:21:30 INFO executor.Executor: Finished task 54.0 in stage 12.0 (TID 65). 1609 bytes result sent to driver
16/03/27 03:21:30 INFO scheduler.TaskSetManager: Starting task 55.0 in stage 12.0 (TID 66, localhost, partition 56,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:30 INFO scheduler.TaskSetManager: Finished task 54.0 in stage 12.0 (TID 65) in 106 ms on localhost (55/199)
16/03/27 03:21:30 INFO executor.Executor: Running task 55.0 in stage 12.0 (TID 66)
16/03/27 03:21:30 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:30 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:30 INFO executor.Executor: Finished task 55.0 in stage 12.0 (TID 66). 1609 bytes result sent to driver
16/03/27 03:21:30 INFO scheduler.TaskSetManager: Starting task 56.0 in stage 12.0 (TID 67, localhost, partition 57,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:30 INFO scheduler.TaskSetManager: Finished task 55.0 in stage 12.0 (TID 66) in 140 ms on localhost (56/199)
16/03/27 03:21:30 INFO executor.Executor: Running task 56.0 in stage 12.0 (TID 67)
16/03/27 03:21:30 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:30 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:30 INFO executor.Executor: Finished task 56.0 in stage 12.0 (TID 67). 1609 bytes result sent to driver
16/03/27 03:21:30 INFO scheduler.TaskSetManager: Starting task 57.0 in stage 12.0 (TID 68, localhost, partition 58,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:30 INFO scheduler.TaskSetManager: Finished task 56.0 in stage 12.0 (TID 67) in 81 ms on localhost (57/199)
16/03/27 03:21:30 INFO executor.Executor: Running task 57.0 in stage 12.0 (TID 68)
16/03/27 03:21:30 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:30 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 64 ms
16/03/27 03:21:30 INFO executor.Executor: Finished task 57.0 in stage 12.0 (TID 68). 1609 bytes result sent to driver
16/03/27 03:21:30 INFO scheduler.TaskSetManager: Starting task 58.0 in stage 12.0 (TID 69, localhost, partition 59,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:30 INFO scheduler.TaskSetManager: Finished task 57.0 in stage 12.0 (TID 68) in 145 ms on localhost (58/199)
16/03/27 03:21:30 INFO executor.Executor: Running task 58.0 in stage 12.0 (TID 69)
16/03/27 03:21:30 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:30 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:30 INFO executor.Executor: Finished task 58.0 in stage 12.0 (TID 69). 1609 bytes result sent to driver
16/03/27 03:21:30 INFO scheduler.TaskSetManager: Starting task 59.0 in stage 12.0 (TID 70, localhost, partition 60,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:30 INFO scheduler.TaskSetManager: Finished task 58.0 in stage 12.0 (TID 69) in 64 ms on localhost (59/199)
16/03/27 03:21:30 INFO executor.Executor: Running task 59.0 in stage 12.0 (TID 70)
16/03/27 03:21:30 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:30 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:31 INFO executor.Executor: Finished task 59.0 in stage 12.0 (TID 70). 1609 bytes result sent to driver
16/03/27 03:21:31 INFO scheduler.TaskSetManager: Starting task 60.0 in stage 12.0 (TID 71, localhost, partition 61,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:31 INFO scheduler.TaskSetManager: Finished task 59.0 in stage 12.0 (TID 70) in 176 ms on localhost (60/199)
16/03/27 03:21:31 INFO executor.Executor: Running task 60.0 in stage 12.0 (TID 71)
16/03/27 03:21:31 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:31 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
16/03/27 03:21:31 INFO executor.Executor: Finished task 60.0 in stage 12.0 (TID 71). 1609 bytes result sent to driver
16/03/27 03:21:31 INFO scheduler.TaskSetManager: Starting task 61.0 in stage 12.0 (TID 72, localhost, partition 62,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:31 INFO scheduler.TaskSetManager: Finished task 60.0 in stage 12.0 (TID 71) in 163 ms on localhost (61/199)
16/03/27 03:21:31 INFO executor.Executor: Running task 61.0 in stage 12.0 (TID 72)
16/03/27 03:21:31 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:31 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:31 INFO executor.Executor: Finished task 61.0 in stage 12.0 (TID 72). 1609 bytes result sent to driver
16/03/27 03:21:31 INFO scheduler.TaskSetManager: Starting task 62.0 in stage 12.0 (TID 73, localhost, partition 63,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:31 INFO scheduler.TaskSetManager: Finished task 61.0 in stage 12.0 (TID 72) in 234 ms on localhost (62/199)
16/03/27 03:21:31 INFO executor.Executor: Running task 62.0 in stage 12.0 (TID 73)
16/03/27 03:21:31 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:31 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:31 INFO executor.Executor: Finished task 62.0 in stage 12.0 (TID 73). 1609 bytes result sent to driver
16/03/27 03:21:31 INFO scheduler.TaskSetManager: Starting task 63.0 in stage 12.0 (TID 74, localhost, partition 64,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:31 INFO scheduler.TaskSetManager: Finished task 62.0 in stage 12.0 (TID 73) in 121 ms on localhost (63/199)
16/03/27 03:21:31 INFO executor.Executor: Running task 63.0 in stage 12.0 (TID 74)
16/03/27 03:21:31 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:31 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:31 INFO executor.Executor: Finished task 63.0 in stage 12.0 (TID 74). 1609 bytes result sent to driver
16/03/27 03:21:31 INFO scheduler.TaskSetManager: Starting task 64.0 in stage 12.0 (TID 75, localhost, partition 65,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:31 INFO scheduler.TaskSetManager: Finished task 63.0 in stage 12.0 (TID 74) in 125 ms on localhost (64/199)
16/03/27 03:21:31 INFO executor.Executor: Running task 64.0 in stage 12.0 (TID 75)
16/03/27 03:21:31 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:31 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:31 INFO executor.Executor: Finished task 64.0 in stage 12.0 (TID 75). 1609 bytes result sent to driver
16/03/27 03:21:31 INFO scheduler.TaskSetManager: Starting task 65.0 in stage 12.0 (TID 76, localhost, partition 66,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:31 INFO scheduler.TaskSetManager: Finished task 64.0 in stage 12.0 (TID 75) in 157 ms on localhost (65/199)
16/03/27 03:21:31 INFO executor.Executor: Running task 65.0 in stage 12.0 (TID 76)
16/03/27 03:21:31 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:31 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
16/03/27 03:21:31 INFO executor.Executor: Finished task 65.0 in stage 12.0 (TID 76). 1609 bytes result sent to driver
16/03/27 03:21:31 INFO scheduler.TaskSetManager: Starting task 66.0 in stage 12.0 (TID 77, localhost, partition 67,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:31 INFO scheduler.TaskSetManager: Finished task 65.0 in stage 12.0 (TID 76) in 187 ms on localhost (66/199)
16/03/27 03:21:31 INFO executor.Executor: Running task 66.0 in stage 12.0 (TID 77)
16/03/27 03:21:31 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:31 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:31 INFO executor.Executor: Finished task 66.0 in stage 12.0 (TID 77). 1609 bytes result sent to driver
16/03/27 03:21:31 INFO scheduler.TaskSetManager: Starting task 67.0 in stage 12.0 (TID 78, localhost, partition 68,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:31 INFO scheduler.TaskSetManager: Finished task 66.0 in stage 12.0 (TID 77) in 81 ms on localhost (67/199)
16/03/27 03:21:32 INFO executor.Executor: Running task 67.0 in stage 12.0 (TID 78)
16/03/27 03:21:32 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:32 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:32 INFO executor.Executor: Finished task 67.0 in stage 12.0 (TID 78). 1609 bytes result sent to driver
16/03/27 03:21:32 INFO scheduler.TaskSetManager: Starting task 68.0 in stage 12.0 (TID 79, localhost, partition 69,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:32 INFO scheduler.TaskSetManager: Finished task 67.0 in stage 12.0 (TID 78) in 156 ms on localhost (68/199)
16/03/27 03:21:32 INFO executor.Executor: Running task 68.0 in stage 12.0 (TID 79)
16/03/27 03:21:32 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:32 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:32 INFO executor.Executor: Finished task 68.0 in stage 12.0 (TID 79). 1609 bytes result sent to driver
16/03/27 03:21:32 INFO scheduler.TaskSetManager: Starting task 69.0 in stage 12.0 (TID 80, localhost, partition 70,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:32 INFO scheduler.TaskSetManager: Finished task 68.0 in stage 12.0 (TID 79) in 81 ms on localhost (69/199)
16/03/27 03:21:32 INFO executor.Executor: Running task 69.0 in stage 12.0 (TID 80)
16/03/27 03:21:32 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:32 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:32 INFO executor.Executor: Finished task 69.0 in stage 12.0 (TID 80). 1609 bytes result sent to driver
16/03/27 03:21:32 INFO scheduler.TaskSetManager: Starting task 70.0 in stage 12.0 (TID 81, localhost, partition 71,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:32 INFO scheduler.TaskSetManager: Finished task 69.0 in stage 12.0 (TID 80) in 108 ms on localhost (70/199)
16/03/27 03:21:32 INFO executor.Executor: Running task 70.0 in stage 12.0 (TID 81)
16/03/27 03:21:32 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:32 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:32 INFO executor.Executor: Finished task 70.0 in stage 12.0 (TID 81). 1609 bytes result sent to driver
16/03/27 03:21:32 INFO scheduler.TaskSetManager: Starting task 71.0 in stage 12.0 (TID 82, localhost, partition 72,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:32 INFO scheduler.TaskSetManager: Finished task 70.0 in stage 12.0 (TID 81) in 126 ms on localhost (71/199)
16/03/27 03:21:32 INFO executor.Executor: Running task 71.0 in stage 12.0 (TID 82)
16/03/27 03:21:32 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:32 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:32 INFO executor.Executor: Finished task 71.0 in stage 12.0 (TID 82). 1609 bytes result sent to driver
16/03/27 03:21:32 INFO scheduler.TaskSetManager: Starting task 72.0 in stage 12.0 (TID 83, localhost, partition 73,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:32 INFO scheduler.TaskSetManager: Finished task 71.0 in stage 12.0 (TID 82) in 159 ms on localhost (72/199)
16/03/27 03:21:32 INFO executor.Executor: Running task 72.0 in stage 12.0 (TID 83)
16/03/27 03:21:32 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:32 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
16/03/27 03:21:32 INFO executor.Executor: Finished task 72.0 in stage 12.0 (TID 83). 1609 bytes result sent to driver
16/03/27 03:21:32 INFO scheduler.TaskSetManager: Starting task 73.0 in stage 12.0 (TID 84, localhost, partition 74,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:32 INFO scheduler.TaskSetManager: Finished task 72.0 in stage 12.0 (TID 83) in 123 ms on localhost (73/199)
16/03/27 03:21:32 INFO executor.Executor: Running task 73.0 in stage 12.0 (TID 84)
16/03/27 03:21:32 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:32 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:32 INFO executor.Executor: Finished task 73.0 in stage 12.0 (TID 84). 1609 bytes result sent to driver
16/03/27 03:21:32 INFO scheduler.TaskSetManager: Starting task 74.0 in stage 12.0 (TID 85, localhost, partition 75,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:32 INFO scheduler.TaskSetManager: Finished task 73.0 in stage 12.0 (TID 84) in 112 ms on localhost (74/199)
16/03/27 03:21:32 INFO executor.Executor: Running task 74.0 in stage 12.0 (TID 85)
16/03/27 03:21:33 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:33 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:33 INFO executor.Executor: Finished task 74.0 in stage 12.0 (TID 85). 1609 bytes result sent to driver
16/03/27 03:21:33 INFO scheduler.TaskSetManager: Starting task 75.0 in stage 12.0 (TID 86, localhost, partition 76,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:33 INFO scheduler.TaskSetManager: Finished task 74.0 in stage 12.0 (TID 85) in 199 ms on localhost (75/199)
16/03/27 03:21:33 INFO executor.Executor: Running task 75.0 in stage 12.0 (TID 86)
16/03/27 03:21:33 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:33 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:33 INFO executor.Executor: Finished task 75.0 in stage 12.0 (TID 86). 1609 bytes result sent to driver
16/03/27 03:21:33 INFO scheduler.TaskSetManager: Starting task 76.0 in stage 12.0 (TID 87, localhost, partition 77,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:33 INFO scheduler.TaskSetManager: Finished task 75.0 in stage 12.0 (TID 86) in 146 ms on localhost (76/199)
16/03/27 03:21:33 INFO executor.Executor: Running task 76.0 in stage 12.0 (TID 87)
16/03/27 03:21:33 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:33 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:33 INFO executor.Executor: Finished task 76.0 in stage 12.0 (TID 87). 1609 bytes result sent to driver
16/03/27 03:21:33 INFO scheduler.TaskSetManager: Starting task 77.0 in stage 12.0 (TID 88, localhost, partition 78,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:33 INFO scheduler.TaskSetManager: Finished task 76.0 in stage 12.0 (TID 87) in 176 ms on localhost (77/199)
16/03/27 03:21:33 INFO executor.Executor: Running task 77.0 in stage 12.0 (TID 88)
16/03/27 03:21:33 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:33 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:33 INFO executor.Executor: Finished task 77.0 in stage 12.0 (TID 88). 1609 bytes result sent to driver
16/03/27 03:21:33 INFO scheduler.TaskSetManager: Starting task 78.0 in stage 12.0 (TID 89, localhost, partition 79,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:33 INFO scheduler.TaskSetManager: Finished task 77.0 in stage 12.0 (TID 88) in 195 ms on localhost (78/199)
16/03/27 03:21:33 INFO executor.Executor: Running task 78.0 in stage 12.0 (TID 89)
16/03/27 03:21:33 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:33 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:33 INFO executor.Executor: Finished task 78.0 in stage 12.0 (TID 89). 1609 bytes result sent to driver
16/03/27 03:21:33 INFO scheduler.TaskSetManager: Starting task 79.0 in stage 12.0 (TID 90, localhost, partition 80,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:33 INFO scheduler.TaskSetManager: Finished task 78.0 in stage 12.0 (TID 89) in 66 ms on localhost (79/199)
16/03/27 03:21:33 INFO executor.Executor: Running task 79.0 in stage 12.0 (TID 90)
16/03/27 03:21:33 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:33 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:33 INFO executor.Executor: Finished task 79.0 in stage 12.0 (TID 90). 1609 bytes result sent to driver
16/03/27 03:21:33 INFO scheduler.TaskSetManager: Starting task 80.0 in stage 12.0 (TID 91, localhost, partition 81,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:33 INFO scheduler.TaskSetManager: Finished task 79.0 in stage 12.0 (TID 90) in 138 ms on localhost (80/199)
16/03/27 03:21:33 INFO executor.Executor: Running task 80.0 in stage 12.0 (TID 91)
16/03/27 03:21:33 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:33 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
16/03/27 03:21:33 INFO executor.Executor: Finished task 80.0 in stage 12.0 (TID 91). 1609 bytes result sent to driver
16/03/27 03:21:33 INFO scheduler.TaskSetManager: Starting task 81.0 in stage 12.0 (TID 92, localhost, partition 82,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:33 INFO scheduler.TaskSetManager: Finished task 80.0 in stage 12.0 (TID 91) in 179 ms on localhost (81/199)
16/03/27 03:21:33 INFO executor.Executor: Running task 81.0 in stage 12.0 (TID 92)
16/03/27 03:21:34 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:34 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
16/03/27 03:21:34 INFO executor.Executor: Finished task 81.0 in stage 12.0 (TID 92). 1609 bytes result sent to driver
16/03/27 03:21:34 INFO scheduler.TaskSetManager: Starting task 82.0 in stage 12.0 (TID 93, localhost, partition 83,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:34 INFO scheduler.TaskSetManager: Finished task 81.0 in stage 12.0 (TID 92) in 246 ms on localhost (82/199)
16/03/27 03:21:34 INFO executor.Executor: Running task 82.0 in stage 12.0 (TID 93)
16/03/27 03:21:34 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:34 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
16/03/27 03:21:34 INFO executor.Executor: Finished task 82.0 in stage 12.0 (TID 93). 1609 bytes result sent to driver
16/03/27 03:21:34 INFO scheduler.TaskSetManager: Starting task 83.0 in stage 12.0 (TID 94, localhost, partition 84,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:34 INFO scheduler.TaskSetManager: Finished task 82.0 in stage 12.0 (TID 93) in 106 ms on localhost (83/199)
16/03/27 03:21:34 INFO executor.Executor: Running task 83.0 in stage 12.0 (TID 94)
16/03/27 03:21:34 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:34 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:34 INFO executor.Executor: Finished task 83.0 in stage 12.0 (TID 94). 1609 bytes result sent to driver
16/03/27 03:21:34 INFO scheduler.TaskSetManager: Starting task 84.0 in stage 12.0 (TID 95, localhost, partition 85,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:34 INFO scheduler.TaskSetManager: Finished task 83.0 in stage 12.0 (TID 94) in 185 ms on localhost (84/199)
16/03/27 03:21:34 INFO executor.Executor: Running task 84.0 in stage 12.0 (TID 95)
16/03/27 03:21:34 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:34 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:34 INFO executor.Executor: Finished task 84.0 in stage 12.0 (TID 95). 1609 bytes result sent to driver
16/03/27 03:21:34 INFO scheduler.TaskSetManager: Starting task 85.0 in stage 12.0 (TID 96, localhost, partition 86,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:34 INFO scheduler.TaskSetManager: Finished task 84.0 in stage 12.0 (TID 95) in 125 ms on localhost (85/199)
16/03/27 03:21:34 INFO executor.Executor: Running task 85.0 in stage 12.0 (TID 96)
16/03/27 03:21:34 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:34 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:34 INFO executor.Executor: Finished task 85.0 in stage 12.0 (TID 96). 1609 bytes result sent to driver
16/03/27 03:21:34 INFO scheduler.TaskSetManager: Starting task 86.0 in stage 12.0 (TID 97, localhost, partition 87,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:34 INFO scheduler.TaskSetManager: Finished task 85.0 in stage 12.0 (TID 96) in 128 ms on localhost (86/199)
16/03/27 03:21:34 INFO executor.Executor: Running task 86.0 in stage 12.0 (TID 97)
16/03/27 03:21:34 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:34 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:34 INFO executor.Executor: Finished task 86.0 in stage 12.0 (TID 97). 1609 bytes result sent to driver
16/03/27 03:21:34 INFO scheduler.TaskSetManager: Starting task 87.0 in stage 12.0 (TID 98, localhost, partition 88,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:34 INFO scheduler.TaskSetManager: Finished task 86.0 in stage 12.0 (TID 97) in 190 ms on localhost (87/199)
16/03/27 03:21:34 INFO executor.Executor: Running task 87.0 in stage 12.0 (TID 98)
16/03/27 03:21:34 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:34 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
16/03/27 03:21:34 INFO executor.Executor: Finished task 87.0 in stage 12.0 (TID 98). 1609 bytes result sent to driver
16/03/27 03:21:34 INFO scheduler.TaskSetManager: Starting task 88.0 in stage 12.0 (TID 99, localhost, partition 89,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:34 INFO scheduler.TaskSetManager: Finished task 87.0 in stage 12.0 (TID 98) in 103 ms on localhost (88/199)
16/03/27 03:21:34 INFO executor.Executor: Running task 88.0 in stage 12.0 (TID 99)
16/03/27 03:21:35 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:35 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms
16/03/27 03:21:35 INFO executor.Executor: Finished task 88.0 in stage 12.0 (TID 99). 1609 bytes result sent to driver
16/03/27 03:21:35 INFO scheduler.TaskSetManager: Starting task 89.0 in stage 12.0 (TID 100, localhost, partition 90,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:35 INFO scheduler.TaskSetManager: Finished task 88.0 in stage 12.0 (TID 99) in 134 ms on localhost (89/199)
16/03/27 03:21:35 INFO executor.Executor: Running task 89.0 in stage 12.0 (TID 100)
16/03/27 03:21:35 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:35 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:35 INFO executor.Executor: Finished task 89.0 in stage 12.0 (TID 100). 1609 bytes result sent to driver
16/03/27 03:21:35 INFO scheduler.TaskSetManager: Starting task 90.0 in stage 12.0 (TID 101, localhost, partition 91,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:35 INFO scheduler.TaskSetManager: Finished task 89.0 in stage 12.0 (TID 100) in 236 ms on localhost (90/199)
16/03/27 03:21:35 INFO executor.Executor: Running task 90.0 in stage 12.0 (TID 101)
16/03/27 03:21:35 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:35 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:35 INFO executor.Executor: Finished task 90.0 in stage 12.0 (TID 101). 1609 bytes result sent to driver
16/03/27 03:21:35 INFO scheduler.TaskSetManager: Starting task 91.0 in stage 12.0 (TID 102, localhost, partition 92,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:35 INFO scheduler.TaskSetManager: Finished task 90.0 in stage 12.0 (TID 101) in 254 ms on localhost (91/199)
16/03/27 03:21:35 INFO executor.Executor: Running task 91.0 in stage 12.0 (TID 102)
16/03/27 03:21:35 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:35 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:35 INFO executor.Executor: Finished task 91.0 in stage 12.0 (TID 102). 1609 bytes result sent to driver
16/03/27 03:21:35 INFO scheduler.TaskSetManager: Starting task 92.0 in stage 12.0 (TID 103, localhost, partition 93,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:35 INFO scheduler.TaskSetManager: Finished task 91.0 in stage 12.0 (TID 102) in 170 ms on localhost (92/199)
16/03/27 03:21:35 INFO executor.Executor: Running task 92.0 in stage 12.0 (TID 103)
16/03/27 03:21:35 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:35 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:35 INFO executor.Executor: Finished task 92.0 in stage 12.0 (TID 103). 1609 bytes result sent to driver
16/03/27 03:21:35 INFO scheduler.TaskSetManager: Starting task 93.0 in stage 12.0 (TID 104, localhost, partition 94,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:35 INFO scheduler.TaskSetManager: Finished task 92.0 in stage 12.0 (TID 103) in 77 ms on localhost (93/199)
16/03/27 03:21:35 INFO executor.Executor: Running task 93.0 in stage 12.0 (TID 104)
16/03/27 03:21:35 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:35 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:35 INFO executor.Executor: Finished task 93.0 in stage 12.0 (TID 104). 1609 bytes result sent to driver
16/03/27 03:21:35 INFO scheduler.TaskSetManager: Starting task 94.0 in stage 12.0 (TID 105, localhost, partition 95,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:35 INFO scheduler.TaskSetManager: Finished task 93.0 in stage 12.0 (TID 104) in 189 ms on localhost (94/199)
16/03/27 03:21:35 INFO executor.Executor: Running task 94.0 in stage 12.0 (TID 105)
16/03/27 03:21:35 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:35 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
16/03/27 03:21:36 INFO executor.Executor: Finished task 94.0 in stage 12.0 (TID 105). 1609 bytes result sent to driver
16/03/27 03:21:36 INFO scheduler.TaskSetManager: Starting task 95.0 in stage 12.0 (TID 106, localhost, partition 96,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:36 INFO scheduler.TaskSetManager: Finished task 94.0 in stage 12.0 (TID 105) in 70 ms on localhost (95/199)
16/03/27 03:21:36 INFO executor.Executor: Running task 95.0 in stage 12.0 (TID 106)
16/03/27 03:21:36 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:36 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:36 INFO executor.Executor: Finished task 95.0 in stage 12.0 (TID 106). 1609 bytes result sent to driver
16/03/27 03:21:36 INFO scheduler.TaskSetManager: Starting task 96.0 in stage 12.0 (TID 107, localhost, partition 97,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:36 INFO scheduler.TaskSetManager: Finished task 95.0 in stage 12.0 (TID 106) in 95 ms on localhost (96/199)
16/03/27 03:21:36 INFO executor.Executor: Running task 96.0 in stage 12.0 (TID 107)
16/03/27 03:21:36 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:36 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 44 ms
16/03/27 03:21:36 INFO executor.Executor: Finished task 96.0 in stage 12.0 (TID 107). 1609 bytes result sent to driver
16/03/27 03:21:36 INFO scheduler.TaskSetManager: Starting task 97.0 in stage 12.0 (TID 108, localhost, partition 98,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:36 INFO scheduler.TaskSetManager: Finished task 96.0 in stage 12.0 (TID 107) in 109 ms on localhost (97/199)
16/03/27 03:21:36 INFO executor.Executor: Running task 97.0 in stage 12.0 (TID 108)
16/03/27 03:21:36 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:36 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 30 ms
16/03/27 03:21:36 INFO executor.Executor: Finished task 97.0 in stage 12.0 (TID 108). 1609 bytes result sent to driver
16/03/27 03:21:36 INFO scheduler.TaskSetManager: Starting task 98.0 in stage 12.0 (TID 109, localhost, partition 99,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:36 INFO scheduler.TaskSetManager: Finished task 97.0 in stage 12.0 (TID 108) in 153 ms on localhost (98/199)
16/03/27 03:21:36 INFO executor.Executor: Running task 98.0 in stage 12.0 (TID 109)
16/03/27 03:21:36 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:36 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:36 INFO executor.Executor: Finished task 98.0 in stage 12.0 (TID 109). 1609 bytes result sent to driver
16/03/27 03:21:36 INFO scheduler.TaskSetManager: Starting task 99.0 in stage 12.0 (TID 110, localhost, partition 100,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:36 INFO scheduler.TaskSetManager: Finished task 98.0 in stage 12.0 (TID 109) in 161 ms on localhost (99/199)
16/03/27 03:21:36 INFO executor.Executor: Running task 99.0 in stage 12.0 (TID 110)
16/03/27 03:21:36 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:36 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:36 INFO executor.Executor: Finished task 99.0 in stage 12.0 (TID 110). 1609 bytes result sent to driver
16/03/27 03:21:36 INFO scheduler.TaskSetManager: Starting task 100.0 in stage 12.0 (TID 111, localhost, partition 101,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:36 INFO scheduler.TaskSetManager: Finished task 99.0 in stage 12.0 (TID 110) in 103 ms on localhost (100/199)
16/03/27 03:21:36 INFO executor.Executor: Running task 100.0 in stage 12.0 (TID 111)
16/03/27 03:21:36 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:36 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
16/03/27 03:21:36 INFO executor.Executor: Finished task 100.0 in stage 12.0 (TID 111). 1609 bytes result sent to driver
16/03/27 03:21:36 INFO scheduler.TaskSetManager: Starting task 101.0 in stage 12.0 (TID 112, localhost, partition 102,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:36 INFO scheduler.TaskSetManager: Finished task 100.0 in stage 12.0 (TID 111) in 131 ms on localhost (101/199)
16/03/27 03:21:36 INFO executor.Executor: Running task 101.0 in stage 12.0 (TID 112)
16/03/27 03:21:36 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:36 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
16/03/27 03:21:36 INFO executor.Executor: Finished task 101.0 in stage 12.0 (TID 112). 1609 bytes result sent to driver
16/03/27 03:21:36 INFO scheduler.TaskSetManager: Starting task 102.0 in stage 12.0 (TID 113, localhost, partition 103,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:36 INFO scheduler.TaskSetManager: Finished task 101.0 in stage 12.0 (TID 112) in 94 ms on localhost (102/199)
16/03/27 03:21:36 INFO executor.Executor: Running task 102.0 in stage 12.0 (TID 113)
16/03/27 03:21:36 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:36 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:36 INFO executor.Executor: Finished task 102.0 in stage 12.0 (TID 113). 1609 bytes result sent to driver
16/03/27 03:21:36 INFO scheduler.TaskSetManager: Starting task 103.0 in stage 12.0 (TID 114, localhost, partition 104,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:36 INFO scheduler.TaskSetManager: Finished task 102.0 in stage 12.0 (TID 113) in 171 ms on localhost (103/199)
16/03/27 03:21:36 INFO executor.Executor: Running task 103.0 in stage 12.0 (TID 114)
16/03/27 03:21:37 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:37 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
16/03/27 03:21:37 INFO executor.Executor: Finished task 103.0 in stage 12.0 (TID 114). 1609 bytes result sent to driver
16/03/27 03:21:37 INFO scheduler.TaskSetManager: Starting task 104.0 in stage 12.0 (TID 115, localhost, partition 105,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:37 INFO scheduler.TaskSetManager: Finished task 103.0 in stage 12.0 (TID 114) in 93 ms on localhost (104/199)
16/03/27 03:21:37 INFO executor.Executor: Running task 104.0 in stage 12.0 (TID 115)
16/03/27 03:21:37 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:37 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:37 INFO executor.Executor: Finished task 104.0 in stage 12.0 (TID 115). 1609 bytes result sent to driver
16/03/27 03:21:37 INFO scheduler.TaskSetManager: Starting task 105.0 in stage 12.0 (TID 116, localhost, partition 106,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:37 INFO scheduler.TaskSetManager: Finished task 104.0 in stage 12.0 (TID 115) in 105 ms on localhost (105/199)
16/03/27 03:21:37 INFO executor.Executor: Running task 105.0 in stage 12.0 (TID 116)
16/03/27 03:21:37 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:37 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:37 INFO executor.Executor: Finished task 105.0 in stage 12.0 (TID 116). 1609 bytes result sent to driver
16/03/27 03:21:37 INFO scheduler.TaskSetManager: Starting task 106.0 in stage 12.0 (TID 117, localhost, partition 107,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:37 INFO scheduler.TaskSetManager: Finished task 105.0 in stage 12.0 (TID 116) in 73 ms on localhost (106/199)
16/03/27 03:21:37 INFO executor.Executor: Running task 106.0 in stage 12.0 (TID 117)
16/03/27 03:21:37 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:37 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
16/03/27 03:21:37 INFO executor.Executor: Finished task 106.0 in stage 12.0 (TID 117). 1609 bytes result sent to driver
16/03/27 03:21:37 INFO scheduler.TaskSetManager: Starting task 107.0 in stage 12.0 (TID 118, localhost, partition 108,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:37 INFO scheduler.TaskSetManager: Finished task 106.0 in stage 12.0 (TID 117) in 77 ms on localhost (107/199)
16/03/27 03:21:37 INFO executor.Executor: Running task 107.0 in stage 12.0 (TID 118)
16/03/27 03:21:37 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:37 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:37 INFO executor.Executor: Finished task 107.0 in stage 12.0 (TID 118). 1609 bytes result sent to driver
16/03/27 03:21:37 INFO scheduler.TaskSetManager: Starting task 108.0 in stage 12.0 (TID 119, localhost, partition 109,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:37 INFO scheduler.TaskSetManager: Finished task 107.0 in stage 12.0 (TID 118) in 94 ms on localhost (108/199)
16/03/27 03:21:37 INFO executor.Executor: Running task 108.0 in stage 12.0 (TID 119)
16/03/27 03:21:37 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:37 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 65 ms
16/03/27 03:21:37 INFO executor.Executor: Finished task 108.0 in stage 12.0 (TID 119). 1609 bytes result sent to driver
16/03/27 03:21:37 INFO scheduler.TaskSetManager: Starting task 109.0 in stage 12.0 (TID 120, localhost, partition 110,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:37 INFO scheduler.TaskSetManager: Finished task 108.0 in stage 12.0 (TID 119) in 154 ms on localhost (109/199)
16/03/27 03:21:37 INFO executor.Executor: Running task 109.0 in stage 12.0 (TID 120)
16/03/27 03:21:37 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:37 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:37 INFO executor.Executor: Finished task 109.0 in stage 12.0 (TID 120). 1609 bytes result sent to driver
16/03/27 03:21:37 INFO scheduler.TaskSetManager: Starting task 110.0 in stage 12.0 (TID 121, localhost, partition 111,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:37 INFO scheduler.TaskSetManager: Finished task 109.0 in stage 12.0 (TID 120) in 151 ms on localhost (110/199)
16/03/27 03:21:37 INFO executor.Executor: Running task 110.0 in stage 12.0 (TID 121)
16/03/27 03:21:37 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:37 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:37 INFO executor.Executor: Finished task 110.0 in stage 12.0 (TID 121). 1609 bytes result sent to driver
16/03/27 03:21:37 INFO scheduler.TaskSetManager: Starting task 111.0 in stage 12.0 (TID 122, localhost, partition 112,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:37 INFO scheduler.TaskSetManager: Finished task 110.0 in stage 12.0 (TID 121) in 155 ms on localhost (111/199)
16/03/27 03:21:37 INFO executor.Executor: Running task 111.0 in stage 12.0 (TID 122)
16/03/27 03:21:37 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:37 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:37 INFO executor.Executor: Finished task 111.0 in stage 12.0 (TID 122). 1609 bytes result sent to driver
16/03/27 03:21:37 INFO scheduler.TaskSetManager: Starting task 112.0 in stage 12.0 (TID 123, localhost, partition 113,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:37 INFO scheduler.TaskSetManager: Finished task 111.0 in stage 12.0 (TID 122) in 158 ms on localhost (112/199)
16/03/27 03:21:37 INFO executor.Executor: Running task 112.0 in stage 12.0 (TID 123)
16/03/27 03:21:38 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:38 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
16/03/27 03:21:38 INFO executor.Executor: Finished task 112.0 in stage 12.0 (TID 123). 1609 bytes result sent to driver
16/03/27 03:21:38 INFO scheduler.TaskSetManager: Starting task 113.0 in stage 12.0 (TID 124, localhost, partition 114,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:38 INFO scheduler.TaskSetManager: Finished task 112.0 in stage 12.0 (TID 123) in 153 ms on localhost (113/199)
16/03/27 03:21:38 INFO executor.Executor: Running task 113.0 in stage 12.0 (TID 124)
16/03/27 03:21:38 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:38 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:38 INFO executor.Executor: Finished task 113.0 in stage 12.0 (TID 124). 1609 bytes result sent to driver
16/03/27 03:21:38 INFO scheduler.TaskSetManager: Starting task 114.0 in stage 12.0 (TID 125, localhost, partition 115,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:38 INFO scheduler.TaskSetManager: Finished task 113.0 in stage 12.0 (TID 124) in 93 ms on localhost (114/199)
16/03/27 03:21:38 INFO executor.Executor: Running task 114.0 in stage 12.0 (TID 125)
16/03/27 03:21:38 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:38 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms
16/03/27 03:21:38 INFO executor.Executor: Finished task 114.0 in stage 12.0 (TID 125). 1609 bytes result sent to driver
16/03/27 03:21:38 INFO scheduler.TaskSetManager: Starting task 115.0 in stage 12.0 (TID 126, localhost, partition 116,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:38 INFO scheduler.TaskSetManager: Finished task 114.0 in stage 12.0 (TID 125) in 157 ms on localhost (115/199)
16/03/27 03:21:38 INFO executor.Executor: Running task 115.0 in stage 12.0 (TID 126)
16/03/27 03:21:38 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:38 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:38 INFO executor.Executor: Finished task 115.0 in stage 12.0 (TID 126). 1609 bytes result sent to driver
16/03/27 03:21:38 INFO scheduler.TaskSetManager: Starting task 116.0 in stage 12.0 (TID 127, localhost, partition 117,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:38 INFO scheduler.TaskSetManager: Finished task 115.0 in stage 12.0 (TID 126) in 148 ms on localhost (116/199)
16/03/27 03:21:38 INFO executor.Executor: Running task 116.0 in stage 12.0 (TID 127)
16/03/27 03:21:38 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:38 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:38 INFO executor.Executor: Finished task 116.0 in stage 12.0 (TID 127). 1609 bytes result sent to driver
16/03/27 03:21:38 INFO scheduler.TaskSetManager: Starting task 117.0 in stage 12.0 (TID 128, localhost, partition 118,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:38 INFO scheduler.TaskSetManager: Finished task 116.0 in stage 12.0 (TID 127) in 150 ms on localhost (117/199)
16/03/27 03:21:38 INFO executor.Executor: Running task 117.0 in stage 12.0 (TID 128)
16/03/27 03:21:38 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:38 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:38 INFO executor.Executor: Finished task 117.0 in stage 12.0 (TID 128). 1609 bytes result sent to driver
16/03/27 03:21:38 INFO scheduler.TaskSetManager: Starting task 118.0 in stage 12.0 (TID 129, localhost, partition 119,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:38 INFO scheduler.TaskSetManager: Finished task 117.0 in stage 12.0 (TID 128) in 133 ms on localhost (118/199)
16/03/27 03:21:38 INFO executor.Executor: Running task 118.0 in stage 12.0 (TID 129)
16/03/27 03:21:38 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:38 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
16/03/27 03:21:38 INFO executor.Executor: Finished task 118.0 in stage 12.0 (TID 129). 1609 bytes result sent to driver
16/03/27 03:21:38 INFO scheduler.TaskSetManager: Starting task 119.0 in stage 12.0 (TID 130, localhost, partition 120,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:38 INFO scheduler.TaskSetManager: Finished task 118.0 in stage 12.0 (TID 129) in 161 ms on localhost (119/199)
16/03/27 03:21:38 INFO executor.Executor: Running task 119.0 in stage 12.0 (TID 130)
16/03/27 03:21:38 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:38 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
16/03/27 03:21:39 INFO executor.Executor: Finished task 119.0 in stage 12.0 (TID 130). 1609 bytes result sent to driver
16/03/27 03:21:39 INFO scheduler.TaskSetManager: Starting task 120.0 in stage 12.0 (TID 131, localhost, partition 121,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:39 INFO scheduler.TaskSetManager: Finished task 119.0 in stage 12.0 (TID 130) in 98 ms on localhost (120/199)
16/03/27 03:21:39 INFO executor.Executor: Running task 120.0 in stage 12.0 (TID 131)
16/03/27 03:21:39 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:39 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
16/03/27 03:21:39 INFO executor.Executor: Finished task 120.0 in stage 12.0 (TID 131). 1609 bytes result sent to driver
16/03/27 03:21:39 INFO scheduler.TaskSetManager: Starting task 121.0 in stage 12.0 (TID 132, localhost, partition 122,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:39 INFO scheduler.TaskSetManager: Finished task 120.0 in stage 12.0 (TID 131) in 114 ms on localhost (121/199)
16/03/27 03:21:39 INFO executor.Executor: Running task 121.0 in stage 12.0 (TID 132)
16/03/27 03:21:39 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:39 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:39 INFO executor.Executor: Finished task 121.0 in stage 12.0 (TID 132). 1609 bytes result sent to driver
16/03/27 03:21:39 INFO scheduler.TaskSetManager: Starting task 122.0 in stage 12.0 (TID 133, localhost, partition 123,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:39 INFO scheduler.TaskSetManager: Finished task 121.0 in stage 12.0 (TID 132) in 114 ms on localhost (122/199)
16/03/27 03:21:39 INFO executor.Executor: Running task 122.0 in stage 12.0 (TID 133)
16/03/27 03:21:39 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:39 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:39 INFO executor.Executor: Finished task 122.0 in stage 12.0 (TID 133). 1609 bytes result sent to driver
16/03/27 03:21:39 INFO scheduler.TaskSetManager: Starting task 123.0 in stage 12.0 (TID 134, localhost, partition 124,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:39 INFO scheduler.TaskSetManager: Finished task 122.0 in stage 12.0 (TID 133) in 127 ms on localhost (123/199)
16/03/27 03:21:39 INFO executor.Executor: Running task 123.0 in stage 12.0 (TID 134)
16/03/27 03:21:39 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:39 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
16/03/27 03:21:39 INFO executor.Executor: Finished task 123.0 in stage 12.0 (TID 134). 1609 bytes result sent to driver
16/03/27 03:21:39 INFO scheduler.TaskSetManager: Starting task 124.0 in stage 12.0 (TID 135, localhost, partition 125,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:39 INFO scheduler.TaskSetManager: Finished task 123.0 in stage 12.0 (TID 134) in 115 ms on localhost (124/199)
16/03/27 03:21:39 INFO executor.Executor: Running task 124.0 in stage 12.0 (TID 135)
16/03/27 03:21:39 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:39 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
16/03/27 03:21:39 INFO executor.Executor: Finished task 124.0 in stage 12.0 (TID 135). 1609 bytes result sent to driver
16/03/27 03:21:39 INFO scheduler.TaskSetManager: Starting task 125.0 in stage 12.0 (TID 136, localhost, partition 126,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:39 INFO scheduler.TaskSetManager: Finished task 124.0 in stage 12.0 (TID 135) in 274 ms on localhost (125/199)
16/03/27 03:21:39 INFO executor.Executor: Running task 125.0 in stage 12.0 (TID 136)
16/03/27 03:21:39 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:39 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:39 INFO executor.Executor: Finished task 125.0 in stage 12.0 (TID 136). 1609 bytes result sent to driver
16/03/27 03:21:39 INFO scheduler.TaskSetManager: Starting task 126.0 in stage 12.0 (TID 137, localhost, partition 127,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:39 INFO scheduler.TaskSetManager: Finished task 125.0 in stage 12.0 (TID 136) in 88 ms on localhost (126/199)
16/03/27 03:21:39 INFO executor.Executor: Running task 126.0 in stage 12.0 (TID 137)
16/03/27 03:21:39 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:39 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
16/03/27 03:21:39 INFO executor.Executor: Finished task 126.0 in stage 12.0 (TID 137). 1609 bytes result sent to driver
16/03/27 03:21:39 INFO scheduler.TaskSetManager: Starting task 127.0 in stage 12.0 (TID 138, localhost, partition 128,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:39 INFO scheduler.TaskSetManager: Finished task 126.0 in stage 12.0 (TID 137) in 171 ms on localhost (127/199)
16/03/27 03:21:40 INFO executor.Executor: Running task 127.0 in stage 12.0 (TID 138)
16/03/27 03:21:40 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:40 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:40 INFO executor.Executor: Finished task 127.0 in stage 12.0 (TID 138). 1609 bytes result sent to driver
16/03/27 03:21:40 INFO scheduler.TaskSetManager: Starting task 128.0 in stage 12.0 (TID 139, localhost, partition 129,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:40 INFO scheduler.TaskSetManager: Finished task 127.0 in stage 12.0 (TID 138) in 152 ms on localhost (128/199)
16/03/27 03:21:40 INFO executor.Executor: Running task 128.0 in stage 12.0 (TID 139)
16/03/27 03:21:40 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:40 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 60 ms
16/03/27 03:21:40 INFO executor.Executor: Finished task 128.0 in stage 12.0 (TID 139). 1609 bytes result sent to driver
16/03/27 03:21:40 INFO scheduler.TaskSetManager: Starting task 129.0 in stage 12.0 (TID 140, localhost, partition 130,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:40 INFO scheduler.TaskSetManager: Finished task 128.0 in stage 12.0 (TID 139) in 108 ms on localhost (129/199)
16/03/27 03:21:40 INFO executor.Executor: Running task 129.0 in stage 12.0 (TID 140)
16/03/27 03:21:40 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:40 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 57 ms
16/03/27 03:21:40 INFO executor.Executor: Finished task 129.0 in stage 12.0 (TID 140). 1609 bytes result sent to driver
16/03/27 03:21:40 INFO scheduler.TaskSetManager: Starting task 130.0 in stage 12.0 (TID 141, localhost, partition 131,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:40 INFO scheduler.TaskSetManager: Finished task 129.0 in stage 12.0 (TID 140) in 104 ms on localhost (130/199)
16/03/27 03:21:40 INFO executor.Executor: Running task 130.0 in stage 12.0 (TID 141)
16/03/27 03:21:40 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:40 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:40 INFO executor.Executor: Finished task 130.0 in stage 12.0 (TID 141). 1609 bytes result sent to driver
16/03/27 03:21:40 INFO scheduler.TaskSetManager: Starting task 131.0 in stage 12.0 (TID 142, localhost, partition 132,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:40 INFO scheduler.TaskSetManager: Finished task 130.0 in stage 12.0 (TID 141) in 123 ms on localhost (131/199)
16/03/27 03:21:40 INFO executor.Executor: Running task 131.0 in stage 12.0 (TID 142)
16/03/27 03:21:40 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:40 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:40 INFO executor.Executor: Finished task 131.0 in stage 12.0 (TID 142). 1609 bytes result sent to driver
16/03/27 03:21:40 INFO scheduler.TaskSetManager: Starting task 132.0 in stage 12.0 (TID 143, localhost, partition 133,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:40 INFO scheduler.TaskSetManager: Finished task 131.0 in stage 12.0 (TID 142) in 105 ms on localhost (132/199)
16/03/27 03:21:40 INFO executor.Executor: Running task 132.0 in stage 12.0 (TID 143)
16/03/27 03:21:40 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:40 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
16/03/27 03:21:40 INFO executor.Executor: Finished task 132.0 in stage 12.0 (TID 143). 1609 bytes result sent to driver
16/03/27 03:21:40 INFO scheduler.TaskSetManager: Starting task 133.0 in stage 12.0 (TID 144, localhost, partition 134,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:40 INFO scheduler.TaskSetManager: Finished task 132.0 in stage 12.0 (TID 143) in 140 ms on localhost (133/199)
16/03/27 03:21:40 INFO executor.Executor: Running task 133.0 in stage 12.0 (TID 144)
16/03/27 03:21:40 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:40 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
16/03/27 03:21:40 INFO executor.Executor: Finished task 133.0 in stage 12.0 (TID 144). 1609 bytes result sent to driver
16/03/27 03:21:40 INFO scheduler.TaskSetManager: Starting task 134.0 in stage 12.0 (TID 145, localhost, partition 135,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:40 INFO scheduler.TaskSetManager: Finished task 133.0 in stage 12.0 (TID 144) in 215 ms on localhost (134/199)
16/03/27 03:21:40 INFO executor.Executor: Running task 134.0 in stage 12.0 (TID 145)
16/03/27 03:21:41 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:41 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:41 INFO executor.Executor: Finished task 134.0 in stage 12.0 (TID 145). 1609 bytes result sent to driver
16/03/27 03:21:41 INFO scheduler.TaskSetManager: Starting task 135.0 in stage 12.0 (TID 146, localhost, partition 136,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:41 INFO scheduler.TaskSetManager: Finished task 134.0 in stage 12.0 (TID 145) in 159 ms on localhost (135/199)
16/03/27 03:21:41 INFO executor.Executor: Running task 135.0 in stage 12.0 (TID 146)
16/03/27 03:21:41 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:41 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:41 INFO executor.Executor: Finished task 135.0 in stage 12.0 (TID 146). 1609 bytes result sent to driver
16/03/27 03:21:41 INFO scheduler.TaskSetManager: Starting task 136.0 in stage 12.0 (TID 147, localhost, partition 137,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:41 INFO scheduler.TaskSetManager: Finished task 135.0 in stage 12.0 (TID 146) in 198 ms on localhost (136/199)
16/03/27 03:21:41 INFO executor.Executor: Running task 136.0 in stage 12.0 (TID 147)
16/03/27 03:21:41 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:41 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:41 INFO executor.Executor: Finished task 136.0 in stage 12.0 (TID 147). 1609 bytes result sent to driver
16/03/27 03:21:41 INFO scheduler.TaskSetManager: Starting task 137.0 in stage 12.0 (TID 148, localhost, partition 138,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:41 INFO scheduler.TaskSetManager: Finished task 136.0 in stage 12.0 (TID 147) in 93 ms on localhost (137/199)
16/03/27 03:21:41 INFO executor.Executor: Running task 137.0 in stage 12.0 (TID 148)
16/03/27 03:21:41 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:41 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:41 INFO executor.Executor: Finished task 137.0 in stage 12.0 (TID 148). 1609 bytes result sent to driver
16/03/27 03:21:41 INFO scheduler.TaskSetManager: Starting task 138.0 in stage 12.0 (TID 149, localhost, partition 139,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:41 INFO scheduler.TaskSetManager: Finished task 137.0 in stage 12.0 (TID 148) in 183 ms on localhost (138/199)
16/03/27 03:21:41 INFO executor.Executor: Running task 138.0 in stage 12.0 (TID 149)
16/03/27 03:21:41 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:41 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:41 INFO executor.Executor: Finished task 138.0 in stage 12.0 (TID 149). 1609 bytes result sent to driver
16/03/27 03:21:41 INFO scheduler.TaskSetManager: Starting task 139.0 in stage 12.0 (TID 150, localhost, partition 140,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:41 INFO scheduler.TaskSetManager: Finished task 138.0 in stage 12.0 (TID 149) in 84 ms on localhost (139/199)
16/03/27 03:21:41 INFO executor.Executor: Running task 139.0 in stage 12.0 (TID 150)
16/03/27 03:21:41 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:41 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:41 INFO executor.Executor: Finished task 139.0 in stage 12.0 (TID 150). 1609 bytes result sent to driver
16/03/27 03:21:41 INFO scheduler.TaskSetManager: Starting task 140.0 in stage 12.0 (TID 151, localhost, partition 141,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:41 INFO scheduler.TaskSetManager: Finished task 139.0 in stage 12.0 (TID 150) in 81 ms on localhost (140/199)
16/03/27 03:21:41 INFO executor.Executor: Running task 140.0 in stage 12.0 (TID 151)
16/03/27 03:21:41 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:41 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 24 ms
16/03/27 03:21:41 INFO executor.Executor: Finished task 140.0 in stage 12.0 (TID 151). 1609 bytes result sent to driver
16/03/27 03:21:41 INFO scheduler.TaskSetManager: Starting task 141.0 in stage 12.0 (TID 152, localhost, partition 142,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:41 INFO executor.Executor: Running task 141.0 in stage 12.0 (TID 152)
16/03/27 03:21:41 INFO scheduler.TaskSetManager: Finished task 140.0 in stage 12.0 (TID 151) in 262 ms on localhost (141/199)
16/03/27 03:21:41 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:41 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:41 INFO executor.Executor: Finished task 141.0 in stage 12.0 (TID 152). 1609 bytes result sent to driver
16/03/27 03:21:41 INFO scheduler.TaskSetManager: Starting task 142.0 in stage 12.0 (TID 153, localhost, partition 143,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:42 INFO scheduler.TaskSetManager: Finished task 141.0 in stage 12.0 (TID 152) in 130 ms on localhost (142/199)
16/03/27 03:21:42 INFO executor.Executor: Running task 142.0 in stage 12.0 (TID 153)
16/03/27 03:21:42 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:42 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms
16/03/27 03:21:42 INFO executor.Executor: Finished task 142.0 in stage 12.0 (TID 153). 1609 bytes result sent to driver
16/03/27 03:21:42 INFO scheduler.TaskSetManager: Starting task 143.0 in stage 12.0 (TID 154, localhost, partition 144,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:42 INFO scheduler.TaskSetManager: Finished task 142.0 in stage 12.0 (TID 153) in 105 ms on localhost (143/199)
16/03/27 03:21:42 INFO executor.Executor: Running task 143.0 in stage 12.0 (TID 154)
16/03/27 03:21:42 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:42 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:42 INFO executor.Executor: Finished task 143.0 in stage 12.0 (TID 154). 1609 bytes result sent to driver
16/03/27 03:21:42 INFO scheduler.TaskSetManager: Starting task 144.0 in stage 12.0 (TID 155, localhost, partition 145,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:42 INFO scheduler.TaskSetManager: Finished task 143.0 in stage 12.0 (TID 154) in 151 ms on localhost (144/199)
16/03/27 03:21:42 INFO executor.Executor: Running task 144.0 in stage 12.0 (TID 155)
16/03/27 03:21:42 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:42 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 3 ms
16/03/27 03:21:42 INFO executor.Executor: Finished task 144.0 in stage 12.0 (TID 155). 1609 bytes result sent to driver
16/03/27 03:21:42 INFO scheduler.TaskSetManager: Starting task 145.0 in stage 12.0 (TID 156, localhost, partition 146,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:42 INFO scheduler.TaskSetManager: Finished task 144.0 in stage 12.0 (TID 155) in 101 ms on localhost (145/199)
16/03/27 03:21:42 INFO executor.Executor: Running task 145.0 in stage 12.0 (TID 156)
16/03/27 03:21:42 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:42 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:42 INFO executor.Executor: Finished task 145.0 in stage 12.0 (TID 156). 1609 bytes result sent to driver
16/03/27 03:21:42 INFO scheduler.TaskSetManager: Starting task 146.0 in stage 12.0 (TID 157, localhost, partition 147,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:42 INFO scheduler.TaskSetManager: Finished task 145.0 in stage 12.0 (TID 156) in 132 ms on localhost (146/199)
16/03/27 03:21:42 INFO executor.Executor: Running task 146.0 in stage 12.0 (TID 157)
16/03/27 03:21:42 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:42 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
16/03/27 03:21:42 INFO executor.Executor: Finished task 146.0 in stage 12.0 (TID 157). 1609 bytes result sent to driver
16/03/27 03:21:42 INFO scheduler.TaskSetManager: Starting task 147.0 in stage 12.0 (TID 158, localhost, partition 148,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:42 INFO scheduler.TaskSetManager: Finished task 146.0 in stage 12.0 (TID 157) in 140 ms on localhost (147/199)
16/03/27 03:21:42 INFO executor.Executor: Running task 147.0 in stage 12.0 (TID 158)
16/03/27 03:21:42 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:42 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:42 INFO executor.Executor: Finished task 147.0 in stage 12.0 (TID 158). 1609 bytes result sent to driver
16/03/27 03:21:42 INFO scheduler.TaskSetManager: Starting task 148.0 in stage 12.0 (TID 159, localhost, partition 149,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:42 INFO scheduler.TaskSetManager: Finished task 147.0 in stage 12.0 (TID 158) in 85 ms on localhost (148/199)
16/03/27 03:21:42 INFO executor.Executor: Running task 148.0 in stage 12.0 (TID 159)
16/03/27 03:21:42 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:42 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 49 ms
16/03/27 03:21:42 INFO executor.Executor: Finished task 148.0 in stage 12.0 (TID 159). 1609 bytes result sent to driver
16/03/27 03:21:42 INFO scheduler.TaskSetManager: Starting task 149.0 in stage 12.0 (TID 160, localhost, partition 150,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:42 INFO scheduler.TaskSetManager: Finished task 148.0 in stage 12.0 (TID 159) in 86 ms on localhost (149/199)
16/03/27 03:21:42 INFO executor.Executor: Running task 149.0 in stage 12.0 (TID 160)
16/03/27 03:21:42 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:42 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:42 INFO executor.Executor: Finished task 149.0 in stage 12.0 (TID 160). 1609 bytes result sent to driver
16/03/27 03:21:42 INFO scheduler.TaskSetManager: Starting task 150.0 in stage 12.0 (TID 161, localhost, partition 151,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:42 INFO scheduler.TaskSetManager: Finished task 149.0 in stage 12.0 (TID 160) in 94 ms on localhost (150/199)
16/03/27 03:21:42 INFO executor.Executor: Running task 150.0 in stage 12.0 (TID 161)
16/03/27 03:21:42 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:42 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:42 INFO executor.Executor: Finished task 150.0 in stage 12.0 (TID 161). 1609 bytes result sent to driver
16/03/27 03:21:42 INFO scheduler.TaskSetManager: Starting task 151.0 in stage 12.0 (TID 162, localhost, partition 152,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:42 INFO scheduler.TaskSetManager: Finished task 150.0 in stage 12.0 (TID 161) in 37 ms on localhost (151/199)
16/03/27 03:21:42 INFO executor.Executor: Running task 151.0 in stage 12.0 (TID 162)
16/03/27 03:21:42 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:42 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
16/03/27 03:21:43 INFO executor.Executor: Finished task 151.0 in stage 12.0 (TID 162). 1609 bytes result sent to driver
16/03/27 03:21:43 INFO scheduler.TaskSetManager: Starting task 152.0 in stage 12.0 (TID 163, localhost, partition 153,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:43 INFO scheduler.TaskSetManager: Finished task 151.0 in stage 12.0 (TID 162) in 123 ms on localhost (152/199)
16/03/27 03:21:43 INFO executor.Executor: Running task 152.0 in stage 12.0 (TID 163)
16/03/27 03:21:43 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:43 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 26 ms
16/03/27 03:21:43 INFO executor.Executor: Finished task 152.0 in stage 12.0 (TID 163). 1609 bytes result sent to driver
16/03/27 03:21:43 INFO scheduler.TaskSetManager: Starting task 153.0 in stage 12.0 (TID 164, localhost, partition 154,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:43 INFO scheduler.TaskSetManager: Finished task 152.0 in stage 12.0 (TID 163) in 52 ms on localhost (153/199)
16/03/27 03:21:43 INFO executor.Executor: Running task 153.0 in stage 12.0 (TID 164)
16/03/27 03:21:43 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:43 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:43 INFO executor.Executor: Finished task 153.0 in stage 12.0 (TID 164). 1609 bytes result sent to driver
16/03/27 03:21:43 INFO scheduler.TaskSetManager: Starting task 154.0 in stage 12.0 (TID 165, localhost, partition 155,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:43 INFO scheduler.TaskSetManager: Finished task 153.0 in stage 12.0 (TID 164) in 114 ms on localhost (154/199)
16/03/27 03:21:43 INFO executor.Executor: Running task 154.0 in stage 12.0 (TID 165)
16/03/27 03:21:43 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:43 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
16/03/27 03:21:43 INFO executor.Executor: Finished task 154.0 in stage 12.0 (TID 165). 1609 bytes result sent to driver
16/03/27 03:21:43 INFO scheduler.TaskSetManager: Starting task 155.0 in stage 12.0 (TID 166, localhost, partition 156,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:43 INFO scheduler.TaskSetManager: Finished task 154.0 in stage 12.0 (TID 165) in 42 ms on localhost (155/199)
16/03/27 03:21:43 INFO executor.Executor: Running task 155.0 in stage 12.0 (TID 166)
16/03/27 03:21:43 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:43 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:43 INFO executor.Executor: Finished task 155.0 in stage 12.0 (TID 166). 1609 bytes result sent to driver
16/03/27 03:21:43 INFO scheduler.TaskSetManager: Starting task 156.0 in stage 12.0 (TID 167, localhost, partition 157,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:43 INFO scheduler.TaskSetManager: Finished task 155.0 in stage 12.0 (TID 166) in 103 ms on localhost (156/199)
16/03/27 03:21:43 INFO executor.Executor: Running task 156.0 in stage 12.0 (TID 167)
16/03/27 03:21:43 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:43 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 44 ms
16/03/27 03:21:43 INFO executor.Executor: Finished task 156.0 in stage 12.0 (TID 167). 1609 bytes result sent to driver
16/03/27 03:21:43 INFO scheduler.TaskSetManager: Starting task 157.0 in stage 12.0 (TID 168, localhost, partition 158,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:43 INFO scheduler.TaskSetManager: Finished task 156.0 in stage 12.0 (TID 167) in 90 ms on localhost (157/199)
16/03/27 03:21:43 INFO executor.Executor: Running task 157.0 in stage 12.0 (TID 168)
16/03/27 03:21:43 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:43 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 59 ms
16/03/27 03:21:43 INFO executor.Executor: Finished task 157.0 in stage 12.0 (TID 168). 1609 bytes result sent to driver
16/03/27 03:21:43 INFO scheduler.TaskSetManager: Starting task 158.0 in stage 12.0 (TID 169, localhost, partition 159,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:43 INFO scheduler.TaskSetManager: Finished task 157.0 in stage 12.0 (TID 168) in 85 ms on localhost (158/199)
16/03/27 03:21:43 INFO executor.Executor: Running task 158.0 in stage 12.0 (TID 169)
16/03/27 03:21:43 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:43 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
16/03/27 03:21:43 INFO executor.Executor: Finished task 158.0 in stage 12.0 (TID 169). 1609 bytes result sent to driver
16/03/27 03:21:43 INFO scheduler.TaskSetManager: Starting task 159.0 in stage 12.0 (TID 170, localhost, partition 160,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:43 INFO scheduler.TaskSetManager: Finished task 158.0 in stage 12.0 (TID 169) in 133 ms on localhost (159/199)
16/03/27 03:21:43 INFO executor.Executor: Running task 159.0 in stage 12.0 (TID 170)
16/03/27 03:21:43 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:43 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
16/03/27 03:21:43 INFO executor.Executor: Finished task 159.0 in stage 12.0 (TID 170). 1609 bytes result sent to driver
16/03/27 03:21:43 INFO scheduler.TaskSetManager: Starting task 160.0 in stage 12.0 (TID 171, localhost, partition 161,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:43 INFO scheduler.TaskSetManager: Finished task 159.0 in stage 12.0 (TID 170) in 148 ms on localhost (160/199)
16/03/27 03:21:43 INFO executor.Executor: Running task 160.0 in stage 12.0 (TID 171)
16/03/27 03:21:43 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:43 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:43 INFO executor.Executor: Finished task 160.0 in stage 12.0 (TID 171). 1609 bytes result sent to driver
16/03/27 03:21:43 INFO scheduler.TaskSetManager: Starting task 161.0 in stage 12.0 (TID 172, localhost, partition 162,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:43 INFO scheduler.TaskSetManager: Finished task 160.0 in stage 12.0 (TID 171) in 137 ms on localhost (161/199)
16/03/27 03:21:43 INFO executor.Executor: Running task 161.0 in stage 12.0 (TID 172)
16/03/27 03:21:43 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:43 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:43 INFO executor.Executor: Finished task 161.0 in stage 12.0 (TID 172). 1609 bytes result sent to driver
16/03/27 03:21:43 INFO scheduler.TaskSetManager: Starting task 162.0 in stage 12.0 (TID 173, localhost, partition 163,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:44 INFO scheduler.TaskSetManager: Finished task 161.0 in stage 12.0 (TID 172) in 110 ms on localhost (162/199)
16/03/27 03:21:44 INFO executor.Executor: Running task 162.0 in stage 12.0 (TID 173)
16/03/27 03:21:44 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:44 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
16/03/27 03:21:44 INFO executor.Executor: Finished task 162.0 in stage 12.0 (TID 173). 1609 bytes result sent to driver
16/03/27 03:21:44 INFO scheduler.TaskSetManager: Starting task 163.0 in stage 12.0 (TID 174, localhost, partition 164,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:44 INFO scheduler.TaskSetManager: Finished task 162.0 in stage 12.0 (TID 173) in 142 ms on localhost (163/199)
16/03/27 03:21:44 INFO executor.Executor: Running task 163.0 in stage 12.0 (TID 174)
16/03/27 03:21:44 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:44 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:44 INFO executor.Executor: Finished task 163.0 in stage 12.0 (TID 174). 1609 bytes result sent to driver
16/03/27 03:21:44 INFO scheduler.TaskSetManager: Starting task 164.0 in stage 12.0 (TID 175, localhost, partition 165,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:44 INFO scheduler.TaskSetManager: Finished task 163.0 in stage 12.0 (TID 174) in 73 ms on localhost (164/199)
16/03/27 03:21:44 INFO executor.Executor: Running task 164.0 in stage 12.0 (TID 175)
16/03/27 03:21:44 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:44 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:44 INFO executor.Executor: Finished task 164.0 in stage 12.0 (TID 175). 1609 bytes result sent to driver
16/03/27 03:21:44 INFO scheduler.TaskSetManager: Starting task 165.0 in stage 12.0 (TID 176, localhost, partition 166,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:44 INFO scheduler.TaskSetManager: Finished task 164.0 in stage 12.0 (TID 175) in 108 ms on localhost (165/199)
16/03/27 03:21:44 INFO executor.Executor: Running task 165.0 in stage 12.0 (TID 176)
16/03/27 03:21:44 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:44 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:44 INFO executor.Executor: Finished task 165.0 in stage 12.0 (TID 176). 1609 bytes result sent to driver
16/03/27 03:21:44 INFO scheduler.TaskSetManager: Starting task 166.0 in stage 12.0 (TID 177, localhost, partition 167,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:44 INFO scheduler.TaskSetManager: Finished task 165.0 in stage 12.0 (TID 176) in 95 ms on localhost (166/199)
16/03/27 03:21:44 INFO executor.Executor: Running task 166.0 in stage 12.0 (TID 177)
16/03/27 03:21:44 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:44 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms
16/03/27 03:21:44 INFO executor.Executor: Finished task 166.0 in stage 12.0 (TID 177). 1609 bytes result sent to driver
16/03/27 03:21:44 INFO scheduler.TaskSetManager: Starting task 167.0 in stage 12.0 (TID 178, localhost, partition 168,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:44 INFO scheduler.TaskSetManager: Finished task 166.0 in stage 12.0 (TID 177) in 96 ms on localhost (167/199)
16/03/27 03:21:44 INFO executor.Executor: Running task 167.0 in stage 12.0 (TID 178)
16/03/27 03:21:44 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:44 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
16/03/27 03:21:44 INFO executor.Executor: Finished task 167.0 in stage 12.0 (TID 178). 1609 bytes result sent to driver
16/03/27 03:21:44 INFO scheduler.TaskSetManager: Starting task 168.0 in stage 12.0 (TID 179, localhost, partition 169,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:44 INFO scheduler.TaskSetManager: Finished task 167.0 in stage 12.0 (TID 178) in 137 ms on localhost (168/199)
16/03/27 03:21:44 INFO executor.Executor: Running task 168.0 in stage 12.0 (TID 179)
16/03/27 03:21:44 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:44 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 25 ms
16/03/27 03:21:45 INFO codegen.GenerateMutableProjection: Code generated in 384.040213 ms
16/03/27 03:21:45 INFO codegen.GenerateUnsafeProjection: Code generated in 192.868769 ms
16/03/27 03:21:45 INFO executor.Executor: Finished task 168.0 in stage 12.0 (TID 179). 1895 bytes result sent to driver
16/03/27 03:21:45 INFO scheduler.TaskSetManager: Starting task 169.0 in stage 12.0 (TID 180, localhost, partition 170,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:45 INFO scheduler.TaskSetManager: Finished task 168.0 in stage 12.0 (TID 179) in 926 ms on localhost (169/199)
16/03/27 03:21:45 INFO executor.Executor: Running task 169.0 in stage 12.0 (TID 180)
16/03/27 03:21:45 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:45 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 69 ms
16/03/27 03:21:45 INFO executor.Executor: Finished task 169.0 in stage 12.0 (TID 180). 1609 bytes result sent to driver
16/03/27 03:21:46 INFO scheduler.TaskSetManager: Starting task 170.0 in stage 12.0 (TID 181, localhost, partition 171,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:46 INFO scheduler.TaskSetManager: Finished task 169.0 in stage 12.0 (TID 180) in 620 ms on localhost (170/199)
16/03/27 03:21:46 INFO executor.Executor: Running task 170.0 in stage 12.0 (TID 181)
16/03/27 03:21:46 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:46 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:46 INFO executor.Executor: Finished task 170.0 in stage 12.0 (TID 181). 1609 bytes result sent to driver
16/03/27 03:21:46 INFO scheduler.TaskSetManager: Starting task 171.0 in stage 12.0 (TID 182, localhost, partition 172,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:46 INFO scheduler.TaskSetManager: Finished task 170.0 in stage 12.0 (TID 181) in 585 ms on localhost (171/199)
16/03/27 03:21:46 INFO executor.Executor: Running task 171.0 in stage 12.0 (TID 182)
16/03/27 03:21:46 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:46 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:46 INFO executor.Executor: Finished task 171.0 in stage 12.0 (TID 182). 1609 bytes result sent to driver
16/03/27 03:21:46 INFO scheduler.TaskSetManager: Starting task 172.0 in stage 12.0 (TID 183, localhost, partition 173,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:46 INFO scheduler.TaskSetManager: Finished task 171.0 in stage 12.0 (TID 182) in 252 ms on localhost (172/199)
16/03/27 03:21:46 INFO executor.Executor: Running task 172.0 in stage 12.0 (TID 183)
16/03/27 03:21:46 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:46 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:46 INFO executor.Executor: Finished task 172.0 in stage 12.0 (TID 183). 1609 bytes result sent to driver
16/03/27 03:21:46 INFO scheduler.TaskSetManager: Starting task 173.0 in stage 12.0 (TID 184, localhost, partition 174,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:46 INFO executor.Executor: Running task 173.0 in stage 12.0 (TID 184)
16/03/27 03:21:46 INFO scheduler.TaskSetManager: Finished task 172.0 in stage 12.0 (TID 183) in 217 ms on localhost (173/199)
16/03/27 03:21:46 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:46 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
16/03/27 03:21:46 INFO executor.Executor: Finished task 173.0 in stage 12.0 (TID 184). 1609 bytes result sent to driver
16/03/27 03:21:46 INFO scheduler.TaskSetManager: Starting task 174.0 in stage 12.0 (TID 185, localhost, partition 175,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:46 INFO scheduler.TaskSetManager: Finished task 173.0 in stage 12.0 (TID 184) in 228 ms on localhost (174/199)
16/03/27 03:21:46 INFO executor.Executor: Running task 174.0 in stage 12.0 (TID 185)
16/03/27 03:21:47 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:47 INFO executor.Executor: Finished task 174.0 in stage 12.0 (TID 185). 1609 bytes result sent to driver
16/03/27 03:21:47 INFO scheduler.TaskSetManager: Starting task 175.0 in stage 12.0 (TID 186, localhost, partition 176,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:47 INFO scheduler.TaskSetManager: Finished task 174.0 in stage 12.0 (TID 185) in 137 ms on localhost (175/199)
16/03/27 03:21:47 INFO executor.Executor: Running task 175.0 in stage 12.0 (TID 186)
16/03/27 03:21:47 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:47 INFO executor.Executor: Finished task 175.0 in stage 12.0 (TID 186). 1609 bytes result sent to driver
16/03/27 03:21:47 INFO scheduler.TaskSetManager: Starting task 176.0 in stage 12.0 (TID 187, localhost, partition 177,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:47 INFO scheduler.TaskSetManager: Finished task 175.0 in stage 12.0 (TID 186) in 187 ms on localhost (176/199)
16/03/27 03:21:47 INFO executor.Executor: Running task 176.0 in stage 12.0 (TID 187)
16/03/27 03:21:47 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms
16/03/27 03:21:47 INFO executor.Executor: Finished task 176.0 in stage 12.0 (TID 187). 1609 bytes result sent to driver
16/03/27 03:21:47 INFO scheduler.TaskSetManager: Starting task 177.0 in stage 12.0 (TID 188, localhost, partition 178,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:47 INFO scheduler.TaskSetManager: Finished task 176.0 in stage 12.0 (TID 187) in 214 ms on localhost (177/199)
16/03/27 03:21:47 INFO executor.Executor: Running task 177.0 in stage 12.0 (TID 188)
16/03/27 03:21:47 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:47 INFO executor.Executor: Finished task 177.0 in stage 12.0 (TID 188). 1609 bytes result sent to driver
16/03/27 03:21:47 INFO scheduler.TaskSetManager: Starting task 178.0 in stage 12.0 (TID 189, localhost, partition 179,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:47 INFO scheduler.TaskSetManager: Finished task 177.0 in stage 12.0 (TID 188) in 118 ms on localhost (178/199)
16/03/27 03:21:47 INFO executor.Executor: Running task 178.0 in stage 12.0 (TID 189)
16/03/27 03:21:47 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
16/03/27 03:21:47 INFO executor.Executor: Finished task 178.0 in stage 12.0 (TID 189). 1609 bytes result sent to driver
16/03/27 03:21:47 INFO scheduler.TaskSetManager: Starting task 179.0 in stage 12.0 (TID 190, localhost, partition 180,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:47 INFO scheduler.TaskSetManager: Finished task 178.0 in stage 12.0 (TID 189) in 154 ms on localhost (179/199)
16/03/27 03:21:47 INFO executor.Executor: Running task 179.0 in stage 12.0 (TID 190)
16/03/27 03:21:47 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:47 INFO executor.Executor: Finished task 179.0 in stage 12.0 (TID 190). 1609 bytes result sent to driver
16/03/27 03:21:47 INFO scheduler.TaskSetManager: Starting task 180.0 in stage 12.0 (TID 191, localhost, partition 181,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:47 INFO scheduler.TaskSetManager: Finished task 179.0 in stage 12.0 (TID 190) in 115 ms on localhost (180/199)
16/03/27 03:21:47 INFO executor.Executor: Running task 180.0 in stage 12.0 (TID 191)
16/03/27 03:21:47 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:47 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
16/03/27 03:21:47 INFO executor.Executor: Finished task 180.0 in stage 12.0 (TID 191). 1609 bytes result sent to driver
16/03/27 03:21:47 INFO scheduler.TaskSetManager: Starting task 181.0 in stage 12.0 (TID 192, localhost, partition 182,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:47 INFO scheduler.TaskSetManager: Finished task 180.0 in stage 12.0 (TID 191) in 106 ms on localhost (181/199)
16/03/27 03:21:48 INFO executor.Executor: Running task 181.0 in stage 12.0 (TID 192)
16/03/27 03:21:48 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:48 INFO executor.Executor: Finished task 181.0 in stage 12.0 (TID 192). 1609 bytes result sent to driver
16/03/27 03:21:48 INFO scheduler.TaskSetManager: Starting task 182.0 in stage 12.0 (TID 193, localhost, partition 183,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:48 INFO scheduler.TaskSetManager: Finished task 181.0 in stage 12.0 (TID 192) in 220 ms on localhost (182/199)
16/03/27 03:21:48 INFO executor.Executor: Running task 182.0 in stage 12.0 (TID 193)
16/03/27 03:21:48 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
16/03/27 03:21:48 INFO executor.Executor: Finished task 182.0 in stage 12.0 (TID 193). 1609 bytes result sent to driver
16/03/27 03:21:48 INFO scheduler.TaskSetManager: Starting task 183.0 in stage 12.0 (TID 194, localhost, partition 184,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:48 INFO scheduler.TaskSetManager: Finished task 182.0 in stage 12.0 (TID 193) in 111 ms on localhost (183/199)
16/03/27 03:21:48 INFO executor.Executor: Running task 183.0 in stage 12.0 (TID 194)
16/03/27 03:21:48 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
16/03/27 03:21:48 INFO executor.Executor: Finished task 183.0 in stage 12.0 (TID 194). 1609 bytes result sent to driver
16/03/27 03:21:48 INFO scheduler.TaskSetManager: Starting task 184.0 in stage 12.0 (TID 195, localhost, partition 185,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:48 INFO scheduler.TaskSetManager: Finished task 183.0 in stage 12.0 (TID 194) in 113 ms on localhost (184/199)
16/03/27 03:21:48 INFO executor.Executor: Running task 184.0 in stage 12.0 (TID 195)
16/03/27 03:21:48 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms
16/03/27 03:21:48 INFO executor.Executor: Finished task 184.0 in stage 12.0 (TID 195). 1609 bytes result sent to driver
16/03/27 03:21:48 INFO scheduler.TaskSetManager: Starting task 185.0 in stage 12.0 (TID 196, localhost, partition 186,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:48 INFO scheduler.TaskSetManager: Finished task 184.0 in stage 12.0 (TID 195) in 83 ms on localhost (185/199)
16/03/27 03:21:48 INFO executor.Executor: Running task 185.0 in stage 12.0 (TID 196)
16/03/27 03:21:48 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:48 INFO executor.Executor: Finished task 185.0 in stage 12.0 (TID 196). 1609 bytes result sent to driver
16/03/27 03:21:48 INFO scheduler.TaskSetManager: Starting task 186.0 in stage 12.0 (TID 197, localhost, partition 187,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:48 INFO scheduler.TaskSetManager: Finished task 185.0 in stage 12.0 (TID 196) in 197 ms on localhost (186/199)
16/03/27 03:21:48 INFO executor.Executor: Running task 186.0 in stage 12.0 (TID 197)
16/03/27 03:21:48 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:48 INFO executor.Executor: Finished task 186.0 in stage 12.0 (TID 197). 1609 bytes result sent to driver
16/03/27 03:21:48 INFO scheduler.TaskSetManager: Starting task 187.0 in stage 12.0 (TID 198, localhost, partition 188,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:48 INFO scheduler.TaskSetManager: Finished task 186.0 in stage 12.0 (TID 197) in 97 ms on localhost (187/199)
16/03/27 03:21:48 INFO executor.Executor: Running task 187.0 in stage 12.0 (TID 198)
16/03/27 03:21:48 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:48 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:48 INFO executor.Executor: Finished task 187.0 in stage 12.0 (TID 198). 1908 bytes result sent to driver
16/03/27 03:21:48 INFO scheduler.TaskSetManager: Starting task 188.0 in stage 12.0 (TID 199, localhost, partition 189,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:49 INFO scheduler.TaskSetManager: Finished task 187.0 in stage 12.0 (TID 198) in 249 ms on localhost (188/199)
16/03/27 03:21:49 INFO executor.Executor: Running task 188.0 in stage 12.0 (TID 199)
16/03/27 03:21:49 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:49 INFO executor.Executor: Finished task 188.0 in stage 12.0 (TID 199). 1609 bytes result sent to driver
16/03/27 03:21:49 INFO scheduler.TaskSetManager: Starting task 189.0 in stage 12.0 (TID 200, localhost, partition 190,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:49 INFO scheduler.TaskSetManager: Finished task 188.0 in stage 12.0 (TID 199) in 164 ms on localhost (189/199)
16/03/27 03:21:49 INFO executor.Executor: Running task 189.0 in stage 12.0 (TID 200)
16/03/27 03:21:49 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:49 INFO executor.Executor: Finished task 189.0 in stage 12.0 (TID 200). 1609 bytes result sent to driver
16/03/27 03:21:49 INFO scheduler.TaskSetManager: Starting task 190.0 in stage 12.0 (TID 201, localhost, partition 191,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:49 INFO scheduler.TaskSetManager: Finished task 189.0 in stage 12.0 (TID 200) in 157 ms on localhost (190/199)
16/03/27 03:21:49 INFO executor.Executor: Running task 190.0 in stage 12.0 (TID 201)
16/03/27 03:21:49 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms
16/03/27 03:21:49 INFO executor.Executor: Finished task 190.0 in stage 12.0 (TID 201). 1609 bytes result sent to driver
16/03/27 03:21:49 INFO scheduler.TaskSetManager: Starting task 191.0 in stage 12.0 (TID 202, localhost, partition 192,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:49 INFO scheduler.TaskSetManager: Finished task 190.0 in stage 12.0 (TID 201) in 115 ms on localhost (191/199)
16/03/27 03:21:49 INFO executor.Executor: Running task 191.0 in stage 12.0 (TID 202)
16/03/27 03:21:49 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
16/03/27 03:21:49 INFO executor.Executor: Finished task 191.0 in stage 12.0 (TID 202). 1609 bytes result sent to driver
16/03/27 03:21:49 INFO scheduler.TaskSetManager: Starting task 192.0 in stage 12.0 (TID 203, localhost, partition 193,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:49 INFO scheduler.TaskSetManager: Finished task 191.0 in stage 12.0 (TID 202) in 173 ms on localhost (192/199)
16/03/27 03:21:49 INFO executor.Executor: Running task 192.0 in stage 12.0 (TID 203)
16/03/27 03:21:49 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
16/03/27 03:21:49 INFO executor.Executor: Finished task 192.0 in stage 12.0 (TID 203). 1609 bytes result sent to driver
16/03/27 03:21:49 INFO scheduler.TaskSetManager: Starting task 193.0 in stage 12.0 (TID 204, localhost, partition 194,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:49 INFO scheduler.TaskSetManager: Finished task 192.0 in stage 12.0 (TID 203) in 70 ms on localhost (193/199)
16/03/27 03:21:49 INFO executor.Executor: Running task 193.0 in stage 12.0 (TID 204)
16/03/27 03:21:49 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:49 INFO executor.Executor: Finished task 193.0 in stage 12.0 (TID 204). 1609 bytes result sent to driver
16/03/27 03:21:49 INFO scheduler.TaskSetManager: Starting task 194.0 in stage 12.0 (TID 205, localhost, partition 195,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:49 INFO scheduler.TaskSetManager: Finished task 193.0 in stage 12.0 (TID 204) in 122 ms on localhost (194/199)
16/03/27 03:21:49 INFO executor.Executor: Running task 194.0 in stage 12.0 (TID 205)
16/03/27 03:21:49 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:49 INFO executor.Executor: Finished task 194.0 in stage 12.0 (TID 205). 1609 bytes result sent to driver
16/03/27 03:21:49 INFO scheduler.TaskSetManager: Starting task 195.0 in stage 12.0 (TID 206, localhost, partition 196,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:49 INFO scheduler.TaskSetManager: Finished task 194.0 in stage 12.0 (TID 205) in 116 ms on localhost (195/199)
16/03/27 03:21:49 INFO executor.Executor: Running task 195.0 in stage 12.0 (TID 206)
16/03/27 03:21:49 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
16/03/27 03:21:49 INFO executor.Executor: Finished task 195.0 in stage 12.0 (TID 206). 1609 bytes result sent to driver
16/03/27 03:21:49 INFO scheduler.TaskSetManager: Starting task 196.0 in stage 12.0 (TID 207, localhost, partition 197,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:49 INFO scheduler.TaskSetManager: Finished task 195.0 in stage 12.0 (TID 206) in 89 ms on localhost (196/199)
16/03/27 03:21:49 INFO executor.Executor: Running task 196.0 in stage 12.0 (TID 207)
16/03/27 03:21:49 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:49 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
16/03/27 03:21:50 INFO executor.Executor: Finished task 196.0 in stage 12.0 (TID 207). 1609 bytes result sent to driver
16/03/27 03:21:50 INFO scheduler.TaskSetManager: Starting task 197.0 in stage 12.0 (TID 208, localhost, partition 198,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:50 INFO scheduler.TaskSetManager: Finished task 196.0 in stage 12.0 (TID 207) in 72 ms on localhost (197/199)
16/03/27 03:21:50 INFO executor.Executor: Running task 197.0 in stage 12.0 (TID 208)
16/03/27 03:21:50 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:50 INFO executor.Executor: Finished task 197.0 in stage 12.0 (TID 208). 1609 bytes result sent to driver
16/03/27 03:21:50 INFO scheduler.TaskSetManager: Starting task 198.0 in stage 12.0 (TID 209, localhost, partition 199,NODE_LOCAL, 1999 bytes)
16/03/27 03:21:50 INFO scheduler.TaskSetManager: Finished task 197.0 in stage 12.0 (TID 208) in 101 ms on localhost (198/199)
16/03/27 03:21:50 INFO executor.Executor: Running task 198.0 in stage 12.0 (TID 209)
16/03/27 03:21:50 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
16/03/27 03:21:50 INFO storage.ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
16/03/27 03:21:50 INFO executor.Executor: Finished task 198.0 in stage 12.0 (TID 209). 1908 bytes result sent to driver
16/03/27 03:21:50 INFO scheduler.TaskSetManager: Finished task 198.0 in stage 12.0 (TID 209) in 194 ms on localhost (199/199)
16/03/27 03:21:50 INFO scheduler.TaskSchedulerImpl: Removed TaskSet 12.0, whose tasks have all completed, from pool
16/03/27 03:21:50 INFO scheduler.DAGScheduler: ResultStage 12 (show at :28) finished in 28.290 s
16/03/27 03:21:50 INFO scheduler.DAGScheduler: Job 9 finished: show at :28, took 29.282706 s
+----+-----+
| age|count|
+----+-----+
|null| 1|
| 19| 1|
| 30| 1|

+----+-----+

以上内容是王家林老师DT大数据梦工厂《 IMF传奇行动》第57课的学习笔记。
王家林老师是Spark、Flink、Docker、Android技术中国区布道师。Spark亚太研究院院长和首席专家，DT大数据梦工厂创始人，Android软硬整合源码级专家，英语发音魔术师，健身狂热爱好者。

微信公众账号：DT_Spark

联系邮箱[email protected]

电话：18610086859

QQ:1740415547

微信号：18610086859

新浪微博：ilovepains

第57课：SparkSQL案例实战学习笔记

你可能感兴趣的:(Spark)