spark python编程 林子雨_林子雨编著《Spark编程基础(Python版)》教材第5章的命令行和代码...

林子雨、郑海山、赖永炫编著《Spark编程基础(Python版)》(教材官网)教材中的代码，在纸质教材中的印刷效果，可能会影响读者对代码的理解，为了方便读者正确理解代码或者直接拷贝代码用于上机实验，这里提供全书配套的所有代码。

查看所有章节代码

第5章 Spark SQL

from pyspark import SparkContext,SparkConf

from pyspark.sql import SparkSession

spark = SparkSession.builder.config(conf = SparkConf()).getOrCreate()

>>> df=spark.read.json("file:///usr/local/spark/examples/src/main/resources/people.json")

>>> df.show()

>>> peopleDF = spark.read.format("json").\

... load("file:///usr/local/spark/examples/src/main/resources/people.json")

>>> peopleDF.select("name", "age").write.format("json").\

... save("file:///usr/local/spark/mycode/sparksql/newpeople.json")

>>> peopleDF.select("name").write.format("text").\

... save("file:///usr/local/spark/mycode/sparksql/newpeople.txt")

>>> peopleDF = spark.read.format("json").\

... load("file:///usr/local/spark/mycode/sparksql/newpeople.json")

>>> peopleDF.show()

>>> df=spark.read.json("file:///usr/local/spark/examples/src/main/resources/people.j