Getting ready 

We require a working installation of Spark. This means that you would have followed the steps outlined in the previous chapter. As a reminder, to start PySpark shell for your local Spark cluster, you can run this command:

./bin/pyspark --master local[n]

Where n is the number of cores.