封面
版权信息
Packt Upsell
Why subscribe?
PacktPub.com
Contributors
About the authors
About the reviewer
Packt is searching for authors like you
Preface
Who this book is for
What this book covers
To get the most out of this book
Download the example code files
Download the color images
Conventions used
Sections
Getting ready
How to do it...
How it works...
There's more...
See also
Get in touch
Reviews
Installing and Configuring Spark
Introduction
Installing Spark requirements
Getting ready
How to do it...
How it works...
There's more...
Installing Java
Installing Python
Installing R
Installing Scala
Installing Maven
Updating PATH
Installing Spark from sources
Getting ready
How to do it...
How it works...
There's more...
See also
Installing Spark from binaries
Getting ready
How to do it...
How it works...
There's more...
Configuring a local instance of Spark
Getting ready
How to do it...
How it works...
See also
Configuring a multi-node instance of Spark
Getting ready
How to do it...
How it works...
See also
Installing Jupyter
Getting ready
How to do it...
How it works...
There's more...
See also
Configuring a session in Jupyter
Getting ready
How to do it...
How it works...
There's more...
See also
Working with Cloudera Spark images
Getting ready
How to do it...
How it works...
Abstracting Data with RDDs
Introduction
Creating RDDs
Getting ready
How to do it...
How it works...
Spark context parallelize method
.take(...) method
Reading data from files
Getting ready
How to do it...
How it works...
.textFile(...) method
.map(...) method
Partitions and performance
Overview of RDD transformations
Getting ready
How to do it...
.map(...) transformation
.filter(...) transformation
.flatMap(...) transformation
.distinct() transformation
.sample(...) transformation
.join(...) transformation
.repartition(...) transformation
.zipWithIndex() transformation
.reduceByKey(...) transformation
.sortByKey(...) transformation
.union(...) transformation
.mapPartitionsWithIndex(...) transformation
How it works...
Overview of RDD actions
Getting ready
How to do it...
.take(...) action
.collect() action
.reduce(...) action
.count() action
.saveAsTextFile(...) action
How it works...
Pitfalls of using RDDs
Getting ready
How to do it...
How it works...
Abstracting Data with DataFrames
Introduction
Creating DataFrames
Getting ready
How to do it...
How it works...
There's more...
From JSON
From CSV
See also
Accessing underlying RDDs
Getting ready
How to do it...
How it works...
Performance optimizations
Getting ready
How to do it...
How it works...
There's more...
See also
Inferring the schema using reflection
Getting ready
How to do it...
How it works...
See also
Specifying the schema programmatically
Getting ready
How to do it...
How it works...
See also
Creating a temporary table
Getting ready
How to do it...
How it works...
There's more...
Using SQL to interact with DataFrames
Getting ready
How to do it...
How it works...
There's more...
Overview of DataFrame transformations
Getting ready
How to do it...
The .select(...) transformation
The .filter(...) transformation
The .groupBy(...) transformation
The .orderBy(...) transformation
The .withColumn(...) transformation
The .join(...) transformation
The .unionAll(...) transformation
The .distinct(...) transformation
The .repartition(...) transformation
The .fillna(...) transformation
The .dropna(...) transformation
The .dropDuplicates(...) transformation
The .summary() and .describe() transformations
The .freqItems(...) transformation
See also
Overview of DataFrame actions
Getting ready
How to do it...
The .show(...) action
The .collect() action
The .take(...) action
The .toPandas() action
See also
Preparing Data for Modeling
Introduction
Handling duplicates
Getting ready
How to do it...
How it works...
There's more...
Only IDs differ
ID collisions
Handling missing observations
Getting ready
How to do it...
How it works...
Missing observations per row
Missing observations per column
There's more...
See also
Handling outliers
Getting ready
How to do it...
How it works...
See also
Exploring descriptive statistics
Getting ready
How to do it...
How it works...
There's more...
Descriptive statistics for aggregated columns
See also
Computing correlations
Getting ready
How to do it...
How it works...
There's more...
Drawing histograms
Getting ready
How to do it...
How it works...
There's more...
See also
Visualizing interactions between features
Getting ready
How to do it...
How it works...
There's more...
Machine Learning with MLlib
Loading the data
Getting ready
How to do it...
How it works...
There's more...
Exploring the data
Getting ready
How to do it...
How it works...
Numerical features
Categorical features
There's more...
See also
Testing the data
Getting ready
How to do it...
How it works...
See also...
Transforming the data
Getting ready
How to do it...
How it works...
There's more...
See also...
Standardizing the data
Getting ready
How to do it...
How it works...
Creating an RDD for training
Getting ready
How to do it...
Classification
Regression
How it works...
There's more...
See also
Predicting hours of work for census respondents
Getting ready
How to do it...
How it works...
Forecasting the income levels of census respondents
Getting ready
How to do it...
How it works...
There's more...
Building a clustering models
Getting ready
How to do it...
How it works...
There's more...
See also
Computing performance statistics
Getting ready
How to do it...
How it works...
Regression metrics
Classification metrics
See also
Machine Learning with the ML Module
Introducing Transformers
Getting ready
How to do it...
How it works...
There's more...
See also
Introducing Estimators
Getting ready
How to do it...
How it works...
There's more...
Introducing Pipelines
Getting ready
How to do it...
How it works...
See also
Selecting the most predictable features
Getting ready
How to do it...
How it works...
There's more...
See also
Predicting forest coverage types
Getting ready
How to do it...
How it works...
There's more...
Estimating forest elevation
Getting ready
How to do it...
How it works...
There's more...
Clustering forest cover types
Getting ready
How to do it...
How it works...
See also
Tuning hyperparameters
Getting ready
How to do it...
How it works...
There's more...
Extracting features from text
Getting ready
How to do it...
How it works...
There's more...
See also
Discretizing continuous variables
Getting ready
How to do it...
How it works...
Standardizing continuous variables
Getting ready
How to do it...
How it works...
Topic mining
Getting ready
How to do it...
How it works...
Structured Streaming with PySpark
Introduction
Understanding Spark Streaming
Understanding DStreams
Getting ready
How to do it...
Terminal 1 – Netcat window
Terminal 2 – Spark Streaming window
How it works...
There's more...
Understanding global aggregations
Getting ready
How to do it...
Terminal 1 – Netcat window
Terminal 2 – Spark Streaming window
How it works...
Continuous aggregation with structured streaming
Getting ready
How to do it...
Terminal 1 – Netcat window
Terminal 2 – Spark Streaming window
How it works...
GraphFrames – Graph Theory with PySpark
Introduction
Installing GraphFrames
Getting ready
How to do it...
How it works...
Preparing the data
Getting ready
How to do it...
How it works...
There's more...
Building the graph
How to do it...
How it works...
Running queries against the graph
Getting ready
How to do it...
How it works...
Understanding the graph
Getting ready
How to do it...
How it works...
Using PageRank to determine airport ranking
Getting ready
How to do it...
How it works...
Finding the fewest number of connections
Getting ready
How to do it...
How it works...
There's more...
See also
Visualizing the graph
Getting ready
How to do it...
How it works...
更新时间：2021-06-18 19:07:31
完结共407章
倒序