Databricks recently announced GraphFrames, awesome Spark extension to implement graph processing using DataFrames.
I performed graph analysis and visualized beautiful ball movement network of Golden State Warriors using rich data provided by NBA.com’s stats
Pass network of Warriors
### Passes received & made
The league’s MVP Stephen Curry received the most passes and the team’s MVP Draymond Green provides the most passes.
We’ve seen most of the offense start with their pick & roll or Curry’s off-ball cuts with Green as a pass provider.
Label Propagation is an algorithm to find communities in a graph network.
The algorithm nicely classifies players into backcourt and frontcourt without providing label!
McAdoo, James Michael
PageRank can detect important nodes (players in this case) in a network.
It’s no surprise that Stephen Curry, Draymond Green and Klay Thompson are the top three.
The algoritm detects Shaun Livingston and Andre Iguodala play key roles in the Warriors’ passing games.
McAdoo, James Michael
Here is a network visualization using the results of above.
Node size: pagerank
Node color: community
Link width: passes received & made
I used the endpoint playerdashptpass and saved data for all the players in the team into local JSON files.
The data is about who passed how many times in 2015-16 season
JSON -> Panda’s DataFrame
Then I combined all the individual JSON files into a single DataFrame for later aggregation.
Prepare vertices and edges
You need a special data format for GraphFrames in Spark, vertices and edges.
Vertices are lis of nodes and IDs in a graph.
Edges are the relathionship of the nodes.
You can pass additional features like weight but I couldn’t find out a way to utilize there features well in later analysis.
A workaround I took below is brute force and not even a proper graph operation but works (suggestions/comments are very welcome).
Bring the local vertices and edges to Spark and let it spark.
Visualise the network
When you run gsw_passing_network.py in my github repo, you have passes.csv, groups.csv and size.csv in your working directory.
I used networkD3 package in R to make a cool interactive D3 chart.