**Data Imaginist - R posts**, and kindly contributed to R-bloggers)

I’m very pleased to announce that my new package `tidygraph`

is now

available on CRAN. As the name

suggests, `tidygraph`

is an entry into the tidyverse that provides a tidy

framework for all things relational (networks/graphs, trees, etc.). `tidygraph`

is a relatively big package in terms of exported functions (280 exported

symbols) so all functions will not be covered in this release note. I will

however provide an overview of all the areas that `tidygraph`

touches upon so you

should have a pretty good grasp on what the package can do for you.

## Tidy network data?

There’s a discrepancy between relational data and the tidy data idea, in that

relational data cannot in any meaningful way be encoded as a single tidy

data frame. On the other hand, both node and edge data by itself fits very well

within the tidy concept as each node and edge is, in a sense, a single

observation. Thus, a close approximation of tidyness for relational data is two

tidy data frames, one describing the node data and one describing the edge data.

### The tbl_graph object

Underneath the hood of `tidygraph`

lies the well-oiled machinery of igraph,

ensuring efficient graph manipulation. Rather than keeping the node and edge

data in a list and creating `igraph`

objects on the fly when needed, `tidygraph`

subclasses `igraph`

with the `tbl_graph`

class and simply exposes it in a tidy

manner. This ensures that all your beloved algorithms that expects `igraph`

objects still works with `tbl_graph`

objects. Further, `tidygraph`

is very

careful not to override any of `igraph`

s exports so the two packages can coexist

quite happily.

To underline the tidyness of the `tbl_graph`

class the print method shows the

object as two tibbles along with additional network information.

`tbl_graph`

objects can be created directly using the `tbl_graph()`

function

that takes a node data.frame and an edge data.frame. On top of that, `tidygraph`

also provides coercion from a huge amount of relational data structures. The

following list gives the packages/classes that can currently be converted to

`tbl_graph`

s, using the `as_tbl_graph`

function:

`data.frame`

,`list`

,`matrix`

from`base`

`igraph`

from`igraph`

`network`

from`network`

`dendrogram`

and`hclust`

from`stats`

`Node`

from`data.tree`

`phylo`

and`evonet`

from`ape`

`graphNEL`

,`graphAM`

,`graphBAM`

from`graph`

(in Bioconductor)

For all of the coercions you can expect that data on the nodes and edges are

kept and available after conversion to `tbl_graph`

:

Lastly, `tidygraph`

also wraps the multitude of graph constructors available in

`igraph`

and exports them under the `create_*()`

family of functions for

deterministic constructors (e.g. the call to `create_ring(10)`

above) and the

`play_*()`

family for constructors that incorporate sampling (e.g.

`play_erdos_renyi()`

for creating graphs with a fixed edge probability). All of

these functions provide a consistent argument naming scheme to make them easier

to use and understand.

### Meet a new verb…

There are many ways a multitable setup could fit into the tidyverse. There could

be an added qualifier to the verbs such as `mutate_nodes()`

and `filter_edges()`

or each verb could take an additional argument specifying what is targeted e.g.

`arrange(..., target = 'nodes')`

. Both of these approachable are viable but

would require a huge amount of typing as well as being taxing to support down

the line.

The approach used by `tidygraph`

is to let the data object itself carry around

a pointer to the active data frame that should be the target of manipulation.

This pointer is changed using the `activate()`

verb, which, on top of changing

which part of the data is being worked on, also changes the print output to show

the currently active data on top:

As can be seen, `activate()`

takes a single argument specifying the part of the

data that should be targeted for subsequent operations as an unquoted symbol.

`tidygraph`

continues the naming conventions from `ggraph`

using `nodes`

and

`edges`

to denote the entities and their connections respectively, but

`vertices`

and `links`

are allowed synonyms inside `activate()`

.

The current active data can always be extracted as a tibble using `as_tibble()`

## The dplyr verbs

Using `activate()`

it is possible to use the well known `dplyr`

verbs as one

would expect without much hassle:

In the above the `.N()`

function is used to gain access to the node data while

manipulating the edge data. Similarly `.E()`

will give you the edge data and

`.G()`

will give you the `tbl_graph`

object itself.

Some verbs have effects outside of the currently active data.

`filter()`

/`slice()`

on node data will remove the edges terminating at the

removed nodes and `arrange()`

on nodes will change the indexes of the `to`

and

`from`

column in the edge data.

While one might expect all of `dplyr`

s verbs to be supported in that manner,

there is a clear limitation in the relational data structure that requires rows

to maintain their identity. Thus, `summarise()`

and `do()`

are not allowed as

there is no clear interpretation of how alterations on the node and edge data

with these verbs should be interpreted. If these operations are required I

suggest applying them to a tibble representation and then joining the result

back in.

Speaking of joining, all joins from `dplyr`

are supported. Nodes and edges are

added and removed as required by the join. New edge data to be joined in must

have a `to`

and `from`

column referencing valid nodes in the existing graph.

### Expanding the vocabulary

On top of what has been showed so far, `tidygraph`

provides an assortment of

graph specific verbs that can be used to power your analysis and manipulation.

Analogous to `bind_rows()`

, `tidygraph`

provides three functions to expand your

data: `bind_nodes()`

and `bind_edges()`

append nodes and edges to the graph

respectively. As with the join functions `bind_edges()`

must contain valid

`from`

and `to`

columns. `bind_graphs()`

allows you to combine multiple graphs

in the same graph structure resulting in each original graph to become a

component in the returned graph.

While `bind_graphs()`

cannot be used to create edges between the merged graphs

`graph_join()`

can do just that. It merges nodes using a `full_join()`

semantic

and keeps the individual edges from both graphs:

The standard `dplyr`

verbs protects the `to`

and `from`

columns in the edge data

in order to avoid accidental modification of the graph topology. If changing of

the terminal nodes are necessary the `reroute()`

verb will come in handy:

As can be seen, reroute works pretty much as a `to`

and `from`

specific

`mutate()`

, with the added benefit of incorporating a subset operator if only a

few edges should be changed.

## Making the most of graphs

While being able to use the `dplyr`

verbs on relational data is nice and all,

one of the reasons we are dealing with graph data in the first place is because

we need some graph-based algorithms for solving our problem at hand. If we need

to break out of the tidy workflow every time this was needed we wouldn’t have

gained much. Because of this `tidygraph`

has wrapped more or less all of

`igraph`

s algorithms in different ways, ensuring a consistent syntax as well as

output that fits into the tidy workflow. In the following we’re going to take a

look at these.

Central to all of these functions is that they know about which graph is being

computed on (in the same way that `n()`

knows about which tibble is currently in

scope). Furthermore they always return results matching the node or edge

position so they can be used directly in `mutate()`

calls.

### Node and edge types

On the top of our list of things we might be interested to know about is whether

nodes or edges are of specific types such as *leaf*, *sink*, *loop*,

etc. All of these functions return a logical vector indicating if the node or

edge belong to the specified group. To easily find functions that queries types,

all functions are prefixed with `node_is_*`

/`edge_is_*`

.

Another example could be to remove loop edges using `filter(!edge_is_loop())`

.

### Centrality

One of the simplest concepts when computing graph based values is that of

*centrality*, i.e. how central is a node or edge in the graph. As this

definition is inherently vague, a lot of different centrality scores exists that

all treat the concept of *central* a bit different. One of the famous ones is

the pagerank algorithm that was powering Google Search in the beginning.

`tidygraph`

currently has 11 different centrality measures and all of these are

prefixed with `centrality_*`

for easy discoverability. All of them returns a

numeric vector matching the nodes (or edges in the case of

`centrality_edge_betweenness()`

).

It is quite difficult to a priori decide which centrality measure makes most

sense for a problem at hand so having easy access to a large range of them in

a common syntax is a boon.

### Clustering

Another common operation is to group nodes based on the graph topology,

sometimes referred to as *community detection* based on its commonality in social

network analysis. All clustering algorithms from `igraph`

is available in

`tidygraph`

using the `group_*`

prefix. All of these functions return an integer

vector with nodes (or edges) sharing the same integer being grouped together.

### Node pairs

Some statistics are a measure between two nodes, such as distance or similarity

between nodes. In a tidy context one of the ends must always be the node defined

by the row, while the other can be any other node. All of the node pair

functions are prefixed with `node_*`

and ends with `_from`

/`_to`

if the measure

is not symmetric and `_with`

if it is; e.g. there’s both a `node_max_flow_to()`

and `node_max_flow_from()`

function while only a single `node_cocitation_with()`

function. The other part of the node pair can be specified as an integer vector

that will get recycled if needed, or a logical vector which will get recycled

and converted to indexes with `which()`

. This means that output from node type

functions can be used directly in the calls, e.g.

### Searches

An integral type of operation on graphs is to perform a search, that is, start

from one node and then traverse the edges until all nodes has been visited. The

most common approaches are either *breath first search* where all neighbors of

a node is visited before moving on to the next node, or *depth first serch*

where you move along to the next node immediately and only backtracks and visit

other neighbors when you’ve hit a dead end. Different statistics from these

searches are available in `tidygraph`

through the `bfs_*()`

and `dfs_*()`

family

of functions e.g. the distance to the start node along the search can be

obtained with `bfs_dist()`

/`dfs_dist()`

. The root node can be specified in the

same way as with node pairs. Sorting based on a search from the node with

highest centrality can thus be done with:

### Local measures

Often we find ourselves interested in the local neighborhood of a node for

various reasons. We might want to know the average degree around a node or the

number of triangles each node participate in. The `local_*()`

family of

functions provide access to a range of node measures that are dependent on the

local neighborhood of each node.

### All the rest

While an ontology of graph operations has been attempted in the different

functions above, there are some that falls outside. These have been lumped

together under the `node_*()`

and `edge_*()`

umbrellas and include things such

as topological ordering and Burt’s constraint among others. All of these

functions ensures a mutate-compatible output.

### Graph measures

Along with computations on the individual nodes and edges it is sometimes

necessary to get summary statistics on the graph itself. These can be simple

measures such as the number of nodes and edges as well as more involved measures

such as assortativity (the propensity of similar nodes to be connected). All of

these measures can be calculated through the `graph_*()`

function family and

they will all return a scalar.

## Mapping over nodes

Just to spice it all up a bit `tidygraph`

pulls `purrr`

into the mix and

provides some additional graph-centric takes on the familiar `map*()`

. More

specifically `tidygraph`

provides functionality to apply a function over nodes

as a breath or depth first search is carried out, while getting access to the

result of the computations coming before, as well as mapping over the local

neighborhood of each node. All of these function returns a list in their

bare bone form, but as with `purrr`

versions exists that ensures the output is of

a certain type (e.g. `map_bfs_dbl()`

).

### Mapping over searches

The search maps comes in two flavors. Either the nodes are mapped in the order

of the search, or they are mapped in the reverse order. In the first version,

each call will have access to the statistics and map results of all the nodes

that lies between itself and the root. In the second version each call will

have access to the results and statistics of all its offspring. Furthermore the

mapping function is passed the graph itself as well as all the search statistics

of the node currently being mapped over. An example would be to propagate the

*species* value in our iris clustering upwards as long as theirs agreement

between the children. For this to work, we will need the reverse version of a

breath first search to make sure that all children have been evaluated prior to

mapping over a node:

### Mapping over neighborhoods

The neighborhood map is exposed through `map_local()`

as well as its type safe

versions. The mapping function has a much simpler format as it simply gets

passed a subgraph representing the local neighborhood as well as the index of

the node in the original graph being mapped over. E.g. to get the number of

edges in the local neighborhood around each node, one would simply do:

## One last thing…

While the functions discussed above makes it easy to make slight changes to your

network topology it is less straightforward to make radical changes. Even more

so if the radical changes are only needed temporarily for the sake of a few

computations. This is where the new `morph()`

verb comes in handy (along with

the accompanying `unmorph()`

and `crystallise()`

verbs). In essence, `morph()`

lets you set up a temporary alternative version of your graph, make computations

on it using the standard `dplyr`

verbs, and then merge the changes back in

using `unmorph()`

. The types of alternative representations are varied and can

be extended by the user. Nodes can be converted to edges and the other way

around, both nodes and edges can be combined, and the alternate representation

does not need to cover the full original graph. Instead of trying to describe it

in words, let’s see how it plays out in use:

As can be seen, the morph syntax both handles multiple graphs, collapsed nodes

and changing edges to nodes, without any change in the mental model of the

operations. All morphing functions are prefixed with `to_*`

for easy discovery

and includes minimum spanning trees, complement graph, dominator tree etc. In

the case where you are interested to continue working with the morphed

representation as a proper `tbl_graph`

you can use the `cystallise()`

verbs that

removes any link to the original graph and returns a tibble with a row per graph

in the morphed representation (as a morph can result in multiple graphs):

## Wrapping it all up

I hope I have given you a small glimpse of what `tidygraph`

is all about. If

working with network data in the past has felt intimidating and strange

`tidygraph`

might feel more at home, but even if you’re a seasoned pro within

network analysis the package should provide a powerful but streamlined interface

to many operations.

### Roadmap

The next goal of my quest to revamp relational data analysis in R will be to

“rebuild” `ggraph`

around `tidygraph`

. This is not to say that the two are

incompatible at the moment — there’s full support through `ggraph`

s support for

`igraph`

— rather I want to only support `tbl_graph`

in the future as all

relevant data structures can be converted to this common format through

`as_tbl_graph()`

.

For `tidygraph`

itself, I have some more ideas I want to explore. Currently

missing from the whole package is any notion of modelling and it will be

interesting to see how this can fit in. Further, I have this wild idea about

providing a `tidygraph`

link to graph databases such as Neo4J in the same way

as `dbplyr`

provides an interface to SQL databases. Lastly, the current focus

has been on supporting the algorithms provided by `igraph`

. While an extensive

package, `igraph`

does not implement everything and there might be stuff lacking

that should be added down the line.

Take care…

**leave a comment**for the author, please follow the link and comment on their blog:

**Data Imaginist - R posts**.

R-bloggers.com offers

**daily e-mail updates**about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...