Operator Notation for Data Transforms

March 25, 2019
By

(This article was first published on R – Win-Vector Blog, and kindly contributed to R-bloggers)

As of cdata version 1.0.8 cdata implements an operator notation for data transform.

The idea is simple, yet powerful.

First let’s start with some data.

d <- wrapr::build_frame(
  "id", "measure", "value" |
    1   , "AUC"    , 0.7     |
    1   , "R2"     , 0.4     |
    2   , "AUC"    , 0.8     |
    2   , "R2"     , 0.5     )

knitr::kable(d)
id measure value
1 AUC 0.7
1 R2 0.4
2 AUC 0.8
2 R2 0.5

In the above data we have two measurements each for two individuals (individuals identified by the "id" column). Using cdata‘s new_record_spec() method we can capture a description of this record structure.

library("cdata")

record_spec <- new_record_spec(
  wrapr::build_frame(
    "measure", "value" |
    "AUC"    , "AUC" |
    "R2"     , "R2"  ),
  recordKeys = "id")

print(record_spec)
## $controlTable
##   measure value
## 1     AUC   AUC
## 2      R2    R2
## 
## $recordKeys
## [1] "id"
## 
## $controlTableKeys
## [1] "measure"
## 
## attr(,"class")
## [1] "cdata_record_spec"

Once we have this specification we can transform the data using operator notation.

We can collect the record blocks into rows by a "division" (or aggregation/projection) step.

knitr::kable(d)
id measure value
1 AUC 0.7
1 R2 0.4
2 AUC 0.8
2 R2 0.5
d2 <- d %//% record_spec

knitr::kable(d2)
id AUC R2
1 0.7 0.4
2 0.8 0.5

We can expand record rows into blocks by a "multiplication" (or join) step.

knitr::kable(d2)
id AUC R2
1 0.7 0.4
2 0.8 0.5
d3 <- d2 %**% record_spec

knitr::kable(d3)
id measure value
1 AUC 0.7
1 R2 0.4
2 AUC 0.8
2 R2 0.5

And that is truly fluid data manipulation.

This article can be found in a vignette here.

To leave a comment for the author, please follow the link and comment on their blog: R – Win-Vector Blog.

R-bloggers.com offers daily e-mail updates about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...



If you got this far, why not subscribe for updates from the site? Choose your flavor: e-mail, twitter, RSS, or facebook...

Comments are closed.

Search R-bloggers

Sponsors

Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)