Articles by Yitao Li

sparklyr 1.3: Higher-order Functions, Avro and Custom Serializers

July 15, 2020 | Yitao Li

sparklyr 1.3 is now available on CRAN, with the following major new features: Higher-order Functions to easily manipulate arrays and structs Support for Apache Avro, a row-oriented data serialization framework Custom Serialization using R functions to read and write any data format Other Improvements such as compatibility with EMR 6.0 & Spark 3.0, and ...
[Read more...]

sparklyr 1.2: Foreach, Spark 3.0 and Databricks Connect

May 5, 2020 | Yitao Li

A new version of sparklyr is now available on CRAN! In this sparklyr 1.2 release, the following new improvements have emerged into spotlight: A registerDoSpark() method to create a foreach parallel backend powered by Spark that enables hundreds of existing R packages to run in Spark. Support for Databricks Connect, allowing ...
[Read more...]

sparklyr 1.2: Foreach, Spark 3.0 and Databricks Connect

April 20, 2020 | Yitao Li

Behold the glory that is sparklyr 1.2! In this release, the following new hotnesses have emerged into spotlight: A registerDoSpark method to create a foreach parallel backend powered by Spark that enables hundreds of existing R packages to run in Spark. Support for Databricks Connect, allowing sparklyr to connect to remote ...
[Read more...]

Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)