As part of the series on development of early career researchers in the lab, we spent three sessions over three weeks learning the basics of R. In my book “The Digital Cell”, I advocate R as the main number-crunching software but the R literacy in my lab is actually quite ...

This post will explore the mathematics behind information gain. We’ll start with the base intuition behind information gain, but then explain why it has the calculation that it does. What is information gain? Information gain is a measure frequently used in decision trees to determine which variable to split ...

To compute Lasso regression, define the soft-thresholding functionThe R function would be soft_thresholding = function(x,a){ sign(x) * pmax(abs(x)-a,0) } To solve our optimization problem, set so that the optimization problem can be written, equivalently hence and one gets or, if we develop Again, if there are ...

R Markdown: what, why and how? Before you start Components of a .Rmd file YAML header Code chunks Text Code inside text Highlight text like it is code Images Tables Additional notes and useful resources If you have spent some time writing cod...

# Re-load data previously stored for purposes of this blog post
pharma <- 
  fread("~/Desktop/David/Projects/xbrl_investment/data/pharma_inc.csv")

Introduction In A Walk Though of Accessing Financial Statements with XBRL in R - Part 1, we went through the first steps of pulling XBRL data for a single company from Edgar into R. Although an improvement over manual plugging of numbers into a Excel, there is still a way ...

Introduction As financial professionals and analytic software lovers, the ability to efficiently load a large number of financial statements, and conduct an analysis has always been a key objective. In previous posts, Redwall Analytics worked with a 15-year time series of municipal Comprehensive Annual Financial Reports (CAFR) for 15 Fairfield County, ...

The software environment R is widely used for data analysis and data visualization in the social sciences and beyond. Additionally, it is becoming increasingly popular as a tool for data and file management. Focusing on these latter aspects, this Methods Bites Tutorial by Marcel Neunhoeffer, Oliver Rittmann and our team ...

Last week I published a screencast demonstrating how to use the tidymodels framework and specifically the recipes package. Today, I’m using this week’s #TidyTuesday dataset on food consumption around the world to show hyperparameter tuning! Here is the code I used in the video, for those who prefer ...

rstudio::conf 2020 is already receding in the rear view mirror, but the wealth of resources generated by the conference will be valuable for quite some time. All of the materials from the workshops, and now all one hundred and four videos of conference talks are available. This unique video collection ...

R Markdown: what, why and how? Before you start Components of a .Rmd file YAML header Code chunks Text Code inside text Images Tables Additional notes and useful resources Photo by Jon Tyson If you have spent some time writing code in R, you probably have heard of generating dynamic ...

The software environment R is widely used for data analysis and data visualization in the social sciences and beyond. Additionally, it is becoming increasingly popular as a tool for data and file management. Focusing on these latter aspects, this Methods Bites Tutorial by Marcel Neunhoeffer, Oliver Rittmann and our team ...

I am always saying that simulation can help illuminate interesting statistical concepts or ideas. The design effect that underlies much of clustered analysis is could benefit from a little exploration through simulation. I’ve written about clustered-related methods so much on this blog that I won’t provide links - ...

A few months ago, I posted a note with some home made codes for quantile regression… there was something odd on the output, but it was because there was a (small) mathematical problem in my equation. So since I should teach those tomorrow, let me fix them. Median Consider a ...

When evaluating a machine learning model if the same data is used to train and test the model this results in overfitting. So the model performs much better in predictive ability  than it would if it was applied on completely new data, this is because the model uses random noise ...

Commonly, the final product that a data scientist or a statistician generates is a report, usually in MS Word format. The officer package enables generating such a report from within R. It also enables generating PowerPoint presentations, but this is… Continue reading →