TL;DR

I wanted something that goes some way to automating my own
time-consuming process of scrolling twitter for cool things to read. I
thought a few other people might feel somewhat similar, so I decided to
build a feed.

R posts you might have missed is an
semi-automated twitter account posting recent R-related content. The
goal is to make it easier to keep up with the most important packages
and news from the community. Links to relevant and popular resources are
gathered from twitter and the R blogosphere before being processed and
lightly curated.

Read on to learn the origin story of the account, how it works and what
comes next!

News overload!

Keeping track of new developments in the data science, open source and R
communities is hard. The number of active developers, application areas
and R packages is exploding. Ever since I started writing R code I’ve
found it hard to avoid reinventing solutions to problems that are
already solved by other developers, usually through ignorance of those
developments. Being up-to-date with recent developments equips you with
options that can change the way you approach a new problem.

This is more or less the reason I still use twitter, because it’s still
the place where a majority of R developers hang out and share their
projects and ideas. The problem is that the volume of new stuff is just
too large – and I could easily spend endless hours per week scrolling
twitter, discovering and re-discovering new stuff (and getting very
distracted in the process). This is compounded by twitter’s news feed
algorithm which I think has made it even harder to develop a tailored
feed. So what can you do?

Well you’ve got options of course. R
Bloggers has been around for some time and
aggregates the feeds of several hundred well-known R blogs. I’ve never
found this solves my problem: blog articles are one type of content, but
there are many other types of content that I’d like to see in the same
place, and most of them do not have RSS feeds. The site itself carries a
lot of banner ads and doesn’t render articles very nicely – although
those may be minor considerations if you still use an RSS reader to
access the posts.

Ok so what else? R-Weekly is a terrific
resource. The team gather links to posts, packages, community news and
tweets into a single weekly digest. I think R Weekly is a wonderful
resource, and I still read it every week – it does a particularly good
job of creating a nicely formatted list broken into content types and
topics that were active in the last week. However, this doesn’t scratch
my itch completely. One issue is that it’s not totally automated (AFAIK,
please correct me if that’s false), and there’s always the risk that
something gets excluded. Additionally – any news oriented resource
focusses on what’s occurred most recently (of course, yeah I know) and
by definition excludes older useful resources that keep resurfacing. I
think it’s good for those things to continue to get air-time –
particularly because if I’m not working on a specific topic at the time
of the initial news announcement, I’ll probably forget about it. Or more
likely I just missed the announcement to begin with. I think repeated
exposure and reminders can be important.

Long story short, I wanted something that goes some way to automating my
own time-consuming process of scrolling twitter for cool things to read.
I thought a few other people might feel somewhat similar, so I decided
to build a feed.

R posts you might have missed

R posts you might have missed is a twitter feed with the following
attributes:

• Publishes about 10 posts per day
• Posts are usually blog posts, repos and tutorials containing R code
• Emphasis on non-commercial content that is free to access
• Lightly curated with a lean towards more recent posts and repos
• Ensure the author is directly credited in each post

How does it work?

The recipe underpinning the feed takes the following steps:

1. Gather links from #rstats twitter

• Use Michael Kearney’s rtweet
  package to gather recent #rstats tagged tweets from twitter (last 9
  days)
• Also use rtweet to gather
  tweets from a subset of highly active users – not all of these are
  necessarily #rstats tagged
• Extract the urls embedded inside the tweets

2. Gather new post urls from RSS feeds

• Use Robert Myles McDonnell’s
  tidyRSS package to read
  a large number of RSS
  feeds.
• Extract the urls of posts published in the last week that include
  code chunks

3. Read and filter urls based on content

• Steps 1. and 2. usually result in around 2000 urls per week. Use
  htmldf to download
  page content from the urls.
• Filter out any pages that don’t have code tags in the source and
  that haven’t already been tweeted by R posts you might have missed
  recently.
• Filter out any commercial content, anything that looks spammy. This
  uses some simple lists of sites to exclude. Medium posts are also
  completely filtered out – Medium paywalls it’s content, and also
  tends to have lower quality content in general.
• Extract page titles from ,