One of R’s biggest pitfalls is that eats up memory without letting it go.  This can be a huge problem if you are running really big jobs, have a lot of tasks  to run, or there are multiple users on your local computer or r server.  When I run ... [Read more...]

Percentage change of median price per square foot from July 2012 to July 2013: Percentage change of median price from July 2012 to July 2013: Last November I made a  choropleth of median rental prices in the San Francisco Bay Area using data from my company, Kwelia.  I have wanted to figure out how ... [Read more...]

I have asked by a few people lately to help walk them through using twitter API in R, and I’ve always just directed them to the blog post I wrote last year during the US presidential debates not knowing that Twitter had changed a few things. Having my interest ... [Read more...]

I came up with the following function to tap into the FourSquare trending venues API: library("RCurl", "RJSONIO")   foursquare [Read more...]

Unfortunately the RpostgreSQL package (I’m pretty sure other SQL DBs as well) doesn’t have a provision to UPDATE multiple records (say a whole data.frame) at once or allow placeholders making the UPDATE a one row at a time ordeal, so I built a work around hack to ... [Read more...]

Before heading home for the holidays, I had a large data set (1.6 GB with over 1.25 million rows) with columns of text and integers ripped out of the company (Kwelia) Database and put into a .csv file since I was going to be offline a lot over the break. I tried ... [Read more...]

My company, Kwelia, is sitting on mountains of data, so I decided to try my hand at mapping.  I have played around with JGR but it’s just too buggy, at least on my mac, so I went looking for other alternatives and found a good write up here.  I ... [Read more...]

I’ve been interested in building a web app using R for a while, but never put any time into it until I was informed of the Shiny package.  It looked too easy, so I absolutely had to try it out. First you need to install the package from the ... [Read more...]

The following data was collected during the Presidential Debate on the 22nd of October by tapping into the Facebook social graph API using R. The top three posted links during the debate for each candidate are: Obama- #1     http://bit.ly/QCODJg #2     http://bit.ly/RXstnm #3    http://bit.ly/P8MmJ1 ... [Read more...]

The following are word clouds of tweets for each candidate from the October 16, 2012 debate with the bigger words the more often they were used in tweets (click on each word cloud to enlarge): And the net-negative posts for each candidate: Please note that the bigger the word is in the ... [Read more...]

Click on above graph to enlarge. Background The data for this graph was collected automatically every ~60 seconds of the VP debate on 10/11/2012, with an ending aggregate sample size of 363,163 tweets.  From this dataset duplicate tweets were removed (because of bots), which gave a final dataset of 81,124 remaining unique tweets (52,303-Biden, 28,821... [Read more...]

After watching the Presidential debates and hearing all the opinions on how the candidates performed, I got the hair brained idea of creating a simple function that would do automate the pulling down of tweets for each candidate, analyze the positivity or negativity of tweets, and then graph them out. ... [Read more...]

For a while now I have been contemplating pulling data from our postgreSQL db directly from R, but just never actually pulled the trigger until today.  What I found was that it was a lot easier than I ever could have imagined.  My laptop was already on the VPN, so ... [Read more...]

For a recent project I had to do some mapping of addresses, but I didn’t have there lat/lons do use the Deducer and DeducerSpatial packages in R JGR.  After frustrating myself trying to adapt this code from stackoverflow.com, I found a much easier way of geocoding using ... [Read more...]

I was recently confronted with a problem where I had to compare two very different data sets. The problem was that one data set was observed cross sectional data with dates over the course of three months and the other was weekly averages during those same three months.  After a ... [Read more...]