Now that the election is over, you may want to use polling data in a model of the campaign. Simon Jackman has thoughtfully made his daily state-by-state predictions available for download, but a commonly-used dataset is the RealClearPolitics polling a... [Read more...]

FastRWeb is an infrastructure that allows any webserver to use R scripts for generating dynamic content, such as web pages and graphics. In this post, you’ll learn how to install and set up FastRWeb on a Mac. This tutorial is expendable to any Unix-like operating system. It is an ... [Read more...]

[Read more...]

[Read more...]

Kay Cichini recently wrote a word-cloud R function called GScholarScraper on his blog which when given a search string will scrape the associated search results returned by Google Scholar, across pages, and then produce a word-cloud visualisation. This was of interest to me because around the same time I posted ... [Read more...]

This is a followup to a post I uploaded earlier today about web scraping data off Google Scholar. In that post I was frustrated because I’m not smart enough to use xpathSApply to get the kind of results I wanted. However fast-forward to the evening whilst having dinner with ... [Read more...]

I wanted to scrape the information returned by a Google Scholar web search into an R data frame as a quick XPath exercise. The following will successfully extract  the ‘title’, ‘url’ , ‘publication’ and ‘description’.  If any of these fields are not available, as in the case of a citation, the ... [Read more...]

Google slightly changed the html code it uses for hyperlinks on search pages last Thursday, thus causing one of my scripts to stop working. Thankfully, this is easily solved in R thanks to the XML package and the power and simplicity of XPath expressions: Lovely jubbly! P.S. I know ... [Read more...]

[Read more...]

The intersection of mapping APIs, fast database operations and user engagement offers a lot of very cool crowdsourcing applications ranging from the benign and powerful (Google’s Person Finder) to the minor and questionable (A DUI checkpoints app). Most intriguing in … Continue reading → [Read more...]

ables are pretty common in web pages as data sources, and the most direct way to get these data is probably to copy and paste. This is OK if there are only two or three tables, and when we need to grab 5000 tables in 1000 web pages, we may not really ... [Read more...]

[Read more...]

[Read more...]

[Read more...]

[Read more...]

[Read more...]

[Read more...]

[Read more...]

[Read more...]

[Read more...]