Create a Web Crawler in R

February 8, 2011

(This article was first published on Brock's Data Adventure » R, and kindly contributed to R-bloggers)

Admittedly I am not the best R coder, and I certainly have a lot to learn, but the code at the link below should provide you with an example of how easy it is to create a very (repeat: very) basic web crawler in R.  If you wanted to do this in SPSS, and I have, , you would have to step outside of the normal syntax language and leverage the Python plugin.

I simply use a function to do my data manipulation, and access that function within a loop before saving out a CSV of the dataset.  Granted this only works when you know the structure of the URLs that you want to scrape, but this is pretty easy and hopefully straightforward.

The code is located here.

Filed under: Fantasy HOckey, R, Tutorial Tagged: NHL, R, Web Mining

To leave a comment for the author, please follow the link and comment on their blog: Brock's Data Adventure » R. offers daily e-mail updates about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...

If you got this far, why not subscribe for updates from the site? Choose your flavor: e-mail, twitter, RSS, or facebook...

Tags: , , , ,

Comments are closed.


Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)