Scraping a website with 5 lines of R code

January 24, 2018
By

(This article was first published on Revolutions, and kindly contributed to R-bloggers)

In what is rapidly becoming a series — cool things you can do with R in a tweet — Julia Silge demonstrates scraping the list of members of the US house of representatives on Wikipedia in just 5 R statements:

Since Twitter munges the URL in the third line when you cut-and-paste, here's a plain-text version of Julia's code:

library(rvest)
library(tidyverse)

h <- read_html("https://en.wikipedia.org/wiki/Current_members_of_the_United_States_House_of_Representatives")

reps <- h %>%
 html_node("#mw-content-text > div > table:nth-child(18)") %>%
 html_table()

reps <- reps[,c(1:2,4:9)] %>% as_tibble()

And sure enough, here's what the reps object looks like in the RStudio viewer:

Scrape

As Julia notes it's not perfect, but you're still 95% of the way there to gathering data from a page intended for human rather than computer consumption. Impressive!

To leave a comment for the author, please follow the link and comment on their blog: Revolutions.

R-bloggers.com offers daily e-mail updates about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...



If you got this far, why not subscribe for updates from the site? Choose your flavor: e-mail, twitter, RSS, or facebook...

Comments are closed.

Search R-bloggers

Sponsors

Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)