GET Hackernews Front Page Results using REST API in R

May 17, 2020
By

[This article was first published on r-bloggers on Programming with R, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

Whenever we talk about Data Collection, We usually think about Web Scraping. What’s often forgotten is that a lot of websites / web apps usually offer API to access their data in the right way. This video tutorial explains how you can use httr package to use GET requests (REST API Calls) to collected data from Hacker News, a very popular website for Tech News. The objective of this post it that it can outline how to simple use httr’s GET() to start making REST API calls and also to parse the response object and extract desired data.

Youtube: https://www.youtube.com/watch?v=zlYNx-Tnw3w

Code

library(httr)
library(jsonlite)

url <- "https://hacker-news.firebaseio.com/v0/topstories.json?print=pretty"

response <- httr::GET(url)

top500 <- unlist(content(response))

top20 <- top500[1:20]

titles <- c()
scores <- c()
urls <- c()

for(one in top20){
  story_url <- paste0("https://hacker-news.firebaseio.com/v0/item/",one,".json?print=pretty")
  #print(story_url)
  story_response <- httr::GET(story_url)
  
  titles <- c(titles,content(story_response)$title)
  scores <- c(scores,content(story_response)$score)
  urls <- c(urls,content(story_response)$url)
  
}

hn_top20 <- as.data.frame(cbind(titles,urls,scores))

Related Posts:

To leave a comment for the author, please follow the link and comment on their blog: r-bloggers on Programming with R.

R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.



If you got this far, why not subscribe for updates from the site? Choose your flavor: e-mail, twitter, RSS, or facebook...

Comments are closed.

Search R-bloggers

Sponsors

Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)