Retrieving Reading Levels with R

April 28, 2017
By

(This article was first published on More or Less Numbers, and kindly contributed to R-bloggers)

For those that don’t work in education or aren’t aware, there is a measurement for a child’s reading level called a Lexile ® Level.  There are ways in which this level can be retrieved using different reading assessments.  The measurement can be used to match a child’s reading level to books that are the same level.  So the company maintains a database of books that they have assigned levels.  There are other assessments that measure how well a child is reading, but I’m not aware of any systems that assign books as comprehensively.  The library at the school I work at lacked Lexile Levels for their books.  This is unfortunate because of the fact that teachers are not able provide students with books for their respective Lexile Level.  Fortunately though the library had a list of ISBNs for the books.

On the Lexile website there is a way to search for books using ISBN numbers to retrieve their Lexile ® Level if the book is available in their database.  Entering every ISBN number available is a task fit for something not human.

rvest to the rescue.

Below is the script to retrieve the Lexile Levels of books if a list of ISBNs is available.  This was an incredible time save provided by some R code and hopefully someone else out there could use it.
 

library(rvest) library(httr)
library(htmltools)
library(dplyr)
##Prep for things used later
url<-https://www.lexile.com/fab/results/?keyword=
url2<-https://lexile.com/book/details/
##CSV file with ISBN numbers
dat1<-read.csv(~isbns.csv,header=FALSE)
##dat1<-data.frame(dat1[203:634,])
dat<-as.character(dat1[,1])%>%trimws()
##dat<-dat[41:51]
blank<-as.character(NA)
blank1<-as.character(NA)
##blank2<-as.character(“NA”)
##blank3<-as.character(“NA”)
all<-data.frame(A,B,C)
colnames(all)<-c(name,lexiledat,num)
all<-data.frame(all[1,])
for(i in dat) {
sites<-paste(url,i,sep=)
x <- GET(sites, add_headers(user-agent = r))
webpath<-x$url%>%includeHTML%>%read_html()
##Book Name
name<-webpath%>%html_nodes(xpath=///div[2]/div/div[2]/h4/a)%>%html_text()%>%trimws()
##Lexile Range
lexile<-webpath%>%html_nodes(xpath=///div[2]/div/div[3]/div[1])%>%html_text()%>%trimws()%>%as.character()
##CSS change sometimes
lexiledat<-ifelse(is.na(lexile[2])==TRUE,lexile,lexile[2])
test1<-data.frame(lexiledat,NA)
##Breaks every now and then when adding Author/Pages
##Author Name
##author<-webpath%>%html_nodes(xpath=’///div[2]/div/div[2]/span’)%>%html_text()%>%as.character()%>%trimws()
##author<-sub(“by: “,””,author)
##Pages
##pages<-webpath%>%html_nodes(xpath=’///div[2]/div/div[2]/div/div[1]’)%>%html_text()%>%as.character()%>%trimws()
##pages<-sub(“Pages: “,””,pages)
##Some books not found, this excludes them and replaces with NA values
df<-if(is.na(test1)) data.frame(blank,blank1) else data.frame(name,lexiledat,stringsAsFactors = FALSE)
colnames(df)<-c(name,lexiledat)
df$num <- i
all<-bind_rows(all,df)
}
master<-rbind(all1,all)

   
Link to code

To leave a comment for the author, please follow the link and comment on their blog: More or Less Numbers.

R-bloggers.com offers daily e-mail updates about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...



If you got this far, why not subscribe for updates from the site? Choose your flavor: e-mail, twitter, RSS, or facebook...

Comments are closed.

Sponsors

Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)