Extracting EOD Data from NSE

[This article was first published on My Paper Trades, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

My prime interest being the Indian financial markets, the first step would be to get the data to play around. NSE India provides EOD of data as bhavcopies. The same are stored as zipped files at their servers. Downloading them one by one for a larger time frame will be very tedious, hence I will attempt to automate the process.

There is a great tool for statistical computing called R. It is open-source with a lot of development being done across various packages. This interests me a lot because of it simplicity and power. I would make my attempt of automating bhavcopy downloads using this software. If you want to try the same, you can visit the downloads section of R-Project and get the latest version


Objective: Download Bhavcopy (Equity) from http://www.nseindia.com and save only relevant columns Date, Symbol, Open, High, Low, Close, Last and Volume.

To download the Bhavcopy (Equity) from http://www.bseindia.com refer to this post.


Here is the R Code for the same

#Define Working Directory, where files would be saved
setwd('D:/')
 
#Define start and end dates, and convert them into date format
startDate = as.Date("2010-12-25", order="ymd")
endDate = as.Date("2011-01-05", order="ymd")
 
#work with date, month, year for which data has to be extracted
myDate = startDate
zippedFile <- tempfile() 
 
while (myDate <= endDate){
  filenameDate = paste(as.character(myDate, "%y%m%d"), ".csv", sep = "")
 monthfilename=paste(as.character(myDate, "%y%m"),".csv", sep = "")
 downloadfilename=paste("cm", toupper(as.character(myDate, "%d%b%Y")), "bhav.csv", sep = "")
 temp =""
 
  #Generate URL
 myURL = paste("http://nseindia.com/content/historical/EQUITIES/", as.character(myDate, "%Y"), "/", toupper(as.character(myDate, "%b")), "/", downloadfilename, ".zip", sep = "")
 
  #retrieve Zipped file
  tryCatch({
  #Download Zipped File
  download.file(myURL,zippedFile, quiet=TRUE, mode="wb")
 
  #Unzip file and save it in temp 
  temp <- read.csv(unzip(zippedFile), sep = ",") 
 
  #Rename Columns Volume and Date
  colnames(temp)[9] <- "VOLUME"
  colnames(temp)[11] <- "DATE"
 
  #Define Date format
  temp$DATE <- as.Date(temp$DATE, format="%d-%b-%Y")
 
  #Reorder Columns and Select relevant columns
  temp<-subset(temp,select=c("DATE","SYMBOL","OPEN","HIGH","LOW","CLOSE","LAST","VOLUME"))
 
  #Write the BHAVCOPY csv - datewise
  write.csv(temp,file=filenameDate,row.names = FALSE)
 
  #Write the csv in Monthly file
  if (file.exists(monthfilename))
  {
   write.table(temp,file=monthfilename,sep=",", eol="\n", row.names = FALSE, col.names = FALSE, append=TRUE)
  }else
  {
   write.table(temp,file=monthfilename,sep=",", eol="\n", row.names = FALSE, col.names = TRUE, append=FALSE)
  }
 
  #Write the file Symbol wise
 
 
  #Print Progress
  #print(paste (myDate, "-Done!", endDate-myDate, "left"))
 }, error=function(err){
  #print(paste(myDate, "-No Record"))
 }
 )
  myDate <- myDate+1
  #print(paste(myDate, "Next Record"))
}
 
 #Delete temp file - Bhavcopy
 junk <- dir(pattern="cm")
 file.remove(junk)

To leave a comment for the author, please follow the link and comment on their blog: My Paper Trades.

R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)