Downloading 24-Hour Weather Updates in R

September 27, 2018
By

[This article was first published on George J. Mount, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

This post serves as a follow-up on a previous post about scheduled collection of Weather.gov’s XML feed in R, which itself was a follow-up to retrieving real-time data from Weather.gov in Excel.

Reflecting on the best way to accomplishing this automation, I noticed something back on Weather.gov’s update pagean option for a 2-day weather history! Duh! Why automate collection every hour when I could use this link to get history every day (or more)?

Turns out this link brings you to an html page with a table recording weather updates on the hour with lots of information — more that the XML page, in fact!

For this I will use R’s htmltab package to read this table into an R table, then do some manipulation before getting it to our workbook.

Let’s get started. First time using R? Check out my free course, “5 Things Excel Users Should Know About R.”

1. Inspect the table

To figure out how to pull this table into R, we need to look under the hood of our website. To do that, right-click somewhere in the table in your browser and click “select element.”

2. Copy the table’s XPath

Here an editor comes up on your page. Notice that when you hover over different parts of this script, different parts of the web page are highlighted. Keep hovering until you see that the table we want to download is highlighted. We need to get some information about this table to write a script to collect it.

Once you have the table highlighted, right-click on this line of code and select Copy – XPath.

We will be using this in the R Script below.

3. Assemble R Script

This script will save the weather information for the past 24 hours as a .csv file based on today’s date. I read in the web page, point to the html table based on the XPath which we identified above, keep the first 24 rows for the first 24 hours (aka, today) of the weather data, and save the file as today’s date.

4. Schedule it to run

For this again you could use the Windows Task Scheduler, setting the script to run every day at midnight. Check out the previous post for more on the Task Scheduler.

So there you have it, a daily download of hourly weather readings from any recording site of the National Weather Service delivered directly to you via the power of R.

To leave a comment for the author, please follow the link and comment on their blog: George J. Mount.

R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.



If you got this far, why not subscribe for updates from the site? Choose your flavor: e-mail, twitter, RSS, or facebook...

Comments are closed.

Search R-bloggers

Sponsors

Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)