Some Code for Dumping Data from Twitter Gardenhose

March 30, 2010

(This article was first published on Byte Mining, and kindly contributed to R-bloggers)

Gardenhose is a Streaming API feed that continuously sends a sample (roughly 15% according to Ryan Sarver at the 140tc in September 2009) of all tweets to feed recipients. This is some code for dumping the tweets to files named by date and hour. It is in PHP which is not my favorite language, but works nonetheless. I received a few requests to post it, so here it is.


$username = '';
$password = '';

while(true) {
         $file = fopen("http://" . $username . ":" . $password . "","r");

         while($data = fgets($file))
             $time = @date("YmdH");
             if ($newTime!=$time)
                 $file2 = fopen("{$time}.txt","a");
             $newTime = $time;
        //need to close the file, but only if it is open!
        try {
        } catch (MyException $e) {}
        try {
        catch (MyException $e) {}


To leave a comment for the author, please follow the link and comment on their blog: Byte Mining. offers daily e-mail updates about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...

If you got this far, why not subscribe for updates from the site? Choose your flavor: e-mail, twitter, RSS, or facebook...

Tags: ,

Comments are closed.


Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)