Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

Continue to dig tweets. After we reviewed how to count positive, negative and neutral tweets in previous post, I discovered another great idea. Suppose positive or negative is not enough and we want to understand the rate of positivity or negativity. For example, “good” in tweet has 4 points rating, but “perfect” has 6. Thus, we can try to measure the rate of satisfaction or opinion in tweets and take chart with trend:

So, we need other dictionary for managing this task, dictionary with rating of words. We can create it or find results of great research of affective ratings (e.g. here).

And of course, our algorithm should bypass Twitter’s API limitation via accumulating historical data. This approach was described in previous post.

Note, I use average rating of evaluated words which I find in tweet. For example, if we found “good” (4 points) and “perfect” (6 points) in one tweet, it would be evaluated as (4+6)/2=5. This is better than use total sum in case of all words in dictionary have positive rating. For example, one “good” (4 points) should be better than three “bad” (1,5 points each). This solves via average value.

Let’s get started. We need to create Twitter Application (https://apps.twitter.com/) for connecting to Twitter’s API. Then we get Consumer Key and Consumer Secret. And finally, our code in R:

#connect all libraries
library(ROAuth)
library(plyr)
library(dplyr)
library(stringr)
library(ggplot2)
#connect to API
consumerKey <- '____________' #put the Consumer Key from Twitter Application
consumerSecret <- '______________'  #put the Consumer Secret from Twitter Application
Cred <- OAuthFactory$new(consumerKey=consumerKey, consumerSecret=consumerSecret, requestURL=reqURL, accessURL=accessURL, authURL=authURL) Cred$handshake(cainfo = system.file('CurlSSL', 'cacert.pem', package = 'RCurl')) #There is URL in Console. You need to go to it, get code and enter it on Console
load('twitter authentication.Rdata') #Once you launch the code first time, you can start from this line in the future (libraries should be connected)
#the function of tweets accessing and analyzing
search <- function(searchterm)
{
#access tweets and create cumulative file
df <- twListToDF(list)
df <- df[, order(names(df))]
df$created <- strftime(df$created, '%Y-%m-%d')
if (file.exists(paste(searchterm, '_stack_val.csv'))==FALSE) write.csv(df, file=paste(searchterm, '_stack_val.csv'), row.names=F)
#merge last access with cumulative file and remove duplicates
stack <- rbind(stack, df)
stack <- subset(stack, !duplicated(stack$text)) write.csv(stack, file=paste(searchterm, '_stack_val.csv'), row.names=F) #evaluation tweets function score.sentiment <- function(sentences, valence, .progress='none') { require(plyr) require(stringr) scores <- laply(sentences, function(sentence, valence){ sentence <- gsub('[[:punct:]]', '', sentence) #cleaning tweets sentence <- gsub('[[:cntrl:]]', '', sentence) #cleaning tweets sentence <- gsub('\d+', '', sentence) #cleaning tweets sentence <- tolower(sentence) #cleaning tweets word.list <- str_split(sentence, '\s+') #separating words words <- unlist(word.list) val.matches <- match(words, valence$Word) #find words from tweet in "Word" column of dictionary
val.match <- valence$Rating[val.matches] #evaluating words which were found (suppose rating is in "Rating" column of dictionary). val.match <- na.omit(val.match) val.match <- as.numeric(val.match) score <- sum(val.match)/length(val.match) #rating of tweet (average value of evaluated words) return(score) }, valence, .progress=.progress) scores.df <- data.frame(score=scores, text=sentences) #save results to the data frame return(scores.df) } valence <- read.csv('dictionary.csv', sep=',' , header=TRUE) #load dictionary from .csv file Dataset <- stack Dataset$text <- as.factor(Dataset$text) scores <- score.sentiment(Dataset$text, valence, .progress='text') #start score function
write.csv(scores, file=paste(searchterm, '_scores_val.csv'), row.names=TRUE) #save evaluation results into the file
#modify evaluation
stat <- scores
stat$created <- stack$created
stat$created <- as.Date(stat$created)
stat <- na.omit(stat) #delete unvalued tweets
write.csv(stat, file=paste(searchterm, '_opin_val.csv'), row.names=TRUE)
#create chart
ggplot(stat, aes(created, score)) + geom_point(size=1) +
stat_summary(fun.data = 'mean_cl_normal', mult = 1, geom = 'smooth') +
ggtitle(searchterm)
ggsave(file=paste(searchterm, '_plot_val.jpeg'))
}
search("______") #enter keyword

Finally, we get 4 files:

• cumulative file of all initial data,
• draft file with tweets rating,
• cleaned (without unvalued tweets) file with tweets and dates,
• and chart where we can see density of tweets ratings and mean as a trend, it looks like: