Mini AI app using TensorFlow and Shiny

January 14, 2016

(This article was first published on Opiate for the masses, and kindly contributed to R-bloggers)


Simple image recognition app using TensorFlow and Shiny



My weekend was full of deep learning and AI programming so as a milestone I made a simple image recognition app that:

  • Takes an image input uploaded to Shiny UI
  • Performs image recognition using TensorFlow
  • Plots detected objects and scores in wordcloud


This app is to demonstrate powerful image recognition functionality using TensorFlow following the first half of this tutorial.
In the backend a pretrained is running, with the model being pretrained by
This Python file takes a jpg/jpeg file as an input and performs image classifications.

I will then use R to handle the classification results and produce wordcloud based on detected objects and their scores.


The app is based on R (shiny and wordcloud packages), Python 2.7 (tensorflow, six and numpy packages) and TensorFlow (Tensorflow itself and this python file).
Please make sure that you have all the above packages installed. For help installing TensorFlow this link should be helpful.


Just like a usual Shiny app, you only need two components; server.R and ui.R in it.
This is optional but you can change number of objects in the image recognition output by changing the line 63 of'num_top_predictions', 5#I changed this to 10,
                            """Display this many predictions.""")


I put comments on almost every line in server.R so you can follow the logic more easily.

shinyServer(function(input, output) {
    PYTHONPATH <- "path/to/your/python"  #should look like /Users/yourname/anaconda/bin if you use anaconda python distribution in OS X
    CLASSIFYIMAGEPATH <- "path/to/your/" #should look like ~/anaconda/lib/python2.7/site-packages/tensorflow/models/image/imagenet
    outputtext <- reactive({
      ###This is to compose image recognition template###
      inFile <- input$file1 #This creates input button that enables image upload
      template <- paste0(PYTHONPATH,"/python ",CLASSIFYIMAGEPATH,"/") #Template to run image recognition using Python
      if (is.null(inFile))
        {res <- system(paste0(template," --image_file /tmp/imagenet/cropped_panda.jpg"),intern=T)} else { #Initially the app classifies cropped_panda.jpg, if you download the model data to a different directory, you should change /tmp/imagenet to the location you use. 
      res <- system(paste0(template," --image_file ",inFile$datapath),intern=T) #Uploaded image will be used for classification
    output$plot <- renderPlot({
      ###This is to create wordcloud based on image recognition results###
      df <- data.frame(gsub(" *\(.*?\) *", "", outputtext()),gsub("[^0-9.]", "", outputtext())) #Make a dataframe using detected objects and scores
      names(df) <- c("Object","Score") #Set column names
      df$Object <- as.character(df$Object) #Convert df$Object to character
      df$Score <- as.numeric(as.character(df$Score)) #Convert df$Score to numeric
      s <- strsplit(as.character(df$Object), ',') #Split rows by comma to separate rows
      df <- data.frame(Object=unlist(s), Score=rep(df$Score, sapply(s, FUN=length))) #Allocate scores to split words
      # By separating long categories into shorter terms, we can avoid "could not be fit on page. It will not be plotted" warning as much as possible
      wordcloud(df$Object, df$Score, scale=c(4,2),
                    colors=brewer.pal(6, "RdBu"),random.order=F) #Make wordcloud
    output$outputImage <- renderImage({
      ###This is to plot uploaded image###
      if (is.null(input$file1)){
        outfile <- "/tmp/imagenet/cropped_panda.jpg"
        contentType <- "image/jpg"
        #Panda image is the default
        outfile <- input$file1$datapath
        contentType <- input$file1$type
        #Uploaded file otherwise
      list(src = outfile,
    }, deleteFile = TRUE)


The ui.R file is rather simple:

  fluidPage(titlePanel("Simple Image Recognition App using TensorFlow and Shiny"),
                     fileInput('file1', '',accept = c('.jpg','.jpeg')),

Shiny App

That’s it!
Here is a checklist to run the app without an error.

  • Make sure you have all the requirements installed
  • You have server.R and ui.R in the same folder
  • Optionally, change num_top_predictions in
  • Upload images should be in jpg/jpeg format

I was personally impressed with what machine finds in abstract paintings or modern art πŸ˜‰


The full codes are available on github.

Mini AI app using TensorFlow and Shiny was originally published by Kirill Pomogajko at Opiate for the masses on January 15, 2016.

To leave a comment for the author, please follow the link and comment on their blog: Opiate for the masses. offers daily e-mail updates about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...

If you got this far, why not subscribe for updates from the site? Choose your flavor: e-mail, twitter, RSS, or facebook...

Comments are closed.


Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)