Analysing Shiny App start-up Times with Google Lighthouse

[This article was first published on The Jumping Rivers Blog, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

This is part one of a three part series on Lighthouse for Shiny Apps.

  • Part 1: Using Google Lighthouse for Web Pages
  • Part 2: Analysing Shiny App start-up Times with Google Lighthouse (This post)
  • Part 3: Effect of Shiny Widgets with Google Lighthouse (Coming soon)


In the last blog I spoke about using Google Lighthouse to test the speed of web pages. I wanted to build upon that and use Lighthouse to test some Shiny apps.

To get a feel for Shiny’s performance in a Lighthouse analysis, I needed a lot of shiny apps that I could test and create a dataset from, so I used the entries to the 2021 Shiny app contest, which is a competition where people enter Shiny apps to be judged on technical merit and artistic achievement. I used the 2021 apps as there has unfortunately not been a competition since. A full list of the submissions can be found on the Posit Community website.

To actually obtain data from these apps I used Google Lighthouse in the same way I described for general web pages in the previous blog in this series. This generated a Lighthouse report for each app.

Google Lighthouse

To test a singular app from the contest it was exactly the same as testing a normal webpage, I simply ran:

lighthouse --output json --output-path data/output_file.json url

Where url is the app I’m testing. You can also test in browser using devtools (as demonstrated in the last blog), but I was testing a lot of apps so I needed to do it programmatically.

Before we get into the data it’s important to point out that Google Lighthouse scores do vary; you may run a report on an app that I’ve covered and get a different score. There are a number of reasons for this covered here, so the devs recommend running multiple tests. I’d also like to point out I have only run the report once for each app due to length of time it would take to run reports on all the apps a few times.

Do you require help building a Shiny app? Would you like someone to take over the maintenance burden? If so, check out our Shiny and Dash services.

App data

The entries to the 2021 Shiny app contest were great! Loads of unique and interesting apps, given it was 2021 there were plenty of COVID- and election-related apps. I ran Lighthouse reports locally on 268 of the Shiny app contest submissions (some of the links were broken), and have compiled a few plots to summarise the performance of the apps.

Below is a histogram showing the distribution of overall performance scores for the apps. The Lighthouse docs give the following advice for apps based on performance scores:

  • 90-100 is an app with good performance;
  • 50-89 is an app that needs some improvement;
  • and 0-49 is an app with poor performance.

As we can see many of the apps (79 / 268) have good performance, whereas the bulk of the apps are in need of some improvement (149 / 268) or have poor overall performance (40 / 268).

Distribution of overall scores for the apps, with the mean score of 73.2 marked.

We can dive deeper into the distribution of the raw values that are used in calculating the overall performance score. The performance score is a weighted sum of some metrics formed from these raw (time) values – see the previous blog for more details on what each of these metrics means. The scores follow a similar trend – most of the measurements fall on the faster side of the spectrum then decrease as the time increases. I think this was to be expected based on the distribution of the performance score seen earlier, as most of the apps scored pretty well.

App metric scores for six different metrics: Cumulative Layout Shift, First Contentful Paint, Largest Contentful Paint, Speed Index, Time to Interactive, Total Blocking Time. The plots are histograms of the time taken to each stage.

The Apps

I don’t highlight any of the apps on the lower end of the performance spectrum here, as it would be unfair on the creators. The Shiny app contest has two sections one for < 1 year’s experience and another for > 1 years experience, so people new to Shiny are likely to have been experimenting with what’s possible in an app and not focusing on performance.

That being said I’d like to reiterate what Colin Fay said in his talk “Destroy All Widgets” about being sensible with widget use within apps, and understanding that they can hinder performance and increase wait times when they are not always necessary. For instance do you really need a {plotly} plot or would a ggplot suffice? The same could be said about interactive data tables.

I will highlight a couple of high scoring apps:

This app by Rabii Bouhestine is a really cool Geoguessr-esque game where you are trying to pinpoint the location of world wonders. This app received the overall score from Lighthouse of 95!

WorldGuesser app screenshot

Another high scoring app is “Mix Things Up” by Sam Parmar a previous competition winner who was a judge on the 2021 contest. This app is a simple yet efficient way to generate random work outs.

Exercise app screenshot

The last one I’m going to highlight is this app by Edgar Cáceres, which is an app for visualising air quality data from the station in La Oroya, Junin, Peru. This app is particularly impressive in it’s scores as it actually has two interactive {leaflet} plots.


Google Lighthouse is a good starting point for testing the start-up times of your apps, however it is worth noting that the score can be misleading. An app may score very highly but not actually load fast for the user. This may happen, for example, if Lighthouse thinks that the contentful paints have loaded when it was the background for the app. A way to check this is looking at the screenshots of the Google Lighthouse report within the browser. You can do this by adding --view after the url argument when running a test in the terminal. I will be using the next blog in this series to investigate this further.

So if you are developing a desktop Shiny app and want see see how it does you can use Lighthouse and this blog for a benchmark, although with a pinch of salt as there are many different kinds of apps that we tested – games and data visualations etc. Roughly, however, if your app scores better than 73 then that’s a good start. If you can’t bring your app load time down for whatever reason, maybe due data processing for example, then something you can do is use a loading screen to let your app-users know that something is happening. This is covered excellently at the start of this blog on Shiny extensions.

In the final blog in this series, we will be investigating the impact various widgets have on Shiny app Lighthouse scores.

For updates and revisions to this article, see the original post

To leave a comment for the author, please follow the link and comment on their blog: The Jumping Rivers Blog. offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)