More fun with great circle arcs. Who doesn’t love a good great circles graphic? This time, however, it is an animation. Using R, not only have I animated path traversal along great circle arcs, but I’ve also projected them onto a three dimensional global surface. It was not without its challenges. And yes, the title indicates there will be more and better to come.

With the high number of thin, rapidly moving arcs, and color and transparency shifts from frame to frame, this video essentially also serves as a demonstration of ways in which a video can be significantly reduced in quality when re-encoded on upload to sites like YouTube and Vimeo for streaming purposes. A much higher quality source video is available here. It can be downloaded (~100 MB) and viewed locally using a standard video player such as VLC.

You will see right away that more work needs to be done. This is a rough first draft. For one thing, I had to perform some serious data reduction strategies to limit the gross amount of data I was attempting to draw on the globe with rgl. The flat map was similarly restricted only because I intended for the 2D and 3D renditions to match one another. You may notice in the globe rendition, for example, that any given great circle arc is only traversed in a single direction. This might be true for some types of network data, but was not actually the case with mine. Also, the lines are okay for a first draft attempt in the 2D map, but quite bothersome in the 3D map in my opinion. I also had to use semi-transparency in ways I did not want to, that allowed me to “cheat” in the face of resource limitations.

To make matters worse, what you see in the video is of a greatly reduced quality compared to the actual draft video. YouTube (or Vimeo, or any other video streaming site) will re-encode a video on upload in order to compress it significantly so that it is easily streamable on various devices and different connection speeds. Whereas the video I rendered was about 120 MB, what you see on YouTube has been compressed to about 20 MB. I even uploaded a 300+ MB, 400+ MB, and even a 1 GB version, but they were all similarly compressed to about 20 MB. No way around it, or if there is, I have not been able to figure it out. Even on the 1080p high resolution setting, with a small viewing window, it still looks significantly worse than the original video.

This compression is not typically noticeable in videos. However, the very nature of this video, with its high number of rapidly moving, overlapping, thin lines of continuously and slightly varying color and transparency levels, makes it the perfect candidate for being horribly degraded by compression algorithms. The only way to see the higher quality video would be to download the original once I host it somewhere and to watch it using a standard media player such as VLC.

To provide a sense of the what the video quality is prior to upload and re-encoding by YouTube, here are a couple still images taken from the original rendered draft video which you can expand to full screen. Ironically, I compressed these by 50 percent just to make the file sizes smaller for this blog post and they still look great.

Even so, video compression is different. What still cannot be seen here is that in the original video, in addition to having a high quality image, the frame to frame transitions are completely smooth. This is mainly how the uploaded version is degraded, compounding frame to frame assumptions about what is changing in the image and what is worth keeping track of across the frames. The non-moving map image at the opening of the video remains good quality on YouTube. It is once things begin to move that the compression algorithms degrade the video. Here is a screenshot of the 1080p-quality YouTube upload to compare with the above image.

Since posting here is never a priority, my posts often lag behind production. In this case I already know there will be even better stuff forthcoming, as I have solved some of the line plotting problems mentioned up top by now – length, color, speed, semi-transparency, etc. (just not the compression problems), or at least transformed these into lesser problems.

I do not have reproducible code for this because of how complex and data-heavy the process is for generating all these lines and plotting in R. However, when I do eventually share a toy example providing code for generating path sequences, even the toy code will allow you to construct path traversal between two points in a way which would look cleaner than what was used here, that is, cleaner than in the original as shown in the first two higher-quality still images above.