brainR: Put your brain on the Cloud!
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.
In my work, we have come across problems where we wanted to visualize data in 4 dimensions (4D). The data came as brains, which are represented as 3D volumes and we wanted to visualize them at different time points (hence 4D). We were using images that are acquired one image per visit (CT structural scans, particularly), where each patient had a maximum of 10 time points. (The methods below have not been tested or adapted for fMRI scans that have hundreds of time points).
As a result, we created the package brainR
(CRAN package) to fit our needs with this. We strive to make figures such as this 4D example.
This figure can be rotated and zoomed. The slider provides a changing opacity for the overall brain (in order to see the structures inside). The buttons for each “visit” represent enhanced lesions in patients with MS detected by SuBLIME (Sweeney et al. 2013). I feel that these figures can convey results that an orthographic figure such as
cannot, nor can a single slice over time as these enhancing lesions are all over the brain.
The steps to make images such asin brainR
is this:
- Read (neuro)imaging data (usually using
oro.dicom
ororo.nifti
packages). - Have 1) a brain to plot and 2) a region of interest (ROI) in the same space as the brain
- Create a
scene
usingcontour3d
frommisc3d
package (Feng & Tierney, 2008). - Write the contents of this scene into 3D objects of formats
STL
) orOBJ
using wrappers forwriteSTL
orwriteOBJ
from rgl package (Adler et al. 2014). - Use the X ToolKit (XTK) and simple JavaScript controls to visualize these structures and switch between them.
The last step is what I call 4D visualization, being able to move around space with rotation, zooming, and translation while moving through another dimension, usually different time points, ROIs, structures, etc. The reason we went this route is a few reasons, but particularly:
- I know
R
rgl
andmisc3d
already exist- JavaScript is “accessible” and has a lot of docs/support
- Going further into 4D visualization in software is essentially video game programming or using a language I don’t know
Number 4. was interesting as I looked into tools like Blender, Paraview, or CINEMA 4D, which are are highly powerful. Many of them have a GUI-only interface and some used for much visualizing more extensive than (and not adapted so well for) biomedical data, such as full scenes and landscapes. Most solutions, such as those for MATLAB, require the user either have MATLAB programming experience or download software, which we found to be a large and unneccesary burden – so we use JavaScript in a browser. Granted, to run brainR
you need to know R
, but the end user just needs to know how to open a browser. If the end user cannot do that, I don’t think 4D visualization is for them anyway.
Now, let’s rundown some of the functions:
write4D
takes in a “scene”, which is a collection of surfaces, and writes them to an html, along with the STL or OBJ files needed to render it.makeScene
is an adaptation ofcontour3d
frommisc3d
that does mutually exclusive leveling. For example, if you have an image that has values {1, 2, 3}, then you can create an individual surface for 1, one for 2, one for 3. Withcontour3d
, they take the values \(\geq\) to the level for the surface, which means that the surface for 2 would contain 2 and 3.writeWebGL_split
is an adaptation ofrgl
‘swriteWebGL
but cuts the objects into individual objects of only65535
vertices (see here for discussion of WebGL number of vertices, see here for discussion with Duncan Murdoch). This is a limitation of WebGL rendering and we simply split the data using the code given by Duncan Murdoch in the second link. (writeWebGL
has a note in the details, but I missed it). Why is this important? If you don’t do the splitting, your surface will not render properly, if at all.
Overall, this is my first package and the paper is submitted to R Journal. The admins at CRAN were very helpful in getting the package ready for production. It wasn’t easy, but it’s done.
The development version of brainR
is located at this gitHub package.
Note, there is some interesting work done with 3D objects (and some 4D functionality) into PDF recently and there is interesting overview of methods tried located here if you’re interested. We chose the web because it rendered faster and had easier integration with JavaScript.
Citations were done using knitcitations
(Adler et al. 2014).
R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.