Analyzing Roam Research Attribute Tables with Python

[This article was first published on R on technistema, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

By: Brad Lindblad

LinkedIn | Github | Blog | Subscribe

Image Source: roamresearch.com

Roam Research is a revolutionary note-taking tool for networked thought. As a data scientist and reader, I take many notes on many different topics. I’ve tried many different note-taking apps from Evernote and OneNote to the bare bones of Simple Note. I’ve always bumped into a problem with these tools, which is that the design and format of the tool restricted how I took notes.

For example, with Evernote you are forced to more or less put an idea on a single card, with little freedom in relating that note to other notes in your Evernote corpus. The search function allowed you to find specific words, but each idea is and was indefinitely separated from the others.

The folders and tagging provide some semblance of structure, but eventually your work or life will change such that you’ll want to rearrange that structure. Not fun.

The human mind doesn’t work that way. Studies have shown that the neurological structure of the brain forms an incomprehensibly complex network that modern machine learning barely intimates.

Why Roam?

Roam allows you to sputter ideas without having to worry about which folder to place them in, or if an idea could fit into multiple folders. You don’t have to pick just one location for that idea to languish in. For instance, if you have a nice python code snippet for working in Databricks that you’d like to save for later, you don’t have to worry about whether to place it in your databricks snippet folder or your python snippet folder; you simply tag the code with both and it will appear in both. An idea can live in two places concurrently, no sweat.

This is huge for allowing you to simply take the note and trust the system to organize for you. Roam has allowed me to consolidate the following activities and functions under a single tool: – Code snippets and cheats – Commonplace book – Bible study – Short-form writing (like this article) – Data science lab notebook – Goal setting and tracking – And, for the purposes of this article, habit tracking.

Habit tracking in Roam

I wanted to track the arthritis in one of my hands along with a few other variables to look for any indication of a relationship. There are many tools and apps that are made for this very thing, but my goal is to do as much as I can in Roam.

We use a feature in Roam called attribute tables to accomplish this. This article on Reddit does a great job of explaining how to set up attribute tables, so check that out if you’ve never made one before. The output of a habit tracking table looks like this:

If you were to look under the hood at the table, you would find that it looks an awful lot like an html table. The Pandas python library has a nice little function for parsing simple html tables called read_html(), and don’t ya know it parses this Roam attribute table real slick.

The python script

The best way I found to parse the table was to download the actual html page with your browsers download function; in Brave it’s as simple as right-clicking on the page, hitting Save as > Complete Webpage, and saving to a location. I like to have a daily page open which usually just has one table. If you have multiple tables, you will have to modify the last line of the script a bit to select it.

After that, this little python script reads your Roam attribute table into a pandas dataframe:

import pandas as pd

# download daily page html to local
FILE = "/home/brad/Desktop/July 12th, 2021.html"

html = pd.read_html(FILE)

df = html[0]

and gives us:

Now you can do any analysis on your habits that you wish, all within the comforts of Roam and python.


Want more content like this?

Subscribe here

To leave a comment for the author, please follow the link and comment on their blog: R on technistema.

R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)