Quick Hit: Handling Cocoa Core Data Timestamps in R

[This article was first published on R – rud.is, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

For the first time ever we got a new riding mower this weekend. We’ve always haggled to keep the one sellers were using with any given house we’ve purchased over the years (that was big enough for a yard that “requires” a riding mower).

We ended up getting a model from John Deere and the manual (yes, I actually read the manual) noted they had a Mower Plus app to track “things”. Given the lack of built-in sensors I figured this was just something kitchy but it’s not bad (they don’t do anything super-evil tracker-wise) and if you use it while mowing it tracks anything your mobile device can and will notify you for prescribed service events.

I ran the app through a local proxy server before using it in the field and it only phoned home to check serial number and get weather info, which meant there was no hidden API to capture and mimic to get access to mowing data. But, that meant it store[ds] everything it needs locally to the app. Since I use an iOS mobile device (you should too, Android is security mess) I figured that meant it stored everything in on-device SQLite databases.

A quick review of an immediate backup confirmed that this posit was correct and I started going crazy (as you’ll see in the aforelinked Twitter thread).

However, it had these cray cray timestamps:

$ ZTIMESTAMP          <dbl> 581100271, 581100272, 581100270, 581100281, 581100290, 581100328, 581100308, 581…

Thanks to some fairly fast poking by @dabdine, it turns out these are Apple Cocoa Core Data timestamps. They can come in two flavors:

  • seconds since 2001-01-01 00:00:00
  • nanoseconds since ^^

The entire reason for this post (and, some of the needless verbosity) was to get this into the internet’s long-ish term memory stores (Google/Internet Archive) so others can reference it as well.

If you come across one of these beasts (for 2019 dates they’ll likely start with either 56, 57, 58, 59 or 60 — enumerating them to make Google searches more able to pick up this post). 2020 dates would start with 60, 61, 62, or 63 (and that should be sufficient for savvy searchers). But, if you’re not having luck using Excel date tricks, {lubridate} functions or {anytime} try transforming a sample timestamp with:

as.POSIXct(sample_timestamp, origin = "2001-01-01")

to see if you get a more “expected”/realistic value and know you’ve got a Core Data timestamp. You can even stick:

from_coredata_ts <- function(x, tz = NULL) {
  .POSIXct(ifelse(
    test = floor(log10(x)) >= 10, # If you're still using R in 2317 then good on ya and edit this
    yes = as.POSIXct(x/10e8, origin = "2001-01-01"), # nanoseconds coredata
    no = as.POSIXct(x, origin = "2001-01-01") # seconds coredata
  ), tz = tz)
}

into your RStudio snippets file or in a personal “misc” package for quick-access to converting both kinds of Core Data timestamps into proper R POSIXct objects.

To leave a comment for the author, please follow the link and comment on their blog: R – rud.is.

R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)