Analyzing Financial Trends: Kalman Filtering for Gold vs Bitcoin

[This article was first published on DataGeeek, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

In their paper “A Synchronized Multi‑IMU Wearable System for Tracking of Joint‑Angles in Sports Motion Analysis” (arXiv:2607.26027v1), Samarasekera and colleagues set out to solve a very practical problem: how to reliably measure joint angles in dynamic sports movements using wearable IMUs. Their goal was to design a synchronized pipeline that could filter out noise, correct drift, and normalize ranges so that biomechanical signals could be trusted in real‑world conditions.

Our goal is different but conceptually parallel: to apply the same pipeline logic to financial time‑series data. Instead of knee flexion angles, we analyze the comparative dynamics of Gold and Bitcoin. The challenge is similar — financial returns are noisy, prone to drift, and difficult to compare directly. By adapting their pipeline, we aim to extract latent trends, mitigate drift, normalize ranges, and highlight relative divergence between assets.

The pipeline consists of four sequential stages:

  1. Indirect Kalman Filtering (IKF) – extracting latent trends from noisy log returns.
    • State equation:

xt=Fxt1+wt,wtN(0,Q)

  • Observation equation:

yt=Hxt+vt,vtN(0,R)

  • IKF separates signal from noise, with Q and R learned via maximum likelihood.
  1. Drift Mitigation via High‑Pass Filtering – removing low‑frequency bias.
    • High‑pass operator:

xtHP=xt1Ni=1Nxti

  • This ensures long‑term drift does not dominate short‑term dynamics.
  1. Range Normalization – mapping filtered values into a comparable scale.
    • Normalization function:

zt=xtHPmin(xHP)max(xHP)min(xHP)100

  • This allows Gold and Bitcoin to be compared on the same 0–100 scale.
  1. Relative Notation (Directional Divergence) – highlighting when assets move in opposite directions.
    • Divergence indicator:

Dt=sign(rtGold)sign(rtBTC)

  • If true, shaded regions show divergence; if false, alignment.

Why This Matters

  • For the biomechanics researchers, the pipeline meant trustworthy joint‑angle tracking.
  • For us, the same architecture means trustworthy comparative trend analysis between financial assets.
  • In both cases, the pipeline’s strength lies in its modularity: IKF for latent signal extraction, high‑pass filtering for drift control, normalization for comparability, and relative notation for divergence detection.

✦ R Packages in Our Pipeline

To make this pipeline reproducible and transparent, we rely on a set of R packages. Each package plays a specific role in the architecture:

  1. tidyverse
    • Purpose: Data manipulation and wrangling.
    • Functions: mutate, rename, inner_join, drop_na.
    • Why: Provides a clean grammar for transforming raw financial data into structured time‑series.
  2. tidyquant
    • Purpose: Financial data acquisition and transformation.
    • Functions: tq_get (download asset prices), tq_transmute (compute log returns).
    • Why: Bridges tidyverse with financial APIs, enabling reproducible market data pipelines.
  3. KFAS
    • Purpose: State‑space modeling and Kalman filtering.
    • Functions: SSModel (define state‑space model), fitSSM (MLE parameter estimation), KFS (Kalman smoothing).
    • Why: Implements the mathematical backbone of IKF, separating latent trends from noise.
  4. zoo
    • Purpose: Rolling window operations.
    • Functions: rollmean.
    • Why: Used for high‑pass filtering by subtracting rolling averages, mitigating drift.
  5. scales
    • Purpose: Normalization and rescaling.
    • Functions: rescale.
    • Why: Maps filtered trends into a 0–100 range, enabling comparability across assets.
  6. ggbraid
    • Purpose: Divergence visualization.
    • Functions: geom_braid.
    • Why: Highlights periods of divergence between Gold and Bitcoin with shaded regions.
  7. ggplot2 (via tidyverse)
    • Purpose: Visualization.
    • Functions: geom_line, labs, theme_classic.
    • Why: Provides the plotting framework for presenting normalized high‑pass Kalman trends.
library(tidyverse)
library(tidyquant)
library(KFAS)
library(ggbraid)

# --- Step 1: Log returns ---
gold <- tq_get("GC=F", from = "2024-01-01") %>%
  tq_transmute(select = close,
               mutate_fun = periodReturn,
               period = "daily",
               type = "log")

btc <- tq_get("BTC-USD", from = "2024-01-01") %>%
  tq_transmute(select = close,
               mutate_fun = periodReturn,
               period = "daily",
               type = "log")

df <- gold %>%
  rename(ret_gold = daily.returns) %>%
  inner_join(btc %>% rename(ret_btc = daily.returns), by="date") %>%
  drop_na()

# --- Step 2: IKF (Kalman filter) ---
model_gold <- SSModel(df$ret_gold ~ SSMtrend(1, Q = list(NA)), H = NA)
model_btc  <- SSModel(df$ret_btc  ~ SSMtrend(1, Q = list(NA)), H = NA)

fit_gold <- fitSSM(model_gold, inits = c(log(var(df$ret_gold)), log(var(df$ret_gold))))
fit_btc  <- fitSSM(model_btc,  inits = c(log(var(df$ret_btc)),  log(var(df$ret_btc))))

kf_gold <- KFS(fit_gold$model, smoothing = c("state"))
kf_btc  <- KFS(fit_btc$model,  smoothing = c("state"))

trend_gold <- as.numeric(kf_gold$alphahat)
trend_btc  <- as.numeric(kf_btc$alphahat)

# --- Step 3: High-pass filter (remove low-frequency drift) ---
hp_gold <- trend_gold - zoo::rollmean(trend_gold, k=30, fill=NA, align="right")
hp_btc  <- trend_btc  - zoo::rollmean(trend_btc,  k=30, fill=NA, align="right")

# --- Step 4: Range normalization ---
gold_norm <- scales::rescale(hp_gold, to=c(0,100))
btc_norm  <- scales::rescale(hp_btc,  to=c(0,100))

# --- Step 5: Relative notation (directional divergence) ---
df_norm <- df %>%
  mutate(
    gold_norm = gold_norm,
    btc_norm  = btc_norm,
    sign_gold = sign(ret_gold),
    sign_btc  = sign(ret_btc),
    divergence = sign_gold != sign_btc
  )

# --- Plot ---
ggplot(df_norm, aes(x=date)) +
  geom_line(aes(y=gold_norm, color="Gold"), linewidth=1.2) +
  geom_line(aes(y=btc_norm, color="Bitcoin"), linewidth=1.2) +
  geom_braid(aes(ymin=gold_norm, ymax=btc_norm, fill=divergence), alpha=0.3) +
  scale_color_manual(values=c("Gold"="#FFD700", "Bitcoin"="#1E3A8A"),
                     name = "") +
  scale_fill_manual(values=c("TRUE"="#FF4C4C", "FALSE"="#87CEFA"),
                    name = "Divergence") +
  scale_y_continuous(limits=c(0,100)) +
  labs(title="Gold vs Bitcoin: Kalman-Filtered High-Pass Trends",
       subtitle="Directional divergence highlighted with shaded regions",
       x="", y="Normalized High-Pass Kalman Trend") +
  theme_classic(base_family = "Roboto Slab") +
  theme(
    legend.position="bottom",
    legend.title=element_text(size=12, face="bold"),
    legend.text=element_text(size=10),
    plot.title=element_text(size=16, face="bold"),
    plot.subtitle=element_text(size=12),
    axis.title=element_text(size=12, face="bold"),
    axis.text=element_text(size=10),
    panel.grid.minor=element_blank()
  )

🔹 Conclusion

By adapting the IKF pipeline from the sports biomechanics paper, we demonstrate how state‑space filtering, drift mitigation, and normalization can reveal comparative dynamics among financial assets. This modular approach is reproducible and extensible to other domains where latent trends and divergence matter.

To leave a comment for the author, please follow the link and comment on their blog: DataGeeek.

R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)