Skip to content

OL Analytics API — Architecture & Data Flows

Generated 2026-10-14 15:27 UTC · c4gen dev

ol-analytics-api is a multi-tenant, read-only FastAPI gateway over StarRocks. Each consumer is a tenant: an independent FastAPI sub-app with its own routes, auth model and OpenAPI document, mounted on one root app that owns the shared StarRocks connection pool, Vault credential refresh, health checks and observability. Two tenants were mounted as of October 2026. b2b_dashboard (/api/v1/analytics) serves k-anonymized B2B site-license aggregates (contract utilization, enrollment funnel, engagement trend, program funnel, content engagement, needs-attention) plus a consent-gated learner-progress view (the consent gate fails open in QA and Production until a consent field exists upstream) to MIT Learn's organization dashboard; it trusts the X-Userinfo header APISIX builds from the mit-learn Keycloak session and confirms org-manager status with a service-authenticated round-trip to MITx Online. b2b_learner_records (/api/v1/learner-records) serves identifiable learner, enrollment and course records to contracted partner integrations over Keycloak client-credentials tokens, which the tenant verifies itself against the realm JWKS. Every query reads materialized views that ol-data-platform's dbt builds in the StarRocks b2b_analytics and b2b_learner_records databases from the dimensional Iceberg layer, which a Dagster schedule rebuilds nightly once enabled (it ships stopped). Tenant isolation is application-layer only: all tenants share one StarRocks pool authenticated as the Vault app role. Deployed by ol-infrastructure (applications/ol_analytics_api) to the data EKS clusters, namespace ol-analytics, image mitodl/ol-analytics-api-app, min 2 replicas; APISIX hosts analytics.ol.mit.edu (Keycloak redirect flow, dedicated ol-analytics-api-client) and analytics.learn.mit.edu (shares mit-learn's OIDC session, unauth_action pass), with QA at analytics-qa.ol.mit.edu / analytics.rc.learn.mit.edu.

This is a C4 view of OL Analytics API within the MIT Open Learning SOA, focused on how data is created and propagated — synchronous request paths and asynchronous (queued, scheduled, event-driven) flows alike. Use it for onboarding and as a holistic reference when realigning flows or hunting harmful cycles and fragile linkages.

How to read these diagrams

These are C4 model diagrams (C4-PlantUML). Read them top-down: System Context (the whole SOA) → Container (one system's runtime units) → Dynamic (a single data flow, step by step).

  • People are rounded boxes; systems and containers are rectangles; databases and queues have distinct shapes.
  • Each arrow is a data flow labelled with what moves.
  • Solid arrows are synchronous (request/response, caller blocks).
  • Amber dashed arrows are asynchronous (queued, scheduled, or event-driven — caller does not block).
  • Drag to pan, scroll to zoom. Boxes with a link drill into the next level.

Contents

  1. System Context — OL Analytics API and the systems it exchanges data with.
  2. Containers — the runtime units inside OL Analytics API.
  3. Data Flows — key interactions, step by step (sync & async).
  4. Dependencies & Cycles — graph-derived coupling, cycles, fragile links.
  5. Components — the code-level building blocks inside expanded containers.

Keeping this current

These pages are generated from a structured model by architecture_maps/c4gen. The cross-service edges are extracted deterministically from the witan-code graph; node prose and scenarios are curated. See the generator README to regenerate after the system changes.