Final Project

Tangible Distributed Systems Simulator

Physical objects for hands-on learning and understanding distributed systems concepts from MIT’s 6.5840 (formerly 6.824) Distributed Systems course.

The project’s aim is to create a tangible simulator where physical blocks representing compute nodes, storage nodes, and network connections work together to demonstrate distributed systems concepts. Each node contains microcontrollers with displays showing live metrics, connected by illuminated wires that visualize data flow step-by-step.

Why

Distributed systems are the backbone of modern cloud infrastructure, but their emergent properties are difficult to understand without significant experience. Concepts like consensus algorithms, replication lag, and network partitions remain abstract and the high-level picture of a live system interaction is not clear simply by looking at code. Diagrams are often used, but they lack the real-time control flow visualizaiton.

What if we could:
physically hold a database replica?
watch data flow through glowing wires?
tip over a node to simulate a crash and see if the system recovers on its own?

Inspiration

This project builds on research in tangible user interfaces and data physicalization:

Project	Contribution
IP Network Design Workbench (MIT Media Lab, 2003)	Physical pucks on sensing table for network topology manipulation
Raft Visualization (thesecretlivesofdata.com)	Canonical digital visualization for consensus algorithms

Diagram

┌─────────────────────────────────────────────────────────────┐
│                    DIGITAL TWIN (Web UI)                     │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────────────┐  │
│  │ Topology    │  │ Time-step   │  │ Distributed Trace   │  │
│  │ View        │  │ Controller  │  │ (Jaeger-style)      │  │
│  └─────────────┘  └─────────────┘  └─────────────────────┘  │
└─────────────────────────────────────────────────────────────┘
                            ▲
                            │ WebSocket
                            ▼
┌─────────────────────────────────────────────────────────────┐
│                 ORCHESTRATOR (Raspberry Pi 4)                │
│  • Simulation engine (controls timing, injects requests)    │
│  • Trace collector                                          │
│  • Network coordinator                                      │
│  • REST API for UI                                          │
└─────────────────────────────────────────────────────────────┘
                            │
            ┌───────────────┼───────────────┐
            │ I2C Bus       │ UART          │ ESP-NOW
            ▼               ▼               ▼
    ┌───────────┐   ┌───────────┐   ┌───────────┐
    │  COMPUTE  │   │  STORAGE  │   │  NETWORK  │
    │   NODE    │───│   NODE    │───│   NODE    │
    └───────────┘   └───────────┘   └───────────┘

Node Types

Compute Nodes (Material: Metal/Blue)

Purpose: Load balancers, application servers, workers, logic executors

Display Views

Golden Signals View

┌────────────────────┐
│ ▓▓▓▓▓░░░  LAT 45ms │
│ ▓▓▓░░░░░  RPS 120  │
│ ▓░░░░░░░  ERR 2.1% │
│ ▓▓▓▓▓▓░░  SAT 78%  │
└────────────────────┘

Connection View

┌────────────────────┐
│    → DB-Primary    │
│    → Cache-01      │
│    ← LB-01         │
│    ↔ Worker-02     │
└────────────────────┘

Live Requests View

┌────────────────────┐
│ GET /api/users     │
│   ├─► db.query()   │
│   └─► cache.get()  │
│ 23ms ✓             │
└────────────────────┘

Storage Nodes (Material: Wood/Green)

Purpose: Databases, caches, key-value stores, file storage

Storage-Specific Displays

Database Mode

┌────────────────────┐
│ PostgreSQL [PRIMARY]│
│ QPS: 1.2k  Conn: 45│
│ Rep Lag: 12ms      │
│ ████████░░ 80% full│
└────────────────────┘

Cache Mode

┌────────────────────┐
│ Redis [REPLICA 2/3]│
│ Hit: 94%  Mem: 2.1G│
│ Keys: 45.2k        │
│ Evictions: 23/min  │
└────────────────────┘

Replication View

┌────────────────────┐
│   ┌───┐            │
│   │ P │ ──► R1 ✓   │
│   └───┘ ──► R2 ✓   │
│          ──► R3 ⚠  │
│     lag: 0, 0, 45ms│
└────────────────────┘

Network Nodes (Material: Clear Acrylic/Orange)

Purpose: Physical representation of network connections

Packet Visualization

Color	Meaning
🔵 Blue	Read request
🟢 Green	Write request
🟡 Yellow	Heartbeat / health check
🔴 Red	Error / timeout
⚪ White	Replication traffic

Packets appear as moving light pulses traveling through the wire, making data flow physically visible.

Time-Step Exploration

A powerful feature for learning is step-by-step execution. Rather than watching requests fly by in real-time, users can pause the simulation and advance one step at a time, watching each operation complete sequentially.

Physical Feedback During Steps

Example: Request Flow Visualization

Step 1: Request arrives at Load Balancer

┌──────┐         ┌──────┐         ┌──────┐
│ LB   │ ← ━━━━━ │      │         │      │
│ ●●●● │   req   │ Srv  │         │ DB   │
└──────┘         └──────┘         └──────┘
   ↑ ACTIVE         dim              dim

Step 2: LB routes to Server

┌──────┐         ┌──────┐         ┌──────┐
│ LB   │ ━━━━━ → │ Srv  │         │ DB   │
│ ○○○○ │   req   │ ●●●● │         │      │
└──────┘         └──────┘         └──────┘
  done           ↑ ACTIVE            dim

Step 3: Server queries Database

┌──────┐         ┌──────┐         ┌──────┐
│ LB   │         │ Srv  │ ━━━━━ → │ DB   │
│      │         │ ◐◐◐◐ │  query  │ ●●●● │
└──────┘         └──────┘         └──────┘
  done           waiting         ↑ ACTIVE

Digital Twin Interface

The web-based companion interface provides:

Simulation Controls

┌─────────────────────────────────────────────────────────────┐
│  ◀◀  ◀  [▶ PLAY]  ▶  ▶▶     Speed: [1x ▼]                 │
│                                                             │
│  Step: 47/312    Time: 00:00:12.450                        │
│  ═══════════════●════════════════════════════════          │
└─────────────────────────────────────────────────────────────┘

Distributed Trace View (Jaeger-style)

┌─────────────────────────────────────────────────────────────────┐
│ Trace: req-7f3a2b   Duration: 127ms   Spans: 5                  │
├─────────────────────────────────────────────────────────────────┤
│ LB-01      ████░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░  3ms       │
│ Worker-02       ██████████████████████████████████████  112ms  │
│  ├─ db.query         ████████████████████████░░░░░░░░░  89ms   │
│  ├─ cache.get                                    ████░  8ms    │
│  └─ transform                                        ██  4ms   │
│ DB-Primary           ████████████████████████░░░░░░░░░  89ms   │
│ Cache-01                                         ████░  8ms    │
│ ──────────────────────────────────────────────────────────────  │
│ 0ms        25ms       50ms       75ms      100ms     127ms      │
└─────────────────────────────────────────────────────────────────┘