🔧 RIS Looking Glass Lite - Technical Documentation
🎯 What We Built: A high-performance BGP Looking Glass Lite that consumes live RIS data,
maintains an in-memory routing table, provides REST APIs, and offers real-time topology visualization.
🏗️ System Architecture
graph TB
subgraph "🌐 External Systems"
RIS[RIPE RIS Live
ris-live.ripe.net/v1/ws]
BROWSER[Browser Clients
JavaScript/HTML]
FS[File System
Parquet Snapshots]
end
subgraph "🏗️ RIS Looking Glass Light"
subgraph "📡 Data Ingestion"
CONSUMER[RIS Live Consumer
WebSocket Client]
PROCESSOR[BGP Event Processor
Parse & Validate]
end
subgraph "⚡ Akka Classic System"
ACTOR[Actor System
ris-looking-glass]
SCHEDULER[Scheduler
Periodic Tasks]
end
subgraph "💾 Data Layer"
ROUTING[In-Memory Routing Table
ConcurrentHashMap
Thread-Safe]
SNAPSHOTS[Snapshot Manager
Parquet Format]
end
subgraph "🌐 HTTP Layer"
HTTP[Akka HTTP Server
Port 8080]
ROUTES[REST API Routes
/api/v1/*]
STATIC[Static Assets
HTML/JS/CSS]
end
subgraph "📊 Services"
QUERY[Query Service
Route Lookups]
LIVE[Live Event Service
Server-Sent Events]
MEMORY[Memory Analyzer
Performance Monitoring]
end
subgraph "🎨 Visualizations"
D3[D3.js Topology
SVG Rendering]
SIGMA[Sigma.js Topology
WebGL Rendering]
TERMINAL[Terminal Interface
Text-based UI]
end
end
%% Data Flow
RIS --> CONSUMER
CONSUMER --> PROCESSOR
PROCESSOR --> ROUTING
ROUTING --> QUERY
ROUTING --> LIVE
%% Akka System
ACTOR --> HTTP
SCHEDULER --> SNAPSHOTS
SNAPSHOTS --> FS
FS --> ROUTING
%% HTTP Layer
HTTP --> ROUTES
HTTP --> STATIC
ROUTES --> QUERY
ROUTES --> LIVE
ROUTES --> MEMORY
%% Client Connections
STATIC --> D3
STATIC --> SIGMA
STATIC --> TERMINAL
BROWSER --> HTTP
%% Live Updates
LIVE --> BROWSER
PROCESSOR --> LIVE
%% Styling
classDef external fill:#e1f5fe,stroke:#01579b,stroke-width:2px
classDef pekko fill:#f3e5f5,stroke:#4a148c,stroke-width:2px
classDef data fill:#e8f5e8,stroke:#1b5e20,stroke-width:2px
classDef http fill:#fff3e0,stroke:#e65100,stroke-width:2px
classDef viz fill:#fce4ec,stroke:#880e4f,stroke-width:2px
class RIS,BROWSER,FS external
class ACTOR,SCHEDULER pekko
class ROUTING,SNAPSHOTS data
class HTTP,ROUTES,STATIC http
class D3,SIGMA,TERMINAL viz
🔄 Data Flow Sequence
sequenceDiagram
participant RIS as RIPE RIS Live
participant WS as WebSocket Consumer
participant BGP as BGP Processor
participant RT as Routing Table
participant LE as Live Events
participant UI as Browser UI
participant API as REST API
Note over RIS,UI: Real-time BGP Data Pipeline
RIS->>WS: BGP Update Stream
WS->>BGP: Raw BGP Events
BGP->>RT: Processed Routes
RT->>LE: Route Changes
LE->>UI: Server-Sent Events
UI->>UI: Update Visualization
Note over API,RT: Query Processing
UI->>API: Route Query
API->>RT: Lookup Request
RT->>API: Route Data
API->>UI: JSON Response
Note over RT,LE: Periodic Operations
loop Every 15 minutes
RT->>RT: Create Snapshot
end
loop Every 2 minutes
RT->>LE: Memory Stats
LE->>UI: Statistics Update
end
⚡ Pekko Implementation Details
🎭 Apache Pekko (Akka Fork)
We're using Apache Pekko, the Apache Software Foundation fork of Akka. This provides the same powerful actor model and streaming capabilities with full Apache 2.0 licensing:
🎯 Actor System
ActorSystem("ris-looking-glass")
Classic actor system with implicit execution context
🌐 HTTP Server
Pekko HTTP
Built on top of Pekko Streams, handles REST API and static files
🔄 Concurrency
Futures & ExecutionContext
Async processing with thread-safe data structures
⏰ Scheduling
system.scheduler
Periodic tasks for snapshots and monitoring
🔄 Migration Note: We recently migrated from Akka to Pekko for licensing reasons.
The migration was seamless - only import statements and dependencies changed, with zero impact on functionality or performance.
implicit val system: ActorSystem = ActorSystem("ris-looking-glass")
implicit val ec: ExecutionContext = system.dispatcher
val httpServer = HttpServer(config, queryService, memoryAnalyzer, liveEventService)
system.scheduler.scheduleWithFixedDelay(15.minutes, 15.minutes) { () =>
snapshotManager.createSnapshot()
}
🌊 Data Flow Architecture
flowchart LR
subgraph "📡 Input"
A[RIS Live WebSocket
BGP Updates]
end
subgraph "🔄 Processing"
B[Event Parser
JSON → BGPEvent]
C[Route Processor
Announcements
Withdrawals]
D[Routing Table
In-Memory Store]
end
subgraph "📤 Output"
E[Live Events
Server-Sent Events]
F[REST API
Query Interface]
G[Visualizations
Real-time UI]
end
A --> B
B --> C
C --> D
D --> E
D --> F
E --> G
F --> G
%% Styling
classDef input fill:#e3f2fd,stroke:#1976d2,stroke-width:2px
classDef process fill:#f1f8e9,stroke:#388e3c,stroke-width:2px
classDef output fill:#fce4ec,stroke:#c2185b,stroke-width:2px
class A input
class B,C,D process
class E,F,G output
🔄 Server-Sent Events (Not Polling!)
The Sigma.js visualization uses Server-Sent Events (SSE) for real-time updates:
const eventsSource = new EventSource('/api/v1/live/events');
eventsSource.addEventListener('bgp-event', function(e) {
const event = JSON.parse(e.data);
processBGPEvent(event);
});
def liveEvents: Route = {
path("events") {
get {
complete {
Source.actorRef[BGPEvent](bufferSize = 1000, OverflowStrategy.dropHead)
.map(event => ServerSentEvent(event.toJson.toString, "bgp-event"))
.keepAlive(30.seconds, () => ServerSentEvent.heartbeat)
}
}
}
}
🚀 Why SSE over WebSockets?
• Simpler implementation for one-way data flow
• Automatic reconnection handling
• Works through firewalls and proxies
• Built-in browser support with EventSource API
🔌 API Endpoints
📊 Query APIs
GET /api/v1/routes/prefix/{prefix} - Route lookup by prefix
GET /api/v1/routes/asn/{asn} - Routes originated by ASN
GET /api/v1/routes/peer/{peer} - Routes from specific peer
GET /api/v1/stats - Routing table statistics
📡 Live Data APIs
GET /api/v1/live/events - Server-Sent Events stream of BGP updates
GET /api/v1/live/stats - Real-time statistics stream
💾 Snapshot APIs
POST /api/v1/snapshots/create - Create new snapshot
GET /api/v1/snapshots/list - List available snapshots
POST /api/v1/snapshots/bootstrap - Bootstrap from snapshot
🎨 Visualization Architecture
🎯 Two Visualization Approaches
| Feature |
D3.js Version |
Sigma.js Version |
| Rendering |
SVG/Canvas |
WebGL (Hardware Accelerated) |
| Max Nodes |
~500 (performance limit) |
~2000+ (optimized for large graphs) |
| Layout Algorithm |
D3 Force Simulation |
Custom Adaptive Physics |
| Data Updates |
Every 10 BGP events |
Real-time with batching |
| Interactivity |
Standard zoom/pan |
Hardware-accelerated interactions |
| Memory Usage |
Higher (DOM nodes) |
Lower (WebGL buffers) |
🧠 Smart Filtering System (BGPPlay-Inspired)
🎯 Prefix Filtering
Focus on specific IP prefixes
?prefix=8.8.8.0/24&prefix-only=true
🏢 AS Type Classification
Tier-1, Content, Regional, Customer
?tier1=true&content=true
⏱️ Time Windows
1h, 6h, 24h, 7d filtering
?time=1h&highlight=true
🔄 Deduplication
Remove identical AS paths
?dedupe=true
⚖️ vs RIPE Looking Glass
🔍 Key Differences
| Aspect |
RIPE Looking Glass |
Our Implementation |
| Data Source |
Historical RIS data + Live updates |
Pure live RIS stream + Snapshots |
| Duplicates |
Shows historical entries (multiple timestamps) |
Current state only (latest wins) |
| Performance |
Database queries |
In-memory hash maps |
| Real-time |
Periodic updates |
Live streaming (SSE) |
| Visualization |
Static tables/text |
Interactive network topology |
| Scalability |
Distributed database |
Single-node, high-performance |
🎯 Why No Duplicates in Our System:
RIPE's Looking Glass shows historical data with timestamps, so you see the evolution of routes over time.
Our system maintains only the current state - when a new BGP announcement arrives for a prefix,
it replaces the previous one. This gives you a clean, current view of the Internet routing table.
🔧 Technical Implementation Details
📊 Data Structures
// Thread-safe routing table
class RoutingTable {
private val routes = new ConcurrentHashMap[String, BGPRoute]()
private val peerStats = new ConcurrentHashMap[String, PeerStatistics]()
private val asnRoutes = new ConcurrentHashMap[Int, Set[String]]()
}
// BGP Event Processing
case class BGPEvent(
eventType: String, // "announcement" or "withdrawal"
prefix: String, // "8.8.8.0/24"
asPath: String, // "3356 15169"
nextHop: String, // "192.168.1.1"
peer: String, // "rrc00-peer-123"
timestamp: Long // Unix timestamp
)
🚀 Performance Optimizations
⚡ Memory Management
• ConcurrentHashMap for thread safety
• Periodic memory analysis
• Configurable heap size recommendations
📦 Snapshot System
• Parquet format for compression
• Periodic snapshots (15 min)
• Bootstrap from largest/latest
🌊 Stream Processing
• Akka Streams for backpressure
• Batched updates to visualization
• Configurable buffer sizes
🎯 Smart Filtering
• Client-side filtering for responsiveness
• Server-side event filtering
• URL parameter state management
🚀 Getting Started
📋 Bootstrap Options
sbt "run --bootstrap-latest"
sbt "run --bootstrap-largest"
sbt run
🌐 Access Points
http://localhost:8080/ - Terminal-style interface
http://localhost:8080/static/bgp-topology.html - D3.js visualization
http://localhost:8080/static/bgp-topology-sigma.html - Sigma.js visualization
http://localhost:8080/static/tech-docs.html - This documentation
🎯 System Status
Core System: Fully operational
RIS Live Consumer: Active with retry logic
HTTP API: All endpoints functional
Visualizations: Both D3.js and Sigma.js working
Snapshot System: Automated backups active
ForceAtlas2: Optional (adaptive physics fallback)
🎉 Congratulations! You now have a production-ready BGP Looking Glass with real-time
topology visualization, comprehensive APIs, and performance monitoring. The system is designed to
handle the full Internet routing table (~900k routes) with sub-second query response times.
🔧 RIS Looking Glass Lite - Built with Scala, Akka, and modern web technologies
Real-time BGP data processing and visualization