Back to Library
gcp
Famous Companies
advanced
Social media feed at global scale

Twitter / X Feed Architecture

Twitter / X

Twitter's feed architecture solves one of the hardest problems in distributed systems: delivering 500M+ tweets per day to personalized timelines in real-time. Built on Google Cloud, the system uses a hybrid fan-out strategy — fan-out on write for regular users (pre-computing timelines into Memorystore Redis caches) and fan-out on read for celebrity accounts with millions of followers to avoid write amplification. Recommended for engineers designing social platforms where millions of users consume personalized content feeds in real time.

Data Flow

Tweet API
Tweet Service
Fan-out Service
Media Storage
Timeline Service
Event Stream
Timeline Cache (Memorystore)
Tweets Store (Bigtable)
Search Indexer

Share this architecture with your network

Service Breakdown (9 services)

Other9 services
Tweet API
  • •Routes API traffic and enforces policies
  • •Manages authentication and rate limiting
  • •Provides a unified API endpoint
Tweet Service
  • •Processes tweet creation, storage, and retrieval
  • •Validates content and extracts hashtags and mentions
  • •Serves tweet data to timeline and search services
Fan-out Service
  • •Pushes new tweets to follower timelines asynchronously
  • •Handles high-follower fan-out with batched writes
  • •Prioritizes active users for immediate delivery
Timeline Service
  • •Assembles personalized timelines from cached entries
  • •Merges pre-computed and on-demand feed items
  • •Ranks tweets by recency and engagement signals
Timeline Cache (Memorystore)
  • •Stores pre-computed timelines for active users
  • •Serves timeline reads with sub-millisecond latency
  • •Evicts stale entries based on TTL and access patterns
Tweets Store (Bigtable)
  • •Stores tweet content and metadata in wide-column format
  • •Optimized for high-throughput sequential reads
  • •Scales horizontally across regions automatically
Event Stream
  • •Streams tweet events to downstream consumers
  • •Enables real-time analytics and notification triggers
  • •Buffers events for reliable at-least-once delivery
Media Storage
  • •Stores uploaded images and video attachments
  • •Serves media via CDN for low-latency delivery
  • •Applies content policies before making media available
Search Indexer
  • •Runs event-driven code without servers
  • •Scales instantly from zero to peak load
  • •Cost-effective for sporadic workloads

Scaling Strategy

Twitter's hybrid fan-out approach is key to scaling. Regular tweets fan out on write to follower timeline caches in Memorystore, while celebrity tweets (10K+ followers) fan out on read at query time. Pub/Sub handles event streaming for analytics and search indexing. The timeline cache uses consistent hashing across Memorystore shards, and Bigtable stores the canonical tweet data optimized for time-series access patterns.

Related Architectures

Netflix Streaming Platform

Netflix

Global video streaming architecture with CDN, microservices, and chaos engineering patterns. Handles 200M+ concurrent streams.

advanced
Famous Companies

Uber Dispatch System

Uber

Real-time ride dispatch with geospatial matching, surge pricing, and event-driven driver allocation at massive scale.

advanced
Famous Companies

Stripe Payment Processing

Stripe

PCI-compliant payment processing with fraud detection, webhook delivery, and multi-currency support.

advanced
Famous Companies

WhatsApp Messaging System

WhatsApp / Meta

End-to-end encrypted real-time messaging with presence tracking, message queuing for offline users, and media delivery.

advanced
Famous Companies
A

Airbnb Marketplace Platform

Airbnb

Two-sided marketplace with search, booking, payments, and review systems. Handles real-time availability, pricing, and fraud prevention.

intermediate
Famous Companies

Slack Real-Time Collaboration

Slack

Workspace messaging with persistent channels, real-time WebSocket delivery, search indexing, and app integrations.

intermediate
Famous Companies