◈Async Job Processing Fundamentals

Learn the fundamentals of asynchronous job processing and message queues

◉Caching

Cache strategies, invalidation, CDNs, Redis patterns, eviction policies, stampede prevention.

◈Consistent hashing

Distribute data across nodes with minimal reshuffling on changes.

◈Databases & storage

SQL vs NoSQL, sharding, replication, indexing, ACID, CAP theorem.

◉Load balancing

L4 vs L7, algorithms (round-robin, least-connections), health checks, sticky sessions.

◈Message queues

Kafka, pub/sub, fan-out, dead-letter queues, backpressure.

◉Synchronous vs Asynchronous Communication

Request/response vs fire-and-forget patterns and their latency implications.

◉REST API Design Principles

Resource naming, HTTP verbs, pagination, versioning, and request/response shape.

◉WebSockets vs SSE vs Polling

Full-duplex, server-push, and client-pull communication patterns compared.

◉Event-Driven Architecture

Events, commands, producers, consumers, and event schema design.

◉Vertical vs Horizontal Scaling

Scaling up vs scaling out, stateless vs stateful services.

◉Back-of-Envelope Estimation

Step-by-step estimation framework: users → RPS → storage → total.

◉Stateless Service Design

Why stateless services scale better and how to move state to Redis or a database.

◉Failure Modes in Distributed Systems

Network partitions, crashes, slow dependencies, and timeout design.

◉Exponential Backoff and Retry

Retry logic, exponential backoff formula, jitter, and thundering herd prevention.

◉Circuit Breakers

CLOSED → OPEN → HALF-OPEN state machine for fast-failing broken dependencies.

◉Idempotency

Idempotency keys, server-side check-lock-process-cache pattern, and TTL design.

◉Graceful Degradation and Fallbacks

Serving stale data, fallback content, and feature flags as degradation controls.

◉Schema Design Fundamentals

Tables, columns, data types, primary keys, and constraints in relational databases.

◉Relationships and Foreign Keys

One-to-many, many-to-many, foreign key constraints, and junction tables.

◉Database Constraints as Data Guards

CHECK, UNIQUE, and composite constraints that enforce data integrity at the DB level.

◉Database Indexes: Making Queries Fast

B-tree indexes, composite indexes, partial indexes, and the cost of over-indexing.

◉Append-Only Audit Tables

Why audit logs must never be updated and how to partition them for performance.

◉Read Replicas and Write Scaling

Primary for writes, replicas for reads, replication lag, and eventual consistency.

◉Redis Deep Dive

Strings, sorted sets, LRU eviction, rate limiting pipelines, and leaderboard patterns.

◉CDN Caching

Edge caching, Cache-Control headers, CDN purging, and when CDN beats Redis.

◉Client-Side Caching

React Query stale-while-revalidate, in-memory LRU cache, and localStorage persistence.

◈When to Use NoSQL

Document stores, key-value stores, and choosing between Postgres and Firestore.

◈Time-Series and Columnar Databases

ClickHouse and BigQuery for analytics, columnar storage, and date partitioning.

◈Object Storage (S3-Compatible)

Blob storage, content-addressable storage, lifecycle rules, and presigned URLs.

◈IndexedDB for Browser-Side Storage

Object stores, indexes, transactions, and using IndexedDB as a local-first data layer.

◈Message Queue Fundamentals

Producer, consumer, broker, message lifecycle, and backpressure handling.

◈Kafka Deep Dive

Topics, partitions, offsets, consumer groups, log retention, and Kafka vs SQS.

◈SQS and Managed Queues

Visibility timeout, long polling, FIFO queues, and redrive policy configuration.

◈Dead-Letter Queues

Why messages fail, DLQ configuration, re-drive workflow, and alerting on DLQ depth.

◈At-Least-Once vs Exactly-Once Delivery

Why at-least-once is the practical default and how to make consumers idempotent.

◈Worker Pool Pattern

Infinite dequeue loop, worker sizing by job type, and graceful SIGTERM shutdown.

◈Queue-per-Channel Pattern

Isolating failure, independent scaling, and different retry policies per channel.

◈Scheduled Jobs and Cron

Cron syntax, idempotent crons, and distributed cron for single-instance guarantees.

◈Async Processing Pipelines

Multi-stage pipelines, fan-out, and fan-in aggregation from parallel workers.

◈Authentication Fundamentals

Cookie sessions, JWT tokens, OAuth 2.0, and secure token storage patterns.

◈Authorisation and RBAC

Role-based access control, row-level security, and server-side permission validation.

◈Payment Integration Fundamentals

Stripe PaymentIntents, PCI compliance, payment webhooks, and subscription modelling.

◈Distributed Payment Architecture

Atomic debits, optimistic vs pessimistic locking, and offline payment resilience.

◈Provider Adapter Pattern

Common interface over third-party APIs with per-provider adapters and automatic failover.

◈Third-Party API Resilience

Circuit breakers, caching for outages, data gap detection, and multi-provider fan-out.

◈Webhooks: Receiving External Events

Endpoint design, HMAC signature validation, idempotent handlers, and retry handling.

◈Rate Limiting Algorithms

Token bucket, fixed window, sliding window, leaky bucket, and Redis implementation.

◈Rate Limiting in Practice

Per-user, per-client, per-provider, and per-channel limits and their enforcement.

◈WebSocket Architecture

Connection lifecycle, shared vs per-component sockets, and scaling with pub/sub fanout.

◈Server-Sent Events (SSE)

Unidirectional server push, HTTP/2 multiplexing, and reconnection with Last-Event-ID.

◈Real-Time State Management

Delta updates, normalised stores, optimistic updates, and staging incoming changes.

◈Presence and Awareness

Typing indicators, live cursors, online/offline status, and Yjs Awareness API.

◈Adaptive Bitrate Streaming (HLS/DASH)

Learn about Adaptive Bitrate Streaming and its key components

◈Audio Streaming Architecture

CDN delivery, progressive vs chunked audio, and gapless playback implementation.

◈Video Processing Pipelines

Upload → transcode → multi-resolution → CDN pipeline with async job queues.

◉Frontend Architecture Patterns

Container/presenter split, Context vs global store, and when to lift or push state.

◉State Management with Zustand

Normalised store shape, slices, selectors, and avoiding unnecessary re-renders.

◉Server State with React Query

Stale-while-revalidate, infinite scroll with useInfiniteQuery, optimistic mutations.

◈State Machines on the Frontend

XState vs useReducer, modelling multi-step flows, and preventing impossible UI states.

◉Virtualisation and Large Lists

Why DOM can't hold 10k nodes, react-window vs tanstack-virtual, and scroll restoration.

◉Lazy Loading Images and Media

Intersection Observer, loading=lazy, LQIP blur-up, and responsive srcset/sizes.

◉Infinite Scroll and Pagination

Cursor vs offset pagination, sentinel element, pre-fetching, and load-more fallback.

◈Web Workers for Heavy Computation

Main vs worker thread, Comlink RPC over postMessage, and Transferable objects.

◈Canvas and WebGL Rendering

Canvas 2D render loop, viewport culling, camera as ref, and RBush spatial indexing.

◈Service Workers

Install/activate/fetch lifecycle, Cache API strategies, and update handling.

◈Offline-First Architecture

IndexedDB as primary store, sync queue, Background Sync API, and conflict resolution.

◈Cross-Tab Synchronisation

BroadcastChannel API, storage event fallback, Web Locks leader election.

◈Intersection Observer API

Shared observer instance, rootMargin pre-loading, and autoplay/lazy-load use cases.

◈Geolocation and Permissions API

Checking permission state, getCurrentPosition vs watchPosition, and GPS fallbacks.

◈File API and File System Access API

Blob, FileReader, drag-and-drop DataTransfer, and showOpenFilePicker.

◈Web Crypto API

AES-GCM symmetric encryption, RSA-OAEP key wrapping, PBKDF2 key derivation.

◈Web Audio API

AudioContext graph, BiquadFilterNode EQ, crossfade with GainNode, and visualiser with AnalyserNode.

◈WebRTC

RTCPeerConnection, SDP offer/answer, ICE, getUserMedia, and mesh vs SFU topology.

◈Conflict-Free Replicated Data Types (CRDTs)

The concurrent edit problem, CRDT vs OT comparison, and Yjs Y.Text/Y.Array/Y.Map.

◈Yjs in Practice

y-websocket sync protocol, Awareness API, offline buffering, and snapshot persistence.

◈Operational Transform (OT)

How OT works, why it needs a server to order operations, and OT vs CRDT tradeoffs.

◈Last-Write-Wins Conflict Resolution

Highest timestamp wins, when LWW causes data loss, and server vs client timestamps.

◈Three-Way Merge

Base + local + remote = merged result, conflict detection, and user-prompted resolution.

◈Delta Sync

Sending only what changed, version vectors for change tracking, and reducing sync payload size.

◈A/B Testing Systems

A/B testing evaluates user responses to different versions of a webpage or app

◈Analytics Pipelines

A comprehensive guide to understanding analytics pipelines and their components

◈Logging and Observability

Structured logging, RED metrics, distributed tracing, OpenTelemetry, and tiered retention.

◈Geospatial Queries

Haversine formula, PostGIS proximity search, and bounding box pre-filter optimisation.

◈IP-Based Localisation

GeoIP databases, CDN edge resolution, fallback chains, and VPN/proxy handling.

◈Real-Time Location Tracking

GPS update frequency, rAF lerp for smooth pins, noise filtering, and camera auto-fit.

◈Password Security

PBKDF2/bcrypt/Argon2id, HaveIBeenPwned k-anonymity, NIST 800-63B rules, and lockout.

◈End-to-End Encryption in the Browser

PBKDF2 → AES-GCM key derivation, per-note data keys, and RSA-OAEP key sharing.

◈File Security and Deduplication

SHA-256 content-addressable storage, reference counting, and convergent encryption.

◈Edge Computing and CDN Architecture

Moving computation to the edge, Cloudflare Workers, and edge vs origin tradeoffs.

◈Microcontroller and Embedded Constraints

RAM/flash/CPU limits, RTOS, flash wear levelling, watchdog timer, and OTA updates.

◈Offline-Capable Hardware Systems

Offline policy design, idempotent transaction replays, and regional auth clusters.

◈Drag-and-Drop Architecture

HTML5 DnD API vs @dnd-kit, ghost preview, snap-to-grid, and collision detection.

◈Grid Layout and Widget Systems

react-grid-layout onLayoutChange, widget registry, undo/redo history, and autosave.