A ride-sharing app has separate services for user management, trip tracking, and payments. What is this an example of?

Multiple independent services communicating together form a distributed system where each service can fail independently.

What is the key danger of a network partition in a distributed system?

The ambiguity is dangerous: a node cannot distinguish between crash, network failure, or slowness, making it hard to respond correctly.

Which of these is NOT a typical cause of network partitions?

High CPU usage affects performance but does not partition the network. Partitions are caused by communication failures, not resource constraints.

If a payment service crashes after deducting money but before sending a confirmation email, what is the risk?

Without idempotency, the user seeing no confirmation might retry, and the restarted service would process the payment again.

What does idempotency prevent in distributed systems?

Idempotency ensures that retrying an operation (due to crash or timeout) does not cause duplicate side effects like double-charging.

A Byzantine failure is characterized by:

Byzantine failures are when a process continues running but behaves incorrectly—much harder to detect than crashes.

Why is a slow dependency sometimes more dangerous than a crashed service?

Threads block waiting for slow responses. With 100 threads all waiting 30 seconds, new requests have no threads available, even though the server itself is healthy.

Your API server has 100 threads. A database becomes very slow (30 seconds per query). What happens?

All 100 threads become stuck waiting. Without timeouts, new requests have nowhere to go and are rejected.

When should you use fail-fast?

Fail-fast is appropriate when proceeding with wrong data is worse than stopping immediately, especially for config and data integrity.

A payment service should fail-fast if:

Missing config means the service cannot run correctly. Fail-fast immediately rather than proceed with broken setup.

What is the key principle of fail-safe design?

Fail-safe means choosing safe defaults like DENY for access control instead of ALLOW when you cannot verify.

A feature flag service goes down. What is the fail-safe behavior?

Defaulting new features OFF is safe. Defaulting ON risks releasing untested code. Fail-safe always chooses the conservative default.

Your news website's recommendation engine fails. With graceful degradation, what should happen?

Graceful degradation keeps core functionality (the article) working while non-critical features (recommendations) fail silently.

Which feature should NOT use graceful degradation?

Payment processing is core functionality; it should fail completely (fail-fast) rather than silently degrade, to avoid charging errors.

What does a timeout do?

A timeout is a time limit; once exceeded, the client stops waiting and handles the failure.

What does a connection timeout measure?

Connection timeout is about establishing the connection. If it fails, the server is unreachable, down, or partitioned.

What does a read timeout measure?

Read timeout applies AFTER connection. If it triggers, the server is slow or stuck processing the request.

Your API call succeeds in connecting but receives no response for 25 seconds. Which timeout triggers?

Connection already succeeded. Waiting for response data is a read timeout situation.

A write timeout is most relevant when:

Write timeouts protect against slowness while sending data, important for large uploads.

In Python requests library, what does `timeout=(3, 10)` mean?

The tuple format is (connection_timeout, read_timeout) in the requests library.

In JavaScript fetch with AbortController, what does calling `controller.abort()` do?

AbortController.abort() immediately terminates the fetch, typically called by a timeout.

What is a good connection timeout value and why?

Connection should be quick. If you cannot connect in 5 seconds, something is very wrong. Long timeouts just waste time.

A simple key-value lookup should have approximately what read timeout?

Simple operations need short timeouts (1-2s). Complex queries need more time; exports can take minutes.

Which operation should have the longest read timeout?

Data exports involve processing and can take minutes. Simple reads should timeout in 1-2 seconds.

What happens when you make an HTTP request with no timeout set and the server never responds?

Without a timeout, the thread blocks indefinitely. Eventually all threads are stuck and the service becomes unresponsive.

If all threads in a server are stuck waiting on no-timeout requests, what is the result?

With no threads available, the server cannot accept new requests. It appears completely down even though it is technically running.

Which approach is best for critical data integrity?

For data integrity, fail-fast immediately stops processing if errors occur, preventing data corruption.

A traffic light controller loses power. What is the safest behavior?

Flashing red is fail-safe: it defaults to the safe state (STOP) rather than the dangerous state (GO).

If connection timeout is 5 seconds and read timeout is 10 seconds, what is the maximum time to wait?

Connection can take up to 5 seconds, then reading response can take up to 10 more, total 15 seconds worst case.

Failure Modes in Distributed Systems — System Design Concept

Q: Which of the following best describes why distributed systems are more complex than single-machine systems?

The core complexity of distributed systems comes from multiple failure points across network connections, not from hardware specs or user network quality.

🍪 TechCookies

Free

📚Failure Modes in Distributed SystemsFree

16 sections

~29 min total

30 quick quizzes

4 SD challenges linked

0 of 16 done·~30 min left

Concepts›Failure Modes in Distributed Systems›What Is a Distributed System?

0 / 16

16 sections~29 min

What Is a Distributed System?

Multiple independent computers communicating over networks as one coherent system

ReadQuizCode

~2 min

⋯

Network Partitions

Nodes stop communicating with each other despite both remaining operational

ReadQuizCode

~2 min

⋯

Process Crashes

Running services abruptly stop, causing loss of in-memory state and incomplete operations

ReadQuizCode

~2 min

⋯

Slow Dependencies

Services respond slower than expected, exhausting resources and degrading system capacity

ReadQuizCode

~2 min

⋯

Fail-Fast vs Fail-Safe vs Graceful Degradation

Three philosophies for handling failures: immediate errors, safe defaults, or reduced functionality

ReadQuiz

~2 min

⋯

Fail-Fast

Detect and immediately raise errors rather than hide failures or produce wrong results

ReadQuizCode

~2 min

⋯

Fail-Safe

Default to safest behavior when failure occurs, maintaining integrity over availability

ReadQuizCode

~2 min

⋯

Graceful Degradation

Continue serving core functionality when non-critical components fail

ReadQuizCode

~2 min

⋯

Timeout Design: Connection Timeout vs Read Timeout

Understanding different timeout types and applying them correctly to prevent hanging

ReadQuiz

~2 min

⋯

What Is a Timeout?

A limit on how long to wait before giving up on an operation

ReadQuiz

~2 min

⋯

The Two Main Types of Timeouts

Connection timeouts limit initial connection establishment; read timeouts limit data reception

ReadQuizCode

~2 min

⋯

A Third Type: Write Timeout

Limits how long to wait while sending data to server during requests

ReadQuiz

~2 min

⋯

Timeout Code Examples

Practical implementations of timeouts in Python, Java, and JavaScript

ReadQuizCode

~2 min

⋯

Choosing the Right Timeout Values

Guidelines for setting appropriate timeouts based on operation type and SLA

ReadQuizCode

~2 min

⋯

The Danger of No Timeout

Omitting timeouts causes thread exhaustion and permanent system hangs

ReadQuizCode

~2 min

⋯

Practice test

30 questions

~10 min

Section 1 of 16ReadQuick quiz

What Is a Distributed System?

Multiple independent computers communicating over networks as one coherent system

~2 min read

2 quick quizzes

Before we dive into failures, let us quickly set the stage. A distributed system is a collection of independent computers (called nodes) that communicate over a network and appear to the user as a single coherent system.

Examples:

A web application that has a frontend server, a backend API server, and a database
A ride-sharing app with separate services for user management, trip tracking, and payments
A news website that fetches data from a weather API, a sports stats API, and an ads server

The key insight is: every connection between components is a potential failure point.

☑ Quick check 1/2

Which of the following best describes why distributed systems are more complex than single-machine systems?

AEvery connection between components is a potential failure point

BDistributed systems require more memory than single machines

CUsers have slower internet connections

DDistributed systems always crash more frequently

Answer the quiz to continue

Notes

🔍

Loading…