Understanding System Models and Properties
1. Understanding CAP Theorem
| Property | Meaning | Sacrifice Under Partition |
|---|---|---|
| Consistency (C) | Every read sees the latest write (linearizable) | — |
| Availability (A) | Every request gets a non-error response | — |
| Partition Tolerance (P) | System continues despite network partitions | Mandatory in real systems |
| CP Systems | HBase, MongoDB (default), etcd, ZooKeeper, Spanner | Reject writes on minority side |
| AP Systems | Cassandra, DynamoDB, Riak, Couchbase | Accept writes; reconcile later |
Note: CAP is asymptotic — in absence of partition, systems can be both C and A. Real choice is between C and A only when P occurs.
2. Understanding PACELC Theorem
| Condition | Choice | Examples |
|---|---|---|
| P (partition) | A vs C | Same as CAP |
| E (else, normal ops) | L (latency) vs C (consistency) | — |
| PA/EL | Available + low latency | Cassandra, DynamoDB, Riak |
| PC/EC | Always consistent | HBase, BigTable, Spanner (mostly) |
| PA/EC | Available on partition, consistent normally | MongoDB |
3. Understanding ACID vs BASE Properties
4. Understanding Fault Models
| Model | Assumption | Tolerance Cost |
|---|---|---|
| Crash-Stop | Failed nodes halt forever | f+1 nodes for f failures |
| Crash-Recovery | Nodes may recover with persistent state | 2f+1 (Paxos/Raft) |
| Omission | Messages may be dropped | Retransmission protocols |
| Byzantine | Arbitrary/malicious behavior | 3f+1 (PBFT) |
5. Understanding Timing Models
| Model | Bound Assumption | Practicality |
|---|---|---|
| Synchronous | Known bounds on delay & clock drift | Real-time/embedded only |
| Asynchronous | No timing assumptions | FLP impossibility for consensus |
| Partial Synchrony | Bounds exist but unknown, eventually hold | Real-world model (Raft, Paxos) |
Note: FLP Impossibility — In a fully async system with even one crash failure, deterministic consensus is impossible. Real systems use timeouts (partial synchrony).
6. Understanding System Guarantees
| Guarantee | Meaning |
|---|---|
| Safety | "Bad things never happen" (e.g., no two leaders) |
| Liveness | "Good things eventually happen" (e.g., progress) |
| At-most-once | Message delivered 0 or 1 times |
| At-least-once | Message delivered ≥ 1 time (duplicates possible) |
| Exactly-once | Effectively once via dedup + idempotency |
7. Understanding Availability Metrics (SLA, SLO, SLI)
| Term | Definition | Example |
|---|---|---|
| SLI | Service Level Indicator — measured metric | p99 latency = 120ms |
| SLO | Service Level Objective — internal target | 99.9% requests < 200ms |
| SLA | Service Level Agreement — contractual + penalty | 99.95% uptime or refund |
| Error Budget | 1 − SLO; allowed failure quota | 0.1% = 43.2 min/month |
8. Understanding Durability Guarantees
| Level | Mechanism | Risk |
|---|---|---|
| Memory only | RAM | Lost on crash |
| Single disk fsync | WAL + fsync | Disk failure |
| RAID | Multiple disks | Host failure |
| Replicated (sync) | Quorum write | Region failure |
| Cross-region | Geo-replication | Multi-region outage |
| 11 nines (S3) | Erasure coding + multi-AZ | ~10⁻¹¹/yr loss |
9. Understanding Failure Modes
| Mode | Symptom | Detection Strategy |
|---|---|---|
| Fail-stop | Process exits | TCP RST, heartbeat loss |
| Fail-slow / gray | Degraded throughput/latency | SLI monitoring, outlier detection |
| Network partition | Subset unreachable | Quorum loss, gossip view |
| Resource exhaustion | OOM, FD limit, disk full | USE metrics, alerts |
| Data corruption | Checksum mismatch | CRC, Merkle trees |
| Configuration drift | Inconsistent behavior | Audit, IaC reconciliation |
10. Understanding Performance Guarantees
| Metric | Common Target |
|---|---|
| p50 latency | 10-50ms |
| p99 latency | 100-500ms |
| p999 latency | < 1-2s |
| Throughput | RPS sustained at SLO |
| Saturation | < 70% utilization for headroom |