brandon-powers / DDIA-1.md

Save brandon-powers/0a433ad73a01336df1439a2485662fe3 to your computer and use it in GitHub Desktop.

Reliable, Scalable, and Maintainable Applications

An application being "data-intensive" vs. "compute-intensive" means that CPU is not the limiting factor -- the amount, complexity, and change rate of the data passing through it is.

They're usually built from common building blocks (smaller data systems + application code = your larger data system):

• store data to be read later (databases)
• remember the result of an expensive computation to speed up reads of it (caches)
• search data by keyword or filter (search indexes)
• batch, stream processing

Reliability

A fault is something that can go wrong in a system. Anticipating and handling faults in an application, making it fault-tolerant (of some kinds of faults), will make it more reliable.

Hardware or software faults, such as a machine going down or a bug trickling through a system, are kinds of faults.

Human error being another. To avoid faults due to humans, keep in mind the following:

• design APIs to do the "right" thing without too much restriction
• use sandbox environments & real data
• test thoroughly (unit + integration + manual)
• detailed & clear monitoring; also called the "telemetry" in other engineering disciplines

Scalability

The ability of a system to cope with increased load.

Describing Load

Use load parameters (e.g. requests pers second to a web server) that are specific to your system.

Describing Performance

What happens when the load increases and the system resources are unchanged -- how is performance affected? What happens when the load increases and we want the performance to be unchanged -- how much of an increase in system resources is required?

Use metrics like throughput and response time (as percentiles, p50 for median, p90, p95, p99).

Scaling up or vertically is when you use a more powerful machine (with more system resources: memory, CPU, etc.). Scaling out or horizontally is when you distribute load across multiple smaller machines; also called a "shared-nothing" architecture.

Maintainability

1. operability: keeping a system running smoothly (visibility into runtime internals and application monitoring is crucial here)
2. simplicity: managing complexity
3. evolvability/extensibility: make it easy to adapt it for unanticipated use cases as (functional or non-functional) requirements inevitably change

Other

An application has to meet various requirements in order to be useful. There are functional requirement (what is should do, such as allowing data to be stored, retrieved, searched, and processed in various ways), and some nonfunctional requirements (general properties like security, reliability, compliance, scalability, compatibility, and maintainability).

This chapter discussed reliability, scalability, and maintainability in more details, but this block was useful.