Discussion

This Item illustrates one significant balance point between Items 8 and 9, “don’t optimize prematurely” and “don’t pessimize prematurely.” That makes this a tough Item to write, lest it be misconstrued as “premature optimization.” It is not that.

Here’s the background and motivation: Memory and disk capacity continue to grow exponentially; for example, from 1988 to 2004 disk capacity grew by about 112% per year (nearly 1,900-fold growth per decade), whereas even Moore’s Law is just 59% per year (100-fold per decade). One clear consequence is that whatever your code does today it may be asked to do tomorrow against more data—much more data. A bad (worse than linear) asymptotic behavior of an algorithm will sooner or later bring the most powerful system to its knees: Just throw enough data at it.