December 23, 2019
Beyond Calculus
Perusing Hacker News last month, a particularly-captivating Ask YC popped up on the first page: what’s the most valuable thing you can learn in an hour? There was, as expected, an array of deep, well-thought-out answers. One particular answer, though, tickled the math side of my brain that gets me cranking up Medium & delving down a path of research.
One of the most up-voted answers on the forum revolved around exponential functions. For a self-filtered group that proudly associates itself with complex STEM topics, I found this answer & it’s popularity oddly-satisfying. An erudite group suggesting that one of the most undervalued lessons revolves around a supposedly elementary concept (exponents)? Captivating indeed.
So, that’s what we’ll review here. We won’t dive into the basic notation & many modeling formulas behind exponential functions; instead, we’ll approach this misconception phenomena with an equal focus on blind spots & shortcuts. First, we’ll review classic examples of exponential misunderstandings to highlight specific mental model errors. Next, we’ll derive rough mental shortcuts, such as the Rule of 72, that’ll provide a frame of reference moving forward. And finally, last, we’ll tie it all together with a parting message.
A tale as old as time, our history is peppered with multiple examples & narratives meant to highlight our troubles with this mystical rate of growth — few are as classic as those passed down through religion. An ancient folklore & a modern fun-fact, both stories shared below help highlight the frequency & depth with which we misinterpret exponential growth.
There is a Hindu temple in Kerala, India, by the name of the Ambalappuzha Sri Krishna Temple. Believed to have been built during the 17th century AD, a long-standing tradition since it’s inception has been the daily offering of paal-payasam (pudding made of rice) to travelers.
According to Hindu folklore, the standing-tradition of serving Paal Paysam to all visitors stems from a legendary game of chess between the local king & the lord Krishna himself. The king, a boisterous chess enthusiast, had a habit of challenging traveling visitors to games of chess. One day, a cloaked, traveling sage openly accepted the kings’ inquiry. Arrogantly, the king offered any reward that the sage could name. The sage modestly asked just for a few grains of rice in the following manner: the king was to put a single grain of rice on the first chess square & double it on every consequent one.
Predictably, the king lost & immediately ordered a bag of rice to be brought to the chessboard. He then started placing rice grains according to the agreed-upon arrangement: 1 grain on the first square, 2 on the second, 4 on the third, 8 on the fourth & so on. Following the exponential growth of the rice payment the king quickly realized that he was unable to fulfill his promise — on the 20th square the king would have had to put 1,000,000 grains of rice, on the 40th, the king would have had to put 1,000,000,000 grains of rice &, finally, on the 64th square the king would have had to put more than 18,000,000,000,000,000,000 grains of rice (the equivalent of about 21- billion tons of rice).
With the king completely befuddled, it was at that point that the Lord Krishna revealed his true identity & told him that he can pay the debt over time. Which is why, to this day, visiting pilgrims are still feasting on Paal Paysam — the king’s debt to Lord Krishna still stands.
The lesson in this parable is straight-forward & simple: we severely underestimate the magnitude of exponential growth. Modeling the example above is clear:
(note, the only reason it’s not simply 2^n is because the parable starts with “1 grain on the first square”).
The second parable, The Birthday Paradox, is a curious misnomer since it’s literally not paradoxical — it’s just recognized as highly counter-intuitive. Less of a religious folklore & more-so a commonly-told “math fun fact,” the Birthday Paradox expounds our weak spots with exponents by cloaking them behind fractions & permutations. It goes as follows:
How many people need to be in a room for there to be a 50% chance that any two people share the same birth date?
Before reading on — what’s your best estimate on that number?
Considering there are 365 unique days per year, most people guesstimate or wrongly assume simple, linear division (365/2) for an answer in the low hundreds. Predictably, they’re shocked to hear that right answer is a rather-paltry 23 people.
The Birthday Paradox, relative to the Legend of Paal Paysam, is significantly more counter-intuitive because it hides behind not one, but multiple cognitive biases. While the former parable highlights our misinterpretation of the magnitude given an exponential function, the Birthday paradox compounds that confusion.
For one, the Legend of Paal Paysam is straight-forward: it’s the exponential function f(n) = 2^(n-1). The Birthday Paradox, on the other hand, is much trickier — very few people intuitively grasp that this riddle is indeed an exponential problem (as we’ll see below). Second, while the former parable deals with a single input, The Birthday Paradox introduces a selfish bias: the question asks for any random two people in a group to match. Most people only consider their birthday & overlook the fact that every person introduced to the group can possibly match with every other person already in the group.
This will be a bit involved, but it’s worth proving to ourselves that this is indeed an exponential function. Let’s start with the simplest example: what’s the probability (p(n)) of two people (a group of n=2) sharing the same birthday? Easy: p(2) = 1/365 = 0.27%.
No big deal. But now, let’s throw a third individual in the mix, what’s the probability of any two people (a pair), out of a group of three people (n=3) , sharing the same birthday? The probability here is a bit more involved. Assuming people A,B,C, the likelihood of person A matching with person B or person C is simply twice the probability of the previous question ((2*1)/365). However, we’re not done here — we can’t forget about a possible match between person B & C. Since we’ve already confirmed that they don’t share the same date as personal A though, our denominator decreases by one to 364. So our final probability here: p(3) = (2/365) + (1/364) ~0.82%
As you can likely guess by now, increasing our set of people, or increasing (n), leads to increasingly-uglier terms. So, here, we’ll instead introduce a nifty probability shortcut that makes the math much more palpable. Instead of calculating the probability of a match, we’re going to seek the logical equivalence of its complement; anecdotally, the probability of a birthday match is the same as one minus the probability of no matches: p(n) = 1 — p(n)’. In this context, the “probability of not matching” simply means the probability of a birthday being unique. Let’s restate p(2) & p(3):
Even with our shortcut, our function increases whenever n increases; however, this new form at least stays compact within a single term as opposed to adding a new term for every increase of n. This next line might require a quick brush-up on permutations, but the generalized form of the above is:
When we graph just the right part of the function above (the complement), it becomes immediately obvious that we are, in fact, dealing with an exponential function
The above graphs the complement p(n)’, the probability of no matches / unique birthday, versus (n), the number of people considered. It’s a concave down graph, but it’s exponential nonetheless. As one would guess, & we can now confirm, the probability of no matches approximates 0 as the number of people approaches 365 (number of possible birthdays).
Simple question? Yes. Simple solution? Not really, no. Easily identifiable as an exponential relationship? Absolutely not. At least theoretically, exponential functions are much more common than we assume — it’s our anchored need to associate linear-change to observed patterns that obfuscates these relationships by hiding them in our blind spot.
Both of the classic tales above serve the same purpose: highlighting how consistently inconsistent we are with our overall grasp of a supposedly very basic operation. Both tales share a linearity-bias where we attempt to frame exponential relations within a linear relation & the result is always the same: large magnitude errors in either underestimating or overestimating the output. It seems that a bias to assign a linear relationship to an observed pattern is inherent to our nature.
The second tale, however, expands our lack of intuition by additionally hiding the exponential relation behind fractions & permutations; in fact, it’s not until we actually graph the function that we’re visually convinced that we’re even looking at an exponential function. There’s likely a selfishness bias involved here as well; most people only ever consider their birthday matching up with someone in the room, not that any two birthdays match.
This section, however, attempts to turn these mental errors on their heads by deriving & introducing mental shortcuts. Below, we specifically cover two well-known rules for dealing with exponential relations; not coincidentally, they’re applicable to the parables above.
The Rule of 72 is a famous shortcut for calculating the amount of time it takes for a principle to double given a fixed interest rate. Derived in the world of finance, the rule is typically applied in terms of investments; however, it’s generally applicable to all matter involving exponential growth such as the size of a population or the number of nodes in a social graph.
In order to derive the Rule of 72, we pose the following question: how long, in years (t), would it take for a principal (p), to double, assuming annually-compounding growth (g)? We start with the formula for compounding growth, which is straight-forward:
This is where the “dirty” in the “quick & dirty” in our approximation happens. To arrive at a nice, comfortable, linear rule-of-thumb, we claim that for *low* interest rates (aka fairly low values of g): ln (1 +g) = g, therefore:
To calculate the amount of time it takes for (p) to double given a constant rate of growth(g), all we need to do is divide .693/g, or, 69.3/g*100 (easier to work without percentages). And that’s it! We’ve derived a clever & semi-accurate tool to calculate the time horizon to double a principle. Admittedly, however, 69.3% is an ugly number to deal with; we’ve already sacrificed perfect accuracy, so why not round this final figure to something much friendlier for a mental division? With multiple factors (2,3,4,6,8,9,12, etc…), 72 is a friendly number for mental operations; our simple doubling-formula changes to t = 72 / g*100.
And now we’ve derived the Rule of 72. The approximation of time required to double a principal assuming consistent annual compounding. In terms of accuracy, the formula above is most accurate for the range of interest rates that fall within 4% — 12%:
In reference to The Birthday Paradox, let’s again re-state the question: how many people do we need to consider before we have a 50% chance that any two random people share a birthday? Like the Rule of 72, we’re going to derive the shortcut. For our starting point, let’s pick up from where we left off with the complement of p (the likelihood that a birthday has no matches):
First, let’s abstract the question to make it a more broad statement: how large does the subset of random items (n), each with a normally distributed property of (y), have to be to guarantee a 50% match of property (y)? To generalize the problem, we’re simply swapping out “people” for (n), & “birthday” (365) for the normally distributed property (y):
Recall that the right-most term (big, long ugly factorial fraction) calculates the probability that every value is different. Let’s start by revisiting this term, first, we’ll re-write it as a series:
We already know that we’re dealing with an exponential function so we’re going to re-write the series above. A little shortcut, when values are close to 0, a rough first-order Taylor approximation for exponents is: e^x = 1+x, so:
With the figure on the right above, we now expand a first-order Taylor series:
We’ll introduce a second short-cut here. Algebraically, adding numbers from 1 to n is logically equivalent to n(n+1)/2. Therefore:
The last shortcut, considering the weight of the exponential term versus the constant added, we’re going to approximate & say that n²+n ~ n². Plugging the above into the high-level probability formula & assuming we’re looking for a 50% match, we’re left with:
Voila! If you take √y items (17% more if you want to be picky) then you have about a 50% chance of getting a match. If you plug in other numbers you can solve for other probabilities; a generalized form of this with desired probability (m), number of elements (n) & the set of distributed property (y) is as follows:
Remember that m is the desired chance of a match (it’s easy to get confused). To confirm our derived shortcut, let’s attempt to find the number of people needed to provide a 50% matching probability:
Plugging the above into a calculator result in 22.49, which is indeed extremely close to the true answer of 23. If you want a 90% chance of matching birthdays, plug m=90% into the equation, we arrive at n=40.99, which is again extremely close to the true value of 42 people. Granted, the above still looks really nasty, however, it works for all desired probabilities, not only 50%. The friendly, final, semi-accurate shortcut from this section revolves around a fixed desired probability of 50%. Commonly seen in cryptography as a shortcut formula for the birthday attack, the shortcut for a 50% collision, with an accepted margin of error of 17% (essentially forgoing the left term in the above) is simply:
Again, it’s semi-accurate at best, but it’s a beautifully-succinct rule of thumb. Let’s try it one last time on our birthday problem. With y = 365, the probability of any two random items with a normal distribution of n matching reaches 50% with 19.1 people. Not perfect, but far closer to 23 than the average guess.
In his eminent lecture, Professor Al Bartlett began his one-hour talk with the opening statement:
The greatest shortcoming of the human race is our inability to understand the exponential function
For the following hour, Professor Bartlett deconstructs exactly how un-intuitive we are at understanding exponential systems & why this is alarming given our planets finite resources. His main point? Highlighting the worrying fact that in an ecosystem of fixed resources, we cannot sustain exponential growth. He then contrasts this data-backed narrative with demonstrably unnerving statements from so-called “experts”, media & political leaders — their statements dramatically inconsistent with the facts. One of the most famous images from his lecture, the following chart, amply demonstrates the power of compounding in the remarkable growth in the world’s population over 1000 years. Bear in mind, this is what 1.5–2% population growth since 1950 actually looks like:
The famous lecture, ‘Arithmetic, Population, Energy’, is still available online today & well-worth a read. People typically learn better through narratives, like the two parables laid above — Bartlett also strongly leverages this by peppering multiple, modern, data-backed narratives to drive home his point:
In one example, a local paper in Boulder, Colorado asked council members for their view on what rate of population growth they’d like to see in the town over the coming 70 years. Members provided a range of answers from 1% to 5%; only to realize that a 5% annual growth results in a 32-fold increase.
In another Bartlett example, a skier is told the price of their season ski pass in Vail, Colorado has doubled in ten years. They are shocked, despite this constituting only a 7% rise each year. This growth rate is hardly worthy of protest & easily evades the attention of an avid skier considering their annual ski pass renewal.
It takes patience to feel truly ripped off. Symmetrically, it takes patience to truly benefit from investments in passive income vehicles. The famous futurist Jason Silva postulated that:
Exponential changes run counter-intuitive to the way our linear brains make projections about change, & so we don’t realize how fast the future is coming
In a very real way, compounding interest is one of the most powerful “forces” governing our lives; we’re all universally-subjected to the passage of time, which, in turn, cements every decision we make in our everyday lives. Compounding growth is the why behind the epithet:
What you do in a day is what you do in your life
Because on a long enough timeline, these consistent decisions grow over time as we act out on ingrained habits. So reflect & carefully analyze the actions & decisions you’re taking today for they’re not independent, but rather one in a series pattern of lifelong decisions. Stay alert to the possibility of consistent growth, it’s…
Crucial when borrowing money
Crucial when saving & investing
Crucial when you’re engaging in unhealthy behavior
Crucial when you’re engaging in healthy behavior
And on & on…
