August 29, 2022

In my introductory computer science class, our professor demanded us to always start approaching problems by writing out example input-output pairs to the program we had to construct. Given a description of the problem at hand, we had to come up with valid test cases for before we actually implemented the algorithm. These test cases would then be verified by the solution algorithm that the teaching staff provided (not visible to us). The professor called this system Examplar .

I didn’t buy this approach at first.

Like a lot of first-semester freshmen at my college, I thought I was too smart for something so slow and tedious. Sometimes finding an interesting test case took hours. I just wanted to finish the assignment as fast as possible to prove my programming skills. I thought, or rather refused to doubt, that I could understand problems perfectly at first pass, or maybe the second without additional work.

As you can guess, I was wrong.

Not only did I often find additional details that I missed at first, sometimes I even found that I had completely misunderstood the problem. Even more, I occasionally gained direct insights into clean solutions by thinking about interesting test cases. Halfway through the course, I was rather embarrassed by my initial rejection of this system. B y the end of the course, I became a fervent advocate for teaching computer science, and really problem solving in general, this way. But what exactly does starting with examples accomplish?

Here is why I think this approach worked well for teaching problem solving. Borrowing from psychology, any problem is represented in your mind as its “problem space”. It is your mental representation of a problem and all the possible ways to solve it (that you know of). That is, as with anything you know, you only have access to a representation of the problem, as opposed to itself. You are unlikely to understand a problem if your representation of it is too inaccurate or incomplete. You are unlikely to solve a problem if your knowledge about the problem is inadequate.

Since knowledge is acquired by having and interacting with accurate representations of the world, let’s focus on the representation part.

Let’s consider a reasonably complex problem. The kind that requires you to actually think, and not just punch numbers into an equation. Unless you are a true genius, your problem space at first glance is overwhelmingly likely to be flawed. You might be missing crucial details, framing things in an inefficient way, or just misunderstanding the problem… What Examplar prompts you to do then, is both expand and correct your problem space. By thinking of unique potential solutions and materializing them, you illuminate dark, unknown areas outside your current problem space. Exemplar’s test case verifier provides an assessment on whether your exploration of the problem is on the right track or not. If you come up with wrong examples, it will tell you so, and you will be able to learn from errors. It is a positive sum game between you and Examplar. You only gain and cannot lose.

For the same reason, mathematicians and physicists like to consider so-called special cases of their problems. Often, special cases are a lot easier to understand, study, and solve when compared to the general case. For instance, when Einstein was coming up with special relativity, he had to take into consideration that whatever he came up with had to reduce to the nonrelativistic theory of Newton at low velocity. Otherwise his theory would make false predictions in the special case. This served as an easy verification method of his results. As another example, it takes a careful observation to realize that the Pythagorean theorem is just a special case of the so-called distance formula. Then, it is possible to use the Pythagorean theorem to help you understand and recall the distance formula, and even generalize it to arbitrarily many dimensions.

By exploring the problem space with examples, we improve our understanding of the problem. More generally, by moving the hand, we enrich the mind. When exploring the uncertain has more upsides than downsides, we say that the process is “ antifragile”, as from Nassim Taleb’s famous book with the same name. Exploring the problem space with Examplar is decidedly antifragile. Indeed, Taleb advocates that the process of learning should start with “random tinkering” of knowledge – Examplar is doing exactly that by encouraging students to think hard about examples and get feedback on their correctness..

This point extends beyond problem solving, too. When you read something, what you have in mind is again a representation of the material – the reading space. By actively thinking about hypothetical situations where this reading space may be challenged, or connecting it to pieces of knowledge from other sources, the reading space becomes more aligned with reality. Though your mental model of the world may not be accurate in the end, it is still more refined than the initial reading space. In contrast to Examplar, such an effort is not necessarily safe because you can make inappropriate connections, but it still can be antifragile when carefully conducted. This is why doing problems is so important for math and physics students. It is too easy to have a flawed reading space from dense textbooks, heavy lectures, and bad professors. Doing problems can significantly correct and expand it, which implies that doing problems but not checking the answers against some form of reference is likely going to be a waste of time. Reading alone does not necessarily make you learn.

I made this exact mistake once. When I started self-studying quantum mechanics from a textbook, I would just read and take notes without trying any of the exercises. On the one hand, I was just lazy. But on the other hand, I was also afraid that I wouldn’t be able to do the exercises – too hurtful for my immature pride. Ironically, not doing the problems meant that I was not able to identify where my understanding was weak or inaccurate. This then made problems later on in the book impossible because I had a weak foundation from before. It formed a negative feedback loop, and my learning was going absolutely nowhere. It was only when I decided to complement the reading with a course and problem sets did I actually become more comfortable with the material. As a specific instance that still sticks with me years later, it was only when I showed that two commuting operators share a basis of eigenfunctions did I truly understand the underlying mathematics of the uncertainty principle.

Exploring mental representations with examples and problems is but one case of what I call the “mind and hand” duality. Or rather, in classical philosophical thought, it is the dichotomy between reason and experience^[a] – the rational and the empirical. The “hand” really is an intangible force realigning your thoughts, revealing the ever-so-subtle but significant details of the world. To create this force, you often have to get your actual hands moving. This is why the best way to come up with startup ideas is not to try to think of them but to look for problems. By “thinking”, we pull from information that we already have, reason and theorize about them, and come to some sort of conclusion. By “looking”, we talk to people, observe meticulously, and work on personal projects to seek valuable problems. The prior is more of the mind, the latter is more of the hand. A startup, after all, is more an endeavor of the hand than of the mind. The mind organizes information; the hand provides information and corrects it. Without either, great ideas and solutions are hard to find.