Using math to solve a novel real-world problem is hard. Cognitive science explains why. It also tells us what we need to do to overcome the feature of the human brain that causes the difficulty.

By Keith Devlin @profkeithdevlin 

 My posts for June and July focused on how we should be thinking about adapting mathematics education to reflect the major changes in mathematical praxis that took place in the late 1980s and through to the end of the 1990s. (A change I have summarized elsewhere in terms of a music metaphor: adapting from an era when being a mathematician was akin to mastering a series of musical instruments in order to play in an orchestra, to life in the post-2000 mathematical world where using mathematics is akin to conducting an orchestra.)

I noted that, because today’s math pros use technological aids to handle all the calculations (numerical or symbolic) and execute all the procedures, the educational focus has to shift from mastery of calculation and execution of procedures, which used to be important, to the mathematical thinking that is required in order to function effectively in the newer environment.

In those earlier posts, I reflected on what it would take to provide an educational experience to meet that requirement. I made a number of observations, based on that discussion:

“First, what  is taught is not, in itself, of any significance. The chances of anyone who finds they need to make use of mathematics at some point in their life or career being able to use any specific high school curriculum method is close to zero. In fact, by the time a student today graduates from university, the mathematics they may find themselves having to use may well have not been developed when they were at school. Such is the pace of change today.”

“Second, what is crucial to effective math learning is what is sometimes called “deep learning”; the ability to think fluidly and creatively, adapting definitions and techniques already mastered, reasoning by analogy with reasoning that has worked (or not worked) on similar problems in the past, and combining (in a creative fashion) known approaches to a novel situation.”

“But here’s the rub. The mass of evidence from research in cognitive science tells us that the only way to acquire that all important “deep learning” is by prolonged engagement with some specific mathematics topics. So, to be effective, any mathematics curriculum has to focus on a small number of specific topics.”

I punted on saying anything about that cognitive science research, promising to return to it in a future post. This month’s post is intended to deliver on that promise—at least to the extent possible in a monthly blog.

It turns out that, in order to develop the flexible thinking required to tackle novel problems—regardless of where those problems come from, what specific mathematical concepts they may embed, and what specific techniques their solution may involve—what is required is experience working in depth on a small number of topics, each of which can be represented and approached from at least two, and ideally more, different perspectives.

If you want the full scoop on the situation (actually, a fuller scoop—though the two meanings of ‘scoop’ lead to mixed metaphors—since human cognition and learning are highly complex domains), I always recommend Daniel Willingham’s excellent little book Why Don’t Children Like School? What you’ll get from me is the ten-cent tour. 

Our starting point is the recognition that the evolution by natural selection of the human brain equipped it to learn naturally from experiences, to better ensure its survival, or to perform in a more self-advantageous fashion, when next faced with a similar experience. Learning through experience in this fashion is automatic, and tends to be robust, but is heavily dependent on the particular circumstances in which that learning occurred. It does not take much variation in the circumstances to render what was learned ineffective. 

Here is a classic example of how we reason about familiar everyday experiences, studied by the psychologists Leda Cosmides and John Tooby in 1992. You have been given the task of chaperoning a group of young people at a birthday party. Those present who are over 21 may drink alcoholic beverages, those younger must stick to coke. To make your task easier, all the party-goers have been asked to place their IDs on the table where they are sitting. You walk over to a table with four young people. 

You see one is drinking a coke, another a beer. Their IDs are face-down so you’d need to pick them up to check their age. The remaining two have their IDs face up, and you see that one is 21 (the legal drinking age) the other 17 (not permitted to drink), but both have drinks in paper cups and you cannot determine what is in them unless you pick them up and sniff them. 

It’s possible for you to confirm that all four are following the rules about alcohol by sniffing one drink and turning over one of the two face-down IDs. Which drink and which ID?

Most people have little difficulty with this problem. You sniff the drink of the person whose ID shows they are 17 and you turn over the face-down ID of the person you see drinking beer. Though there are 8 possible drink-age combinations for the four people, your familiarity with the laws concerning alcohol prepares you well to answer this question with ease.

In contrast, consider the following problem, that presents a highly contrived scenario you are likely not familiar with. (Unless, that is, you have studied psychology. It is a classic problem called the Wason Test, devised by the psychologist Peter Wason in 1966.) In the Wason Test, you are presented with four cards on a table, each of which (you are told) has a letter on one side and a digit on the other. You are given the information that the cards are labeled according to the rule: 

A card with a vowel has an even number on the other side. 

You are then asked to turn two cards over to verify that all four cards are in accordance with that rule. Which cards do you turn over?

Typically, fewer than 10% of people answer this problem correctly when they first meet it.

Almost everyone says you must turn over the card with the letter E, to check that there is an even number on the other side. That is correct. It’s the choice of the second card that trips up the majority. You have to turn over the card with the 7. Because if there is a vowel on the other side, the rule is broken.

On the other hand, turning over the K or the 2 (most people opt for the 2 as their second card) tells you nothing about the validity of the rule.

If you have not seen this before, you may well have to think about this for a while. It’s surprisingly tricky. And that is precisely why it is so interesting, and so illuminating about how our minds work.

One reason it is interesting, is it is logically exactly the same problem as the one about the young partygoers. Just read “drinking alcohol” for “having a vowel”, and “over 21” for “even number” (so “under 21” corresponds to “odd number”). Yet one problem is easily solved, whereas fewer than 10% of people get the other problem correct at first encounter.

There is, it should be said, considerable discussion that can be (and has been) had concerning the reason for the difference in performance. For our present purposes, however, all we need is the empirical evidence that the concrete problem in a familiar scenario is easy, whereas the logically identical problem in a somewhat artificial scenario is not at all easy. (Even though I have known about the Wason Test for many decades, I still have to concentrate to get it right.) For what it shows is that we are not good at taking knowledge acquired in one circumstance and applying it in another, even though the underlying logical structure is identical and the exact same solution works.

Incidentally, if, when I had presented the Wason Test, I had told you that your previous answer to the party problem might help you, the likelihood that you would have got it right is still only 30%; up from 10%, but still low. Even knowing that two problems are connected does not guarantee you can take the solution to a problem learned in one set of circumstances and apply it to a structurally identical problem in another set of circumstances.

Here’s why this is relevant to mathematics. In order to solve mathematical problems (or, more generally, to solve problems using mathematical thinking), you generally need to identify the underlying logical structure, so you can apply (possibly with some adaptation) a mathematical solution you or someone else found for a structurally similar problem. That means digging down beneath the surface features of the problem to uncover the logical structure.

To be sure, once you have mastered—by which I mean fully understood—a particular mathematical concept, structure, or method, it becomes a lot easier to see it as the underlying structure (if indeed it is). For example, if you understand the optimization method known as linear programming, you will be able to identify many optimization problems where it can be applied. Quite simply, your knowledge of the method makes it possible for you to recognize when it may be of use.

But how do you recognize an underlying abstract structure or achieve understanding of an abstract method in the first place? 

Before I give an answer, let me ask an even more fundamental question. How did the person—or persons— who first formulated the concept of, and a solution method for, say, a linear programming problem get to that point?

The answer was (almost certainly, though I do not know the details of this particular example) by working on a number of optimization problems, and over time coming to recognize certain similarities—similarities not of context or presentation, but of underlying structure.  This is a process of pure creativity, and relatively few people exhibit such cognitive leaps. (Possibly because they don’t have sufficient interest in the issue to keep working on it long enough, and intensely enough, for the breakthrough to become possible. I’m not convinced that “creative genius” is anything more than inclination and tenacity, combined with an eternally open mind.)

Once someone has taken that creative first step, however, explaining the abstract structure and method to others, at the same time linking the abstract to a number of concrete instances as examples of how the abstract structure relates to them, can speed up considerably the ability of those others to master and apply the same general method.

It does, however, seem important to fully mastering an abstract structure that we make those connections between the abstract-and-general, to the concrete-and-particular. We learn best from concrete examples we experience, and the mind’s natural inclination is to store knowledge in a fashion closely bound up with the scenarios in which it was first acquired. Overcoming that constraint to learning to recognize abstract structural or logical patterns requires effort, and often the study of more than just a couple of examples. You need examples (plural) and you need to go deep into them.

I’ll illustrate that last observation with one more example from the cognitive psychology research literature. It was first proposed in a classic paper by Karl Duncker: “On problem solving,” Psychological Monographs, 1945, 58(5), i-113. The study of this problem that is most frequently cited in the literature is Mary Gick and Keith Holyoak, “Schema Induction and Analogical Transfer,” Cognitive Psychology 15, 1-38 (1983).

In the “Radiation problem,” you imagine you are a doctor faced with a patient who has a malignant tumor in their stomach. It is impossible to operate on the patient, but unless the tumor is destroyed the patient will die. There is a kind of ray that can be used to destroy the tumor. If the rays reach the tumor all at once at a sufficiently high intensity, the tumor will be destroyed. Unfortunately, at this intensity the healthy tissue that the rays pass through on the way to the tumor will also be destroyed. At lower intensities, the rays are harmless to healthy tissue, but they will not affect the tumor either. What type of procedure might be used to destroy the tumor with the rays, and at the same time avoid destroying the healthy tissue?

You might want to think about this for a while before proceeding to see if you can find a solution. As Guck and Holyoak discovered, fewer than 10% of subjects were able to solve it at first encounter.

Before I give the solution, however, let me tell you about an alternative way that Guck and Holyoak presented the problem to two separate groups of experimental subjects.

For both alternative groups, before presenting them with the radiation problem to solve, they asked them to read a story about a military problem and its solution. In this story, a General wishes to capture a fortress located in the center of a country. There are many paths radiating outward from the fortress. All have been mined so that while small groups of men can pass over the paths safely, any large force will detonate the mines. A full-scale direct attack is therefore impossible. The General’s solution, the story explains, is to divide his army into small groups, send each group along a different path, and have the groups converge simultaneously on the fortress.

As you may have noticed (though the evidence says there is a good chance you did not), there is an analogous “convergence” solution to the radiation problem. The doctor could direct multiple low-intensity rays toward the tumor simultaneously from different directions, so that the healthy tissue will be left unharmed by the low-intensity rays passing through it, but when all the low-intensity rays converge at the tumor they will combine to destroy it.

One of the two groups of subjects who had been exposed to the General’s problem and its solution before being shown the radiation problem, were given an explicit hint to look for a similarity between the two problems. As a result, about 75% of those subjects were able to solve the radiation problem, well up from 10% of those who had not received such priming. 

For the other group of General-story-before-radiation-puzzle subjects, however, the researchers set things up a little differently, in order to see how many of them would be able to spot the similarity in problem structure if they were not explicitly told to look for one. For this group, the subjects were first asked to memorize the General’s story in the guise of a study of story recall, and after completing that they were asked to work on the radiation problem. (More specifically, subjects were told at the outset that the experiment would have two parts, story recall and then problem solving. The delay between the two tasks was minimal.)

Under these conditions, only about 30% of subjects were able to solve the radiation problem.  Given that only about 10% of subjects could solve the radiation problem in the absence of any priming, that indicates that only a third or less of the subjects for whom the analogy was available were able to spontaneously notice it. What you may find surprising is that this disparity arose despite the fact that, knowing they were subjects in a psychology experiment, one might have expected that all subjects would consider how the first part might be related to the second. But they did not make that connection. That’s a significant finding that requires explanation, and from which we can learn a lot. 

A number of similar studies were conducted in the 1980s, in which a second priming problem was used: the Fire Chief Problem. In this scenario, a Fire Chief leads his crew to a woodshed fire just outside a village. When the crew gets there, they find the villagers have formed a line to convey water in buckets from a nearby well to try to douse the fire. But no matter how fast the villagers move the buckets along the line, the fire continues to burn. The Fire Chief’s solution is to get all the villagers to go to the well and fill all the buckets. They then take them to the woodshed and surround it. On the Fire Chief’s command, they all throw the contents of their buckets onto the fire. The combined effect of all the water hitting the woodshed at the same time is to extinguish the flames. The fire is put out.

In this three-problem study, only 10% of people solve the radiation problem when they see it without any priming story. Of subjects primed with the General’s problem, around 30% solve it. For subjects primed with both the General’s problem and the woodshed fire problem, around 80% save the patient. Working with more examples seems to increase the likelihood that we recognize the underlying logical structure.

Willingham, who I cited at the beginning, uses the term “inflexible knowledge” to refer to knowledge that is closely tied to the surface structure of the scenario in which it was acquired. He points out that it takes time, effort, and exposure to two or more different representations of what is in actuality the same problem in order to recognize the underlying structure that is the key to making that knowledge flexible—something to be applied to any novel scenario having the same underlying structure. 

In the case of the radiation problem, 90% of people see it purely in terms of radiation—essentially, a physics problem, unconnected to the General’s problem (a problem about military strategies) or to the Fire Chief problem (which they see as about how to coordinate action to put out a fire).

Yet those three instances of inflexible knowledge become mere applications of a more general strategy of a hub-and-spokes wheel, where activity that follows the spokes from the circumference inwards converges into a single combined action at the hub—ONCE YOU HAVE ACQUIRED THAT UNDERLYING STRUCTURE AND UNDERSTOOD HOW IT IS COMMON TO ALL THREE SCENARIOS.

In fact, once you have acquired that wheel concept, it is easy to recognize new instances where it can be applied. Being on an abstract level, your knowledge is flexible. But as the studies cited (and many others not referenced here) show, getting to that more abstract, flexible-knowledge structure is not automatic. It requires work.

Here’s the educational rub. Actionable, flexible knowledge cannot be taught, as a set of rules. It has to be acquired, through a process of struggling with a number of variants until the crucial underlying structure becomes apparent. There is no fast shortcut.

For anyone curious as to why the brain works the way it does, and why it finds some things particularly difficult to master, particularly the recognition of abstract structure, check out my old book The Math Gene, published in 2001, where I draw on a wide range of results from several scientific disciplines in an attempt to shed light on just how the human brain does mathematics, how it acquired the capacity to do so, and why it finds abstraction so challenging. 


Read archived posts.