Exemplary Questions
By David Bressoud @dbressoud
David Bressoud is DeWitt Wallace Professor Emeritus at Macalester College and former Director of the Conference Board of the Mathematical Sciences. davidbressoud.org
Last month, in Quantitative Reasoning with Definite Integrals, I discussed the work of Oehrtman and Simmons in which they explain the different processes involved in modeling a particular accumulation problem by constructing a definite integral. I listed four of the problems that they used to force students to think through this process and for which they monitored student thinking as they worked toward a solution.
Four problems are not many. There is a rich source of “good questions” that was produced by Maria Terrell and her team at Cornell in the early 2000’s, available at https://pi.math.cornell.edu/~GoodQuestions/. But good questions do not often make it onto calculus exams. The MAA Calculus study, Characteristics of Successful Programs in College Calculus, collected final Calculus I exams from several hundred colleges and universities across the U.S. These were analyzed by Michael Tallman and Marilyn Carlson with the results published in 2016, A Characterization of Calculus I Final Exams in U.S. Colleges and Universities.
Tallman and Carlson analyzed the 3735 questions on 150 of the exams that were randomly chosen from the full set. They found that 87% of the items required either simple memorization of a fact or recall and application of a procedure that had been taught in class. Very few questions tested understanding, the ability to apply understanding, or the ability to analyze. None required evaluating (making judgments based on criteria and standards) or creating (reorganizing existing knowledge into a new pattern or structure). Only 3% of the questions required an explanation.
Given what we test, it is not surprising that many (most?) students emerge from calculus with the belief that the course is all about memorization of facts together with practice of the procedures demonstrated in class. If there are any ideas behind calculus, most students move on to the next course blissfully unaware of them. It is not surprising that this is the state of affairs. Facts and procedures are easy to test and easy to grade. As far as the students are concerned, why bother trying to understand something when understanding will not be tested?
But what if you want students to think about the mathematics they purport to be learning? Cornell’s Good Questions are useful, but even more useful would be some guidance on what makes a good question and how to think about what you are assessing. A guide to questions that probe student understanding is supplied by the paper “Assessing Productive Meanings in Calculus” by Zachary Reed, Michael Tallman, and Michael Oerhtman, published this summer in PRIMUS.
The authors take on four common topics in calculus: related rates, applied optimization, area-based integration problems, and applied integration problems. They deconstruct common exam questions and identify what they lack, then exhibit exemplary questions and talk about why they are good questions and how they probe students understanding. The focus throughout is on quantitative reasoning (Are the students aware of what is being quantified and how the quantities are related?) and covariational reasoning (Are the students using an awareness of how change in one quantity affects changes in another?).
Examples of the questions offered by Reed, Tallman, and Oehrtman include
Related Rates. A man starts walking north at 4 ft/sec from a point P. Five minutes later a woman starts walking south at 5 ft/sec from a point 500 feet due east of P. At what rate are the people moving apart 15minutes after the woman starts walking? [The point being that there is no obvious formula to be employed. Students need to reason about how to describe the distance between the two people as a function of time.]
Optimization. A city wants to lay a new high-voltage line from the power station, located on 21st Avenue (which runs North–South), to the downtown area. Due to existing buildings, the power line must be underground in any region east of 21st Avenue, but it can be built above ground along 21st Avenue. The additional cost of burying the power line doubles the cost per mile of installing it. The downtown area is 5 miles north and 6 miles east of the station. What is the least expensive route for the new power line? [This is similar to many optimization problems with the twist that students need to think about cost as dependent on route and think about how to quantify both cost and route.]
Area-based Integration. See Figure 1. [This tests whether students understand Riemann sums and how they relate to area. Can students recognize that they are being asked to find the area of half a disc of radius 3?]
Figure 1. Area-based integration problem
Applied Integration. An electric train travels on a track with a speed determined by its location. The following table provides speeds v(x) of the train decelerating to a stop. The variable x denotes the distance from the point at which the brakes are first applied. The train is traveling at 18 m/s at this point and comes to a complete stop 40 meters down the track.
x (meters) 0 5 10 15 20 25 30 35 40
v(x) (m/s) 18 17 16 14 12 9 6 3 0
(a) Give the best possible underestimate for the time it took the train to travel 20 meters after the brakes were applied.
(b) Write an integral that gives the time required for the train to stop after applying its brakes.
[I have slightly rewritten the question to clarify the meaning of x and v(x).].
Figure 2. An electric train coming to a stop.
The final exam is the wrong time to suddenly confront students with the questions proposed by Reed, Tallman, and Oehrtman. Such questions on a timed final exam, especially if nothing like them has appeared before, will elicit confusion mixed with anger. But these are the kinds of questions that students should be able to answer if they understand calculus and are given enough time to think about them. They can be effective as group projects. And they need to carry a weight that makes clear that this is what the course is really about.
As I matured in my teaching, I abandoned traditional exams that focus on memorization and reproduction of procedures demonstrated in class. They are a trap that distorts student perception of what they are expected in learn. A few more challenging questions provide much more insight into what students have learned. This can be particularly effective for exams before the end of the term when students are given an opportunity to continue working on these problems after handing in their work. But even in the final when it will be the rare student who completely answers all of the problems, an analysis of student work can be very insightful into what a student has learned. But most importantly, questions that probe understanding communicate what we value most.
Reference
Zackery Reed, Michael A. Tallman & Michael Oehrtman (2023) Assessing Productive Meanings in Calculus, PRIMUS, 33:9, 939-964, DOI: 10.1080/10511970.2023.2222302