By David Bressoud @dbressoud

Online homework systems have become ubiquitous in introductory mathematics courses. The 2009 AMS survey found that 65% of PhD-granting departments of mathematics, 70% of Master’s degree-granting departments, and 43% of Bachelor’s degree-granting departments were using online homework (Kehoe, 2010). For Calculus I, the 2010 MAA survey undertaken as part of Characteristics of Successful Programs in College Calculus (CSPCC, NSF #0910240) showed that of the 429 instructors who answered the questions about their homework assignments, 14% collected no homework. Of those who did grade homework, 46% collected written homework only, 26% collected online homework only, and 28% collected a combination of written and online homework (Sonnert & Ellis, 2015). Incidentally, this last has been my personal policy. Using online homework for procedural knowledge frees me to ask a few penetrating questions on the written homework.

Online homework systems have several advantages over traditional, hand-graded homework. They give the student immediate feedback, they usually provide an opportunity to retry this or a similar problem, and they are able to provide a degree of individualization so that different students see different problems. In addition, they are either inexpensive (WeBWorK) or bundled into the cost of the textbook (WebAssign and MyMathLab). But there have been concerns, chief of which is that students do not need to show their work, eliminating the opportunity to provide feedback that goes beyond whether the answer is right or wrong.

There have been several studies of online homework. Most (Halcrow & Dunnigan, 2012; LaRose, 2010; Zerr, 2007) have compared graded online homework to ungraded assignments. These have generally shown some benefit from online homework, though often that benefit has not been statistically significant. A 2001 study at Rutgers (Hirsch & Weibel, 2003) found that students in general calculus who had some of the written homework problems replaced by WeBWorK assignments did show a small but statistically significant improvement in their test scores.

Last year, Larry Smolinsky and Gestur Olafsson at Louisiana State University (Figure 1) published their results of a controlled comparison of hand-graded versus online-graded homework in Calculus II (Smolinsky et al, 2018). These two lead authors taught four sections of mainstream Calculus II in Fall 2016, two with hand-graded assignments, two using an online system (WebAssign). To control for the effects of class size, there were two small sections (40 students) and two large sections (90–150 students), one each for each type of homework.  To adjust for possible instructor effects, each lesson was taught in all four sections by one of the two instructors. The study also controlled for gender, student ACT/SAT scores, and whether or not a student was on a Pell grant.

Outcome was measured by a composite score based on midterm and final exams, which had both open-ended and multiple choice questions. The authors also tracked student performance on the open-ended and multiple choice questions separately. In addition, they looked to see if hand-graded homework had an effect on student performance on three of the exam questions that dealt with graphing, since only hand-written assignments provide an opportunity for students to draw graphs.

There was no evidence that online grading is detrimental (Figure 2). Looking at class size, grading type, and gender, the only interaction that was even moderately significant (p < 0.056) was for hand-graded homework in large sections, where women performed about 0.1 points above men. The authors concluded that “It does not seem necessary in this era to assign homework that does not provide feedback to students.”

It is also worth mentioning that Smolinsky and Olafsson found that with large and small classes taught with exactly the same lectures, there was no significant difference in composite score performance. This is line with the data collected in CSPCC where class size was not correlated with our outcome variables of confidence, enjoyment of mathematics, or desire to continue the study of calculus. Other variables totally swamped any effect from class size.

The 2015 MAA survey in Progress through Calculus (NSF #1430540) revealed that only 45% of PhD-granting departments and 17% of Master’s degree-granting departments have a uniform policy across multiple sections of mainstream Calculus I on the use of online homework. In most cases, this is left to the discretion of the instructor. This is unfortunate because one of the findings of CSPCC was that the most successful calculus programs have a high degree of coordination among the different sections of each course, including policies on how homework is collected and graded (Rasmussen & Ellis, 2015). Today, with easy access to online homework for single variable calculus, there is no excuse for not having a uniform policy that requires the use of this tool as part of the assessment mix.

Read the Bressoud’s Launchings archive. 

References

Halcrow, C. and Dunnigan, G. (2012). Online homework in Calculus I: Friend or foe? PRIMUS, 22(8), 664–682.

Hirsch, L. and Weibel, C. (2003). Statistical evidence that web-based homework helps. FOCUS, 23(2), 14.

Kehoe, E. (2010). AMS homework software survey. Notices of the American Mathematical Society, 57, 753–757.

LaRose, P.G. (2010). The impact of implementing web homework in second-semester calculus. PRIMUS, 20(8), 664–683.

Rasmussen, C. and Ellis, J. (2015). Calculus coordination at PhD-granting universities: more than just using the same syllabus, textbook, and final exam. In D. M. Bressoud, V. Mesa & C. L. Rasmussen (Eds.), Insights and recommendations from the MAA national study of college calculus (pp. 107–116). Washington, DC: Mathematical Association of America.

Smolinsky, L., Olafsson, G., Marx, B.D., and Wang, G. (2018). Online and handwritten homework in Calculus for STEM majors. Journal of Educational Computing Research. doi.org/10.1177/0735633118800808

Sonnert, G. and Ellis, J. (2015). Survey questions and codebook. In D. M. Bressoud, V. Mesa &C. L. Rasmussen (Eds.), Insights and recommendations from the MAA national study of college calculus (pp. 139–169). Washington, DC: Mathematical Association of America. Data available at https://www.maa.org/CSPCC.

Zerr, R.J. (2007). A quantitative and qualitative analysis of the effectiveness of online homework in first-semester calculus. Journal of Computers in Mathematics and Science Teaching, 26(1), 55–73.