Effectiveness of Online Quizzing in Increasing Class Preparation and Participation
Michael Marcell
Department of Psychology
College of Charleston
Abstract
Research suggests that frequent, regularly scheduled quizzing is associated with pedagogically desirable outcomes such as higher performance on exams and better class attendance. It was hypothesized that requiring students to complete brief, scheduled online quizzes on assigned reading material would also result in increases in the number of students who actually read the material prior to class and in the number of in-class questions and comments. For each of 3 semesters, students in 1 section of introductory psychology who took time-limited, out-of-class, “open-book” quizzes on daily readings were compared to students in another section who did not take quizzes during that unit. Analyses indicated that quizzing was associated with increases in both the number of students who reported that they came to class having read the assigned material and the number of student questions and comments made at the beginning of class. Spontaneous, anonymous comments on end-of-unit surveys suggested that the primary value of online quizzing was in helping some students maintain a regular reading and study schedule.
This should be referenced as:
Marcell, M. (August, 2005). Effectiveness of online quizzing in increasing class preparation and participation. Poster presented at the annual meeting of the American Psychological Association, Washington, D.C.
As a 28-year teacher of undergraduate psychology courses, it has been my experience that many students in introductory psychology classes come to class without having completed their assigned reading. This casual and unsurprising observation receives empirical support from research by Burchfield and Sappington (2000), who found that performance on a brief, unannounced quiz administered early in psychology courses revealed a significant decline in reading compliance over the years of their study (1981-1997). They estimated that only about one-third of undergraduate students -- particularly those in entry-level courses -- complete their reading assignments prior to class. When assessments come infrequently (as in many of my own psychology classes, in which there are typically 4-5 exams during a 14-week semester), it is possible that less mature and academically weaker students will fall further and further behind in their reading. In addition to the increased anxiety that might build in an individual as the completion of assigned course work is delayed (cf. Wesp, 1986), it seems reasonable to suspect that the overall classroom atmosphere might also suffer, with poor preparation contributing to diminished student involvement, lower-quality discussions, and less efficient use of class time.
One tool that might help minimize these problems is daily quizzing. Early research on college teaching revealed the pedagogical usefulness of quizzing in several domains. For example, Fitch, Drucker, and Norton (1951) found that students who had weekly quizzes were more likely to attend voluntary discussion groups and earn higher grades in the class, and Townsend and Wheatley (1975) found that students who took short daily quizzes had higher course grades than students who took longer or less frequent quizzes. Recent research has suggested that scheduled quizzes are associated with several pedagogically-desirable outcomes such as reduced student procrastination in a personalized instruction course (Wesp, 1986), improved understanding of the assigned reading material (Marchant, 2002), higher performance on exams (Maki & Maki, 2001), and better class attendance (Wilder & Flood, 2001).
Recent developments in web-based content delivery software have made it easier for teachers like me (who are unwilling to use precious class time to administer and review quizzes) to implement a regular quizzing regimen. I initially learned how to use WebCT, an interactive, online, content delivery and classroom management system, during a summer faculty technology institute, and then spent the subsequent semester incorporating several of its features into my introductory psychology class. Following this “break-in” period of instructor mistakes, quiz construction, and conversion of audio-visual materials into a digital formal, I began a three-semester pedagogical study on the effectiveness of regularly scheduled online quizzes in encouraging on-time reading and class participation. I hypothesized that requiring students to complete brief quizzes on assigned textbook reading material prior to class would result in the following: a) an increased likelihood that students actually read the material before class, and b) an increased number of student questions and comments at the beginning of class.
Method
Participants
Participants were students enrolled in six sections of Introduction to Psychological Science, a 3-hour undergraduate course taught at a southeastern liberal arts college. Data were collected from two sections per semester during the Spring, 2004, Fall, 2005, and Spring, 2005, semesters. The total number of participants was 172, with each section having an enrollment between 25-35 students. The total sample was 63.4% female and 36.6% male, which is similar to the ratio at the college. 63.4% of the participants were freshmen, 21,5% sophomores, 8.1% juniors, and 7.0% seniors.
Description of the Course, Quizzing Procedure, and Independent Variable
The two sections involved each semester in the project were designed to be as similar as possible. The classes were taught on the same days, in adjacent time periods, in the same computer-equipped classroom, and by the same professor (the author), who used identical online resources, reading assignments, exam schedule, and academic policies (e.g., grading scale, office hours, makeup policy) for each section, and who prepared the same teaching materials for each class.
The course was divided into five regular units with 4-5 class periods per unit and an exam at the end of each unit. Every non-exam class period had a scheduled reading assignment, whose average length was 21 pages. Students were reminded in the syllabus and throughout the semester that they should complete the assigned reading before they come to class because the course is primarily structured as an independent reading, lecture, and discussion class. They were also told in the syllabus that I would begin each class by asking if they had any questions or comments about the assigned reading, and that I would spend as much time as needed clarifying difficult topics encountered in the textbook (Gray, 2002) by either redescribing concepts, using different examples, referring to digital resources posted on WebCT, or conducting classroom demonstrations. Independent reading and learning were touted as major goals of the course, and these goals were backed by emphasizing that approximately 60% of the items on exams would be based on material assigned in the textbook, whether or not it was reviewed during class.
A quiz consisted of 5 multiple-choice items randomly selected by the computer from a pool of 15-20 test-bank items chosen to represent the content of the upcoming class’s assigned reading material (typically, about 5 new items were added to a unit quiz each semester). Quiz questions were not re-used on exams and could not be readily answered by consulting the glossary for a definition or the author index for a page referral to the relevant study. Following Brothen and Wambach’s (2004) procedures, quizzes were administered online over WebCT, unproctored, and followed by immediate feedback on the correctness of the answers and the quiz grade. Because Brothen and Wambach found that untimed quizzes encourage an inefficient study strategy of searching for answers in unread text, we employed their timing parameters and allowed students 7 minutes (8 minutes during the third semester) to complete the 5-item quiz. Students were able to use their textbook and notes while taking a quiz, but were informed that it would be considered an Honor Code violation to receive help from another person.
A quiz was posted at least 48 hours before class and was taken, at the student’s convenience, at least 1 hour before class began. Although a quiz could be taken only once, students were able to review their completed quizzes anytime during the semester. Students who experienced a computer malfunction (e.g., loss of Internet connection while taking a quiz) could arrange to take a hardcopy makeup version of the quiz. Syllabus and in-class reminders stressed that a quiz should be taken after the student has completed an assigned reading for a class, and the syllabus also contained the following advice about taking a quiz:
Note that if you have not read the material, it is unlikely that you will be able to find the answers to 5 questions in 7 [or 8] minutes! The best strategy will be to read the material, answer the questions to the best of your ability, and use any remaining time at the end of a quiz to double-check your answers.
Students in each section were required to take scheduled quizzes on 13 of the 22 assigned readings. Scores on quizzes comprised only 6% of the final point total (the other course grade components were four regular exams [57%], a final exam [32%], and a research requirement [5%]; students were allowed three unexcused class absences without penalty). Students in both sections participated in online quizzing during Unit 1 in order to learn how to use the WebCT software, and the first of the four quizzes in Unit 1 was a non-credit practice quiz that remained available throughout the semester. Before the semester began, a coin flip determined which section took online quizzes during Units 2 and 4 (9 quizzes), and which section took online quizzes during Units 3 and 5 (9 quizzes). Therefore, students in each section experienced the manipulation of having or not having online quizzes associated with the assigned reading. Even when a section was not taking online quizzes, students were still expected to complete the scheduled reading before class.
Although students knew from the syllabus that data were being collected for a research project, they were not told the nature of the project. The syllabus included the following text about the research project:
This semester I will use information normally collected during the course of the semester (such as survey responses, quiz scores, and exam scores) in a “behind-the-scenes” research project. The purpose of the research project is to explore whether certain course practices increase the effectiveness with which this class is taught. My focus will be on class performance, not individual performance. Thus, any data tabulated for the research project will be reported anonymously as class averages and will have no impact on your grade in the class. If you do not wish to participate, then let me know during the first two weeks of class and I will exclude your assessment data from the statistical analyses.
Once again, your participation in this faculty research project is completely voluntary, your involvement is passive [i.e., you need to do nothing special – the data (e.g., exam grades) are collected over the course of a normal semester], your scores (when used for this project) will not be tied to your personal identity, and your choice to participate or not to participate will have absolutely no impact on your grade in the course.
The course included several novel components (e.g., use of digital graphics, video clips, and sounds posted on WebCT; continuous availability of PowerPoint outlines; post-unit surveys for course improvement; discussion threads; scheduled quizzing) that might have been the focus of research.
Dependent Variables
There were two primary outcome measures:
1) During the first 5 minutes of each class, the teacher asked if there were any questions or comments relating to the assigned reading. These were listed on the board and addressed after all questions and comments were solicited. The dependent variable was the number of student questions asked or comments made at the beginning of class, and this figure was inconspicuously recorded after the class ended. No more than two items from a given student were counted towards this total, and only questions and comments about the assigned reading (and not questions about the class schedule, the upcoming exam, the correctness of a quiz item, and so on) were tallied. Other than the preceding distinctions, no attempt was made to distinguish between the quality of the questions and comments. Questions and comments made after the initial period of solicitation were not tallied because of the greater likelihood that they would be prompted by in-class developments more than by out-of-class reading.
2) Students had the option of completing extra-credit, anonymous WebCT surveys that were posted online within a few hours after the last class of a unit and remained available until 24 hours before the exam. The syllabus and in-class reminders stressed that students’ survey responses would not be associated with their identity, and that their name would be used only to allow proper recording of credit. Surveys consisted of 10 randomly-ordered items (Likert scale, multiple-choice, and text box) that assessed overall understanding of material in the unit, ease of use of online WebCT resources for that unit, preferred ways of viewing classroom graphics, two aspects of the unit that they disliked or liked, and so on. The dependent variable was provided by a single text-box question in which students estimated the number of days (out of 4 or 5 class periods) that they came to class prepared, which was defined as having completed all or most of the assigned reading.
Results and Discussion
Independent samples t tests were performed for each unit between the section receiving quizzes and the section not receiving quizzes. Comparisons were 1-tailed, alpha was set at .05, and effect sizes were interpreted using Cohen’s (1988) guidelines. Figure 1 contains the results of analyses performed on the anonymous, self-report survey item in which students estimated the number of days in a unit that they came to class prepared. Across semesters, students who received quizzes reported that they were prepared for more classes than students who did not receive quizzes (7 of the 12 comparisons across semesters were significant with large effect sizes, and 2 of the comparisons were marginally significant with medium effect sizes). Like Ruscio (2001), who found that randomly-administered quizzes increase the likelihood of students completing their assigned reading on time, the current results suggest that regularly-scheduled quizzes also promote well-paced, independent reading of assigned textbook material before it is covered in class. Figure 2 contains the results of analyses performed on number of student questions and comments made at the beginning of class. Across semesters, classes that received quizzes asked more reading-related questions and made more reading-related comments than classes that did not receive quizzes (9 of the 12 comparisons across semesters were significant, 1 of the comparison was marginally significant, and all 10 effect sizes were large). For professors who value student – teacher interaction during class, the finding that regular quizzing encourages greater student participation is a welcome addition to the tool belt of techniques to get students to say more in class.
The results suggest, then, that brief, regularly-scheduled online quizzes are associated with the following positive behaviors in students: a) Completing assigned reading before class, and b) Asking more reading-related questions and making more reading-related comments at the beginning of class. Speaking as a teacher, it was my distinct impression that when a class was not quizzed on a unit’s reading assignments, its members tended to be more passive and less willing to shape the direction of the class’s activities with initial questions and comments.
Interestingly, many students spontaneously reported that regular quizzing provided a structure that encouraged them to pace their reading and a reason to work harder to understand the material. A post hoc tally of students’ anonymous responses to two questions on the end-of –the-unit surveys (“What were two things that you liked [or disliked] about this unit?”) revealed that favorable comments containing the root word “quiz” far outweighed unfavorable comments: Across the 3 semesters, 80%, 70%, and 79% of the unsolicited comments about quizzes were complimentary, with the most frequent response being a complaint about not receiving quizzes! Here are some examples of students’ comments:
I find that the quizzes on WebCT help me a lot to pick out the key points in each chapter and would rather have them every unit.
I wish there had been quizzes because it makes me have to read the chapters and learn key points.
I disliked not taking the online quizzes. I really enjoyed them and felt like while I was reading I absorbed much more just because I KNEW I would be taking a quiz after reading the chapter.
…no quizzing created lack of motivation to read.
I disliked not having the WebCT quizzes because I did not keep up as well without them.
They make me read the material.
In a way I missed the quizzes because I didn't follow the reading as closely
as I had in the previous unit.
As much as I hate to say it, I also didn't like not having quizzes, because
the quizzes make me read on time and more efficiently…
The online quizzes forced you to stay on top reading assignments.
…not having quizzes made it easier to get behind in my reading
One thing I disliked about the unit was the lack of on-line quizzes; only because I felt like they help me to keep up with my readings in class.
These comments are similar to observations made by other researchers (e.g.,
Graham, 1999; Ruscio, 2001; White & Hammer, 2000; Wilder, Flood, & Stromsnes,
2001) who noted that frequent quizzing, whether scheduled or unannounced, is
perceived by students as assisting them in establishing good study habits and
maintaining a regular reading schedule. Future studies may wish to determine
whether Graham’s (2000) finding that the effects of quizzing on test grades
are strongest for mid-range (C) students and weakest for top (A) students also
extend to reading preparation and classroom participation. In any case, it is
clear from the present study that regular quizzing motivates some students to
read assigned material before class and to contribute questions and comments
in class about ideas that they have already made an independent effort to understand.
References
Brothen, T., & Wambach, C. (2004). The value of time limits on Internet quizzes. Teaching of Psychology, 31(1), 62-64.
Burchfield, C.M., & Sappington, J. (2000). Compliance with required reading assignments. Teaching of Psychology, 27(1), 58-60.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Lawrence Erlbaum Associates.
Fitch, M.L., Drucker, A.J., & Norton, J.A. (1951). Frequent testing as a motivating factor in large lecture classes. Journal of Educational Psychology, 42, 1-20.
Graham, R.B. (1999). Unannounced quizzes raise test scores selectively for mid-range students. Teaching of Psychology, 26(4), 271-273.
Gray, P. (2002). Psychology (4th Edition). New York: Worth Publishers.
Maki, W.S., & Maki, R.H. (2001). Mastery quizzes on the Web: Results from a Web-based introductory psychology course. Behavior Research Methods, Instruments & Computers, 33(2), 212-216.
Marchant, G.J. (2002). Student reading of assigned articles: Will this be on the test? Teaching of Psychology, 29(1), 49-51.
Ruscio, J. (2001). Administering quizzes at random to increase students’ reading. Teaching of Psychology, 28(3), 204-206.
Townsend, N.R., & Wheatley, G.H. (1975). Analysis of frequency of tests and varying feedback delays in college mathematics achievement. College Student Journal, 9(1), 32-36.
Wesp, R. (1986). Reducing procrastination through required course involvement. Teaching of Psychology, 13(3), 128-130.
Wilder, D.A., Flood, W.A., & Stromsnes, W. (2001). The use of random extra credit quizzes to increase student attendance. Journal of Instructional Psychology, 28(2), 117-120.
If you would like a copy of this paper and/or the figures below (which could not be pasted here), write me at marcellm@cofc.edu.
Figure 1. Mean Number of Days Prepared for Class as a Function of Section's Experimental Condition (No Quiz vs. Quiz) and Unit [A = First Semester, B = Second Semester, C = Third Semester].
Figure 2. Mean Number of Questions Asked at Start of Class as a Function of Section's Experimental Condition (No Quiz vs. Quiz) and Unit [A = First Semester, B = Second Semester, C = Third Semester].