The Impact of Traditional- and Adaptive-Responsive Online Homework Systems on Student Performance in General Chemistry: Analyzing Extra Credit Participation

The positive impact of online homework has been clearly shown in previous literature. Online homework has become especially important in these times as online education grows. The recent development of the adaptive-responsive homework systems (such as Assessment of Learning in Knowledge Spaces--ALEKS) calls for a comparison of the effectiveness of this new type of system to the traditional-responsive systems (such as Sapling Learning). This study quantitatively compares extra credit scores and final exams and final points between two types of homework systems in a general chemistry course. Results indicate that extra credit completion using ALEKS (the adaptive-responsive system) has a higher correlation with homework performance, final exam performance, and final grade compared to extra credit completion using Sapling (the traditional-responsive system). This relationship difference could be due to a difference in the nature of how extra credit is earned within each system.


Introduction
Over the past few years, student success in chemistry and other science, technology, engineering and mathematics (STEM) fields has declined, with possible reasons being the poor perception of the field, and students' lack of engagement with these fields (Gill, 2011;PECK). It is therefore imperative that educational institutions look for ways to continue to improve engagement and retention of concepts (Erdoğdu, 2019). One of these methods has been the use of technology in the classroom (Chandran, Treagust, & Tobin, 1987;Magalhães, Ferreira, Cunha, & Rosário, 2020;Revell, 2013), which has manifested in a variety of ways, including flipped classrooms (utilizing class time for worked problems and providing online resources to replace the traditional lecture) that allow students to access the material before and after entering the lecture hall (Hibbard, Sung, & Wells, 2015;Schiller, 2013). Technology has also been implemented by allowing smartphones into the classroom to answer questions in class (helping the instructor measure content knowledge and understanding) (Heflin, Shewmaker, & Nguyen, 2017), by using tablets, computers, and clicker systems (Revell, 2013), as well as online homework systems that are used in place of traditional pen and paper homework (Richards-Babb, Drelick, Henry, & Robertson-Honecker, 2011;Roschelle, Feng, Murphy, & Mason, 2016).
One method some instructors are using to improve student success rates in STEM courses is online homework systems (OHS) (Balta, Perera-Rodríguez, & Hervás-Gómez, 2017;Fyneweaver, 2008;Liberatore, 2011;Parker & Loudon, 2012;Richards-Babb et al., 2011). In comparison to traditional paper and pen homework, one of the major benefits of OHS for students is the instant feedback these systems provide, increasing the motivation for studying and reviewing the material. With both positive student outcomes and positive student feedback (Fyneweaver, 2008;Richards-Babb, Curtis, Georgieva, & Penn, 2015), many companies like Sapling Learning (Learning, 2018), WileyPLUS(WileyPLUS, 2019), and Mastering Chemistry (Pearson, 2019) are producing their own versions of online homework. These types of OHS are referred to as the "traditional-responsive" homework systems. They work by giving students the same homework problems (sometimes in a different order for the individual student) and multiple attempts at the problem. For each incorrect attempt, a hint or explanation is given to allow students immediate feedback on how they can improve and where they go wrong. This could potentially be why online homework systems increase student success; because they give students immediate motivation to review the material. It also gives them repetition on these problems to help students retain the information better for exams.
A second type of OHS, known as the "adaptive-responsive" system is a more recent type of OHS. These include systems such as MindTap (Cengage, 2019) and Assessment of LEarning in Knowledge Spaces (ALEKS) (Hill, 2019). These systems use artificial intelligence (AI) to determine the individual's initial (or base) knowledge space for that subject, and based on this pre-determined knowledge space, provide an individualized pathway to help individuals learn the course content through the duration of the course (Hill, 2019). The adaptive-responsive systems give individual students different questions and continue to adapt the difficulty of the questions as the student gets them correct or incorrect. Due to the relative novelty of the adaptive-responsive OHS, there are fewer studies, in comparison to the other types of homework systems, on them. (Richards-Babb, Curtis, Ratcliff, Roy, & Mikalik, 2018) This study seeks to explore more ways to utilize these OHS to further enhance the experiences of students in and outside the classroom.
In addition to technology, the implementation of extra credit has been shown to be another way to increase student engagement (Lei, 2013;Padilla-Walker, 2006;Radhakrishan, Lam, & Ho, 2009). Some studies have shown that there are benefits associated with extra credit opportunities. In an advanced psychology course, Padilla-Walker used daily quizzes that covered material for the lecture that day for extra credit and found that how students performed on these quizzes was a very strong predictor of their performance on exams. She found that these daily quizzes provided positive incentives for students to read ahead, which enhancing their understanding of the material. (Padilla-Walker, 2006) Similarly, in a freshman business management course, Harrison also utilized pre-lecture quizzes to analyze the effectiveness of this kind of quiz on students' performance and also found that there was a direct correlation between having pre-lecture quizzes and the students' performance on exams. (Harrison, 2016) There are several researchers and instructors who have positive views towards extra credit, and there are many studies that have shown that extra credit assignments can help students learn and understand concepts better. In fact, most of the studies about extra credit have shown that these opportunities help motivate students to learn and be engaged with the material, however, there are researchers and instructors who believe extra credit may be harmful to students. There is limited research on the cons of providing extra credit opportunities, but the limited studies that do exist highlight the following as the main drawbacks to using extra credit: i) grade inflation, ii) an indication of low standards by the instructor, iii) encourages students not to take the course as seriously, iv) the students who need it do not take it as seriously, and v) it presents an unnecessary load for the instructors in terms of grading (Maurer, 2006;Mays & Bower, 2005;Pynes, 2014) Overall, extra credit can be regarded as an opportunity for students to help boost their overall grade as well as motivate them to be engaged and want to learn the material (Mays & Bower, 2005). However, the major drawback to providing extra credit opportunities is that most times, the students who need the extra credit the most are not the ones taking advantage of it.
Extra credit opportunities can be administered in a variety of ways, and one popular way is in the form of a pre-lecture quiz. (Padilla-Walker, 2006) , (Harrison, 2016) Another method of providing extra credit is in-lecture extra credit, usually delivered as quiz or clicker questions during class, instead of before class starts (Wilder, Flood, & Stromsnes, 2001). For this study, we compare two different extra credit opportunities, both outside the classroom. The first method is centered around the 'pre-quiz' idea. For each weekly quiz, the students are required to answer between 5-7 questions (depending on the week and topics to be covered) on concepts that have not yet been covered in class, encouraging them to read ahead of the class. The second method for administering extra credit was to encourage the students to continue to work through concepts that had been covered previously in the class, and this happened throughout the semester. The students were rewarded (extra credit) for continuing to work through and master topics throughout the semester. This study seeks to expand the research base by comparing the effectiveness of two different ways of offering extra credit (one way that has not yet been explored based on our literature search) on student success in an introductory general chemistry course by exploring the relationship between the students' performance on the extra credit assignments, their homework grades, final exam scores, and final grades.
This study focuses on the success of students in the course as it relates to extra credit participation. The two main research questions this study seeks to address are: Is there a relationship between how well students do on extra credit assignments and their success in the course?
Is there a significant difference between the two different extra credit methods and student success in the course?
For the purpose of this study, we are defining 'student success' in two ways: performance on the final exam, and final grade in the course. As the final exam is cumulative (60% previous material, 40% current material), we believe that the performance of students on this exam is indicative of how much they have been able to retain over the course of the semester.

Methods
The study was approved by the Institutional Review Board at the University of Oklahoma. Participants included students who were enrolled in in-person General Chemistry I lectures in the Fall 2018 and Spring 2019 semesters. The study was approved by the Institutional Review Board at the University of Oklahoma. Participants included students who were enrolled in in-person General Chemistry I lectures in the Fall 2018 and Spring 2019 semesters. In the fall semester, there were five course sections, while in the spring, there were four sections. General Chemistry I is typically taken during the Fall semester, which accounts for the section difference, as the enrolment is lower in the Spring, since it is considered the "off-semester" for the course. Looking at the average ACT scores (Table 3 and Table 5) in both semesters, there is no statistical difference between these semesters, neither is there a difference in racial demographics between sections in each semester (Table 4 and Table 6). One difference between the sections for both semesters was the instructors. Table 1 and Table 2 show a breakdown of the instructors that taught each semester. Even though multiple instructors taught each semester, the course requirements were the same, as there was a common textbook, syllabus, exams and lab requirements. The other difference among the sections was the homework system students used. In the fall, one section was randomly selected to use ALEKS, while the other four sections utilized Sapling. In the spring, two sections used ALEKS and the other two sections used Sapling. Table 2 displays the homework system each section used, the number of participating students who were in each section and the percentage of students who attempted the extra credit assignments. Homework extra credit was different for each homework system. Sapling homework extra credit was in the form of readings and questions related to the topics of the upcoming week. The students could receive a total of 42 points (3 points each week for 14 weeks) for answering questions on material that had not yet been covered in class. For the second system (Adaptive-responsive, Assessment of LEarning in Knowledge Spaces (ALEKS)), extra credit was given for completion of the pie chart comprising of over 100 topics related to General Chemistry. If a student learned 100% of the topics required by the end of the semester (as evidenced by their pie in ALEKS) (Hill, 2019), they would receive a total of 42 extra credit points at the end of the semester towards their final grade. The full 42 extra credit points accounted for approximately 5% of their final grade The main difference between the two extra credit systems was that the Sapling homework extra credit was used to encourage students to read ahead, while the ALEKS homework extra credit was used to encourage students to continuously review the material that had already been covered.
This quantitative study analyzed student performance on their homework assignments, final exam, final course grade, and homework extra credit received to investigate their mastery in the material. The final course grade includes all components of the course shown in Table 1, while the final exam was specifically the cumulative exam administered during the final week of class. Student mastery was measured by their performance on the final exam. The exams in this course were cumulative, multiple choice exams (with the exception of Exam 1). Exams 1 through 3 consisted of about 30-35 questions, while the final exam ranged from 60 -70 questions. This study also sought to investigate the relationship of homework system to students' success, based on their final grades in general chemistry.

Results and Discussion
There were 1,806 students enrolled in General Chemistry I that participated in this study. In the fall semester, 1,045 students were enrolled; 772 students using Sapling and 273 using ALEKS. In the spring, 761 students were enrolled; 340 students using Sapling and 421 using ALEKS. Specific course components were correlated among each semester by plotting graphs in excel and using the r-squared value and are shown in Table 7.

Correlation between homework extra credit (EC) and homework performance
Homework performance and homework EC correlations were run for fall 2018 semester data comparing the two homework systems. Homework performance was the overall grade received on the homework that was required for the course (a total of 15 homework assignments worth 5 points each were given to both sections over the course of the semester), while the homework EC was the grade out of 42 extra credit points received. The correlation between homework scores and homework extra credit completion was r=0.89 for ALEKS users and r=0.65 for Sapling users (Figure 1).   Figures 1 and 2 show similar patterns. Both ALEKS sections (fall and spring) show similar correlation patterns -a strong correlation between the homework scores and the homework EC (r=0.89 and 0.87 respectively). Both Sapling sections (fall and spring) also show similar patterns, with moderate correlations between both constructs (r=0.65 and 0.70 respectively). These correlations were expected between the homework performance and the homework EC because students have access to outside resources and are not under pressure when completing these homework assignments and EC problems. The correlations were likely stronger in the ALEKS sections because the students had to keep working on the past covered topics throughout the semester to earn their extra credit points. Even though the Sapling sections always had access to the previous topics and homework assignments, there was no incentive for them to go back through topics/assignments already completed.

Correlation between homework EC and final exam scores
The final exam for the course (both in the spring and fall semesters) consisted of about 60 -70 questions and was cumulative. The final exam scores were also correlated with homework extra credit, with r=0.63 for those who used ALEKS and r=0.33 for those who used Sapling for the fall 2018 semester (Figure 3).  As with the previous correlations (homework performance and homework EC), the trends across each homework system are similar. For the ALEKS sections, a moderate correlation is seen in both semesters (r=0.63 and 0.63 respectively), while a weak correlation is seen in both Sapling semesters (r=0.33 and 0.27 respectively). As mentioned earlier, the final exam in this course is a cumulative one, with 60% of the material being previously covered material, and 40% material that was covered 2 -3 weeks before the final exam. It can be argued that the moderate correlation between the final exam scores and the EC points seen for the ALEKS sections is based on how the extra credit opportunity was administered. Since the extra credit opportunity for the ALEKS section gave the students an incentive to review the previous topics covered in the course, more students in the ALEKS sections were likely reviewing the material more often than the Sapling sections, which kept the course materials fresher in their minds -hence the higher correlation between the final exam score and the EC points. Another argument that can be made is that the students in the ALEKS sections are spending more time on task because of how the OHS is designed, which may be responsible for the higher correlations. While this is not a wrong thought process, a counter argument to that is that students are choosing to spend more time on the task given (and not just giving up) because of the incentive to do so.

Correlation between homework EC and final course grade
Final course grades and homework EC were also correlated, showing r=0.76 for students who used ALEKS and r=0.46 for students who used Sapling in the Fall 2018 semester ( Figure 5).  As with the previous two correlations above, the trends for each system for both semesters are similar. These similarities across the semesters for these correlations are important because the population of students for each semester is different. At our institution, in the fall semester, most of the students are pre-professional (e.g. pre-medicine, pre-dental, and pre-pharmacy) and engineering majors, while in the spring semester, most of the students are nursing majors, meteorology majors and sometimes seniors who are looking for that last science course to graduate. In a nutshell, the fall students usually have stronger math and science backgrounds, while the spring students tend to have weaker math and science backgrounds.
For the ALEKS sections, the correlations between the final course grades and homework EC, across both semesters was strong (r=0.76 and 0.74 respectively), while a moderate correlation was observed for the Sapling sections (r=0.46 and 0.47 respectively). Since the final exam makes up a significant portion of the final course grade (see Table 1 for point distribution in the course), these correlations are expected. The total amount of homework EC that could be earned was 42 points, and on average, the students across all four sections earned most of the 42 points (about 80% of the points). This is important to note because it shows that with regards to the extra credit, regardless of the homework system being used, the students were able to receive most of the extra credit. Based on this statistic, the strong correlations observed between the final exam grade and the homework EC, and the final course grade and homework EC for ALEKS sections compared to the Sapling sections was due to more than the extra possible 42 points boost to the grade.
We believe that the ALEKS extra credit opportunity provided the students an incentive to not just complete their assignments, but to consistently review concepts that had been covered previously in the course (not just during the exam period). The periodic 'knowledge checks' (Hill, 2019) ALEKS provides ensures that students are consistently interacting with and reviewing course material, previous, current and future material. This is something we expect students to do, but it isn't something students do naturally. By providing students with an incentive to imbibe this study habit early on in their academic career (for most of the students), we can hopefully instill a culture of learning where students do not just study right before exams or quizzes but have a culture of continuous studying/learning.

Limitations
For this study, correlations were used to determine these relationships, so we cannot soundly say that the extra credit participation is the cause for this. This research, however, does shed light on the potential strength of the ALEKS pie chart as helpful to students. Another limitation is the workload assigned to students. ALEKS pie completion may have included more work and time on task than Sapling, however, a study by Eichler and Peeples (2013) showed that time-on task does not significantly impact student performance. More research may need to be carried out to further investigate the impact of time on task.

Future Directions
The authors recommend continuing to investigate extra credit and the effect that participation and the nature of extra credit has on student performance. Time on task research will also be imperative to understanding the efficacy of ALEKS and Sapling.

Conclusion
In this study, we examined the relationship between student performance and different types of extra credit assignments and investigated whether there was a significant difference between the different extra credit opportunities utilized (ALEKS vs. Sapling) and student success in a first semester general chemistry course. We found that there was a strong correlation between student success in the course and the method of extra credit that encouraged students to review past material, compared to the weaker correlation with the method of extra credit that has students preview material that has not yet been covered (e.g. pre-lecture quizzes).
These findings suggest that extra credit opportunities that are provided to students be those that encourage students to review content that has already been taught, as these consistent reviews help students retain information that has been taught previously.