Ordinal Logistic Regression Analysis in Effective Teaching Practices

: The Association of College and University Educators (ACUE) course in effective teaching practices has been applied to an undergraduate mathematics class at Texas A and M International University (TAMIU) for the past two years (Spring 2022, Fall 2022 and Spring 2023). Practices such as ensuring equitable access to learning, embracing diversity in the classroom, providing useful feedback and teaching powerful note-taking skills were applied throughout the whole semester in class. This study focuses on the comparison between the same class in Spring 2022 when ACUE’s effective teaching practices were not present and the classes in which ACUE was present in Fall 2022 and Spring 2023; in total, 89 students from two classes were analyzed with identification data removed. Ordinal logistic regression is used to analyze how effective teaching practices are related to students' final grades by analyzing students' grades, assignment submissions and class participation in different classes. Fisher’s exact test is applied to data analysis to check how effective teaching practices affect class participation; results show that for students who were from the class with ACUE practices, the odds of being more likely (i.e., “very” with a value of 2 or “somewhat likely” with a value of 1 versus “unlikely” with a value of 0) to have a passing grade is 1.92 times that of students who were from the class without ACUE practices, holding constant all other variables such as homework assignments and class participation. The impact of effective teaching practices is statistically significant on class participation (p-value = 0.04); this study concurs with the previous studies that ACUE’s effective teaching practices have positively improved students’ performance and should be applied to college classrooms. Ordinal logistic regression was the first time applied in analyzing


Introduction
Effective teaching is critical to students' success, especially for many first-generation learners, whose parents or legal guardians have not gone to college.With the effort from the center for effective teaching and learning, California State University in Los Angeles, has experienced a six-year graduation rate increase from 34-46% (Brown and Kurzweil, 2017;Jankowski, 2017).Both Brown and Jankowski have summarized the improvement in classrooms with effective teaching practices.In late 2014, the Association of College and University Educators (ACUE) convinced higher education leaders and experts in pedagogy to develop a scalable and comprehensive program on the essentials of college teaching.The result is ACUE's course in effective teaching practices, in which faculty learn about and implement evidence-based practices that keep students engaged, enrolled and learning.The course articulates five major areas of pedagogical practice, addressing how to design an effective course and class; establish a productive learning environment; use active learning techniques; promote higher-order thinking and assess to inform instruction and promote learning.Within these five units are 28 discrete topics that present over 200 evidence-based teaching practices (Mangum, 2017;ACUE, 2023).Mangum summarized that ACUE has become the link between faculty members and student success by providing a comprehensive, scalable and evidence-based program that prepares and credentials faculty in effective instruction.ACUE released its mission in 2023 which is to ensure student success through quality instruction and it delivers the highest-quality courses, pathways and certifications with quick-implement practices that make a huge impact on student success.In 2021, the Texas A and M University System (TAMUS) announced a 3-year continuation of its partnership with ACUE to expand the initiative to promote quality instruction, student success and equity (TAMUS, 2023).How are ACUE effective practices impacting first-generation students in a border university?effective teaching practices in Texas A and M International University (TAMIU) have been studied previously (Wang, 2024), showing that ACUE effective practices have increased students' performance and course satisfaction by 14% in students' evaluation at the end of the semester.In this study, the researcher compares two classes from the same course.One class applied ACUE practices while the other class did not apply ACUE practices.Using statistical analysis tools such as ordinal logistic regression and Fisher's exact test, how ACUE effective practices are correlated with students' performance and class participation is analyzed.Ordinal logistic regression is a statistical analysis method that can be used to model the relationship between an ordinal response variable and one or more explanatory variables (Parry, 2020).Cornell gave a very detailed description of the definition of ordinal logistic regression, how it is used in software with different packages and how to interpret the results.Ordinal logistic regression has been applied to many areas such as medical health and students' success in school.Singh et al. (2020) used an ordinal logistic regression model to describe factors associated with the extent of nodal involvement in oral cancer patients and its prospective validation.Ordinal logistic regression can also be used to analyze students' performance (Sesay et al., 2021).Compared with other models such as ordered logistic regression model and ANOVA, the advantages of ordinal logistic regression have been studied by Wang et al. (2022).Wang pointed out that when the outcome variable is non-interval and ordered, ordinal logistic regression works better than other statistical analyses such as ordinary least squares, ANOVA, or multinomial logistic regression.
The ACUE effective practices applied in this research have been displayed in Table 1 based on (Wang, 2024).
This study sets out to answer the following question: For students from the class with ACUE practices, how much more likely were they to have a passing grade than students from a class without ACUE practices.

Data and Variable
The undergraduate class Math 1325 Business Math II was selected to apply ACUE effective teaching practices in fall 2022 and Spring 2023.As a control group, the same course in Spring 2022, which did not apply ACUE effective teaching practices, was selected.In total, 89 students' information such as overall letter grade (A, B, C, D, F), homework assignments grade and class participation, was entered for the data analysis.There are four variables.The grade is the student's final letter grade in this course, which is categorized as grade = 2 for letter grades A and B, grade = 1 for letter grades C and D and grade = 0 for letter grade F. ACUE, which has two values: 0 and 1, shows that either the student is from the class with ACUE practices (ACUE = 1) or from the class without ACUE practices (ACUE = 0).There were ten homework assignments throughout the whole semester.The homework assignment average was calculated and scored between 0-100.HW is for the homework assignment average.For any student whose homework assignment average is above 80, HW has a value of 2. If the homework assignment average is between 60-80, then HW has a value of 1. Otherwise, HW has a value of 0. CP is for class participation.Based on students' involvement throughout the semester, CP has three values: 2 for frequent participation such as almost every week, 1 for participation from time to time and 0 for rare or zero participation.The distribution of each variable is displayed in Table 2.The variable ACUE has values 0 and 1 since classes are either applied ACUE effective practices or not applied ACUE effective practices.For any other variables such as Grade, HW and CP, there are three values (0, 1, 2).In Table 2, each entry is the number of students in each category.The researcher removed students' identification data before use.

Ordinal Logistic Regression
An ordinal variable is a categorical variable for which there is a clear ordering of the category levels.Ordinal logistic regression is an extension of logistic regression where the log odds (or logit) of a binary response are linearly related to the independent variables.The definition of logit has been explained in previous studies (Wang et al., 2022;Foley, 2020).It can be summarized as the following: where, j[1, j-1]are the levels of the ordinal outcome variable , which is the grade in this research.The proportional odds model assumes there is a common set of slope parameters  for the predictors.The ordinal outcomes are distinguished by the j-1 intercepts j.
Therefore, the ordinal logistic regression can be rewritten as follows: where,  is the number of predictors.
In this study, the response variable, Grade, has three levels: 0, 1 and 2, then there are 2 logits.There are three variables: ACUE, HW and CP.The assumption this research considers is the assumption of proportional odds: The effect of variable ACUE is constant for each increase in the level of grade.The ordinal logistic regression model and data analysis are done in Rstudio.Packages used are foreign, ggplot, MASS and Hmisc.
The research question in this study is: Is there any significant difference in grades among students from the class with ACUE practices and the class without ACUE practices?If there is a difference, how does ACUE impact the grades, homework assignment submission and class participation.

Estimated Model
The definition of the ordinal logistic regression model was studied previously by Wang et al. (2022).
The results from Rstudio are shown in Table 4.The estimated model can be written as:  The odds ratios and 95% confidence intervals of ACUE, HW and CP are as follows.
The first column in Table 3, OR, or odds ratios, explains how much more likely to have a passing grade when comparing students from classes with ACUE and without ACUE.For students who were from the class with ACUE practices, the odds of being more likely (i.e., "very" with a value of 2 or "somewhat likely" with a value of 1 versus "unlikely" with a value of 0) to have a passing grade is 1.92 times higher that of students who were from the class without ACUE practices, holding constant HW and CP.
For students who had good homework assignment submissions, the odds of being more likely (i.e., "very" with a value of 2 or "somewhat likely" with a value of 1 versus "unlikely" with a value of 0) to have a passing grade is 5.10 times higher that of students who did not have good homework assignment submissions.
For students with good class participation, the odds of being more likely (i.e., "very" with a value of 2 or "somewhat likely" with a value of 1 versus "unlikely" with a value of 0) to have a passing grade is 1.87 times higher that of students who did not have good class participation.

Proportional Odds Assumption
If the proportional odds assumption (Harrell, 2022) holds, for each predictor variable, the distance between the symbols for each set of categories of the dependent variable should remain similar.To help demonstrate this concept, the researcher normalizes all the first set of coefficients to zero so there is a common reference point.Looking at the coefficients for the variable ACUE in Fig. 1, it can be noticed that the distance between the two sets of coefficients is similar, suggesting that the proportional odds assumption holds for predictor ACUE.

Fisher's Exact Test to Check the Impact of ACUE on Class Participation
Fisher's exact test (Zach, 2020) is used to determine whether there is a significant association between two categorical variables.It is used when one or more of the cell counts is less than 5.In Table 5, it is noticed that there are two cells with less than 5. Thus, fisher's exact test is applied.From the Rstudio result, the p-value is 0.04, a statistically significant impact of ACUE on class participation.

Discussion
ACUE has been popular in education.Previous studies (Wang, 2024;BusinessWire, 2022) have shown promising findings of how ACUE has improved students' performance and class satisfaction.Wang (2024) noticed that in the same course, students' evaluation scores increased by 14% with ACUE practices compared with non-ACUE practices.A recent meta-analysis summarized (Business Wire, 2022) from Northwestern University indicates that students from ACUE partners can expect on average a 1 percentage point increase in course completion rates, a 3% point decrease in DFW rates and a 06 point increase in students' average course grades.One professor of anatomy and physiology at the University of Wisconsin system shared that after participating in the faculty development institute and adjusting her teaching approaches, the class average for the first test was 10% higher than ever before for the past years (Mangum, 2017).At the end of the term, the percentage of students who completed her course with a C or higher also increased by 10%.In this study, based on the comparison between students from the ACUE practices class and non-ACUE practices class, the results demonstrate a correlation between ACUE practices and students' grades.Students from the ACUE practices class performed better in class participation and final grades.In line with the hypothesis, for students from the class with ACUE practices, the odds of being more likely (i.e., "very" with a value of 2 or "somewhat likely" with a value of 1 versus "unlikely" with a value of 0) to have a passing grade is 1.92 times that of students who were from class without ACUE practices, holding constant all other variables.The class participation in the ACUE practices class was higher than the class without ACUE practices (p-value = 0.04).These results build on existing evidence that high-quality, systematic faculty training in pedagogy is associated with higher rates of student retention and graduation (Brown and Kurzweil, 2017;Jankowski, 2017).
The generalizability of the results in this study is limited by the number of students from the ACUE practice classes and non-ACUE practice classes.This research only collected data from one mathematics class over three semesters (2022, fall 2022 and 2023) with a total of 89 students.The limitations of this study are the following.First, the time period for a sample is small.A total of 89 students from two academic years is a relatively small period considering the time period of ACUE application in teaching.More semesters would be considered.Secondly, the time in this study can be a constraint on the research process.Three semesters might not be long enough to observe the changes from a new teaching method.Last but not least, is the selection bias.Students in this research are from Math 1325, a Mathematics class.Considering only one major class can lead to a biased sample that is not representative of the whole population at TAMIU.
Avenues for future research include promoting ACUE in other classes from different majors such as mathematics, engineering, biology, nursing and business school and increasing the sample size for both the experiment and control groups.Considering a longer experiment period, such as 2-3 academic years, should show better differences in students' performance and satisfaction between ACUE practice classes and non-ACUE practice classes.Also, attendance matters because if students fail to attend class, ACUE practices are not practical for students.Therefore, students' attendance can be included as a predictor of ACUE's impact on students.

Conclusion
This study analyzes the impact of ACUE effective teaching practices in an undergraduate mathematics course over three semesters by ordinal logistic regression ACUE HW CP Overall model.The results show that the ACUE practices have increased the odds of being more likely to have a passing grade when the comparison is done between students from classes with ACUE practices and classes without ACUE practices.Therefore, a recommendation is made that ACUE effective teaching practices should be implemented in colleges' teaching by tenure-track faculty members and non-tenure-track faculty members, such as full-time instructors and part-time instructors.
Ordinal logistic regression has many advantages over other models in the case of the ordinal form of the outcome (Lelisho et al., 2022).As an individual's or group's social position or class, socioeconomic status is determined by a combination of education, income and occupation.The ordered variable socioeconomic status has three levels: High, middle and low.Lelisho et al. (2022) concluded that ordinal logistic regression worked better than any other models.Among all the research studies done recently business wire 2022 (Morrison et al., 2017) about ACUE, this is one of the very few analyses done by the ordinal logistic regression model.Not only this research will contribute to the literature about the ordinal logistic regression model, but also it will share the impact of ACUE on college students from statistics.

Table 1 :
ACUE practice in class

Table 2 :
Distribution of each variable