When discussing Learning Management Systems, I blogged about the types of data an LMS can provide and how a teacher can use data to guide instruction. Now, as I focus on evaluation, I want to return to analytics to show some examples of data and how teachers can use it to evaluate. I’ll look at three different sets of information covering enrollment data, observational data, and student data.
Enrollment and Attendance
Enrollment data will help the teacher anticipate the size of the class. This seems basic, but the number of students will help the teacher determine how much need there is for the class and how popular the course’s topic is. Figure 1 shows that more students take Economics than American Government or AP Microeconomics. If need for Economics grew, a school might consider offering an additional section. Also, since the total number of students taking an economics course is 72, I know that if AP Microeconomics were dropped, another Economics class would need to be added.
Attendance data tells more about the individual students. For example, Figure 2 shows that there are a few gifted students, so the teacher would want differentiated assignments for them. Also, the teacher can see what day students viewed the orientation video. I find it unsettling that there is a large gap between the course’s start date, listed in Figure 1 as January 7, and the date many of the students watched the orientation video. I wonder if the students felt that the video assignment was not important, or if the teacher needs to communicate better with students at the beginning of the course. Lastly, this data may indicate how dedicated a student is to the course. For example, the last student watched the orientation video in February! Either the student enrolled late, or that student has not been working diligently. Were I the teacher, I would reach out to that student and ask what’s up.
Observational data includes both notes and observations documented by the teacher as well as graphs that show student engagement.
Figure 3 is a snap shot of a teacher’s communication with a single student over a period of time. The report shows that after three weeks of the course, the teacher reached out to the parent because the student has a 24% in the class. The communication log also shows the method of communication. This is important because it allows the teacher to track which forms of communication work best. If there were no change after the last voicemail, the teacher might consider emailing instead of calling. Teachers should be specific when writing comments for better analysis and reflection afterward. For example, the last entry only describes the student’s grade as “below a 70%,” so I wonder if the student still has a 24%, or if he/she raised it.
Throughout the course, the teacher can also view reports that show student progress. Figure 4 lists the individual students; the left column in gray shows completed assignments / total assignments, and the remaining columns show the number of kilobytes the student used within the LMS.
First, the chart shows that the course began September 4, and the graph only shows progress through September 17. Teachers should always remember the timeframe each chart displays so they avoid jumping to conclusions about student success or failure. For example, we see the bottom few students have not completed many assignments and not accessed the LMS much, if at all, but since the course still has weeks remaining, students still have opportunities to improve their grades. The teacher should contact these students immediately and find out why the student is progressing in this manner. A teacher may also want to contact the student third from the bottom and ask how he/she was able to complete so many assignments (26/59) only after five kilobytes of information. Perhaps the course is too easy, the student has background knowledge in the course, or the student is using another source for information.
This information also reveals a little about individual work habits. The top two students access the LMS and are very engaged with the content, but the third from the top student accesses the information less frequently while still completing many of the assignments. Either these students are very dedicated to the course, or they have a lot of knowledge in subject. The discrepancy, however, between the top students who have completed most of the assignments and the bottom students who rarely access the LMS, tells me that the most students are not working at the same pace. If I were teaching this course, I would build in more synchronous sessions and require students use the discussion boards to communicate with their peers. This might give some students the support they need while allowing the high-achievers a chance to showcase their work and take on leadership roles within the digital community.
Figure 5 is a sample student report that shows assignments, grades, and teacher feedback. For the first assignment, the teacher offered some suggestions on how to improve the assignment, and this challenge may have motivated the student to try harder on the last assignment. The student’s high grades indicate that the student cares about his/her grade, so the teacher may question why the second assignment is incomplete. Perhaps the assignment itself needs to be more engaging.
Figure 6 shows the feedback a student receives on dropbox assignments. Students can access and use this information to improve their work.
When taken together, this data shows most students progress well throughout the course, but there is definitely a lack of continuous engagement. The activity chart shows it is possible for some student to complete almost half of the assignments while only accessing the course a few times. That lets me know that the teacher needs to build more community among the students since they don’t feel the need to check in regularly. The teacher could fix this by incorporating more discussion-based assignments or synchronous events. It may be too late to make these changes to impact this course, but the teacher should consider redesigning the course in the future.