Alliance for Catholic Education
Lewis Poche
This article is the final in our series explaining the HPL Adaptive Software Review Standards. We have identified 26 criteria categorized into six domains for evaluating software. This review framework addresses the question, "How do students learn online?" This resource aims to offer educators a more profound understanding of the online learning process for students when utilizing online adaptive programs. Read our two articles in the series,
- “Oops to Aha! Putting the "Adaptive" in Adaptive Software.”
- F.U.N. - Adaptive Software’s Role in Facilitating User Navigation
If you are already a member of the HPL Network, you can access the template for our software review standards by clicking here. If you still need to become a member, you can join for free by completing this form.
Furthermore, we have employed these standards to assess 25 commonly used adaptive software programs. To learn more about accessing these reviews or seeking guidance from our team about adaptive software, please refer to the HPL Adaptive Software Library page.
A Story Without Words
Once upon a time, Napoleon tried to invade Russia in the middle of winter. Let me show you the story using a famously brilliant image:
Source: Wikipedia
At a glance, you can see how disastrous this campaign was for the French. The width of the light grey line represents the size of the French army from Kowno (modern-day Kaunas, Lithuania) to Moscow, and the black line represents its size on its return. In a moment, this picture tells a tragic story of loss. Upon further inspection, it also includes the temperature measurements, distance, and route taken. Yale Professor Edward Tufte remarked that this "may well be the best statistical graphic ever drawn." For a data (and history) nerd like me, this chart tells a complete story, instantly.
What story might this image tell?
Even without classroom context, this screenshot from Zearn tells a story and provides teachers with important information that might inform their teaching. Which students are consistently meeting their weekly target? Which students have been struggling and might need additional support? Teachers can “read the stories” of struggle and success at a glance.
The HPL Adaptive Software Review Standards aim to gauge the quality of the stories adaptive software programs can tell and how teachers can leverage this information to inform their teaching. In this blog post, we will explore how adaptive software programs, such as Zearn and many others, collect diagnostic information from students, present actionable reports to teachers, and provide instructional guidance.
Research Basis for Data-Informed Teaching
In the following section entitled “My ‘Absolutely Delightful Racoon’ Dream,” we will outline the three HPL Adaptive Software Review Standards in the Data and Intervention domain. But first, let’s talk about why collecting, analyzing, and using data is essential in the classroom.
In his dissertation, Fr. Nate narrowed the broad definition of "blended learning" to reflect effective practices he observed in classrooms. He minted the term “powered learning” to mean “a form of learning where…teachers employ high-yield instructional practices and optimize their role, including utilizing data from the software in regular and thoughtful ways to make targeted interventions” (Wills, 2015). Effective adaptive software programs both collect and display data as well as provide resources for making targeted interventions. Data is typically collected through a benchmark or diagnostic test at the beginning of the year and a few times throughout the year (e.g., Fall, Winter, and Spring MAP testing). Additionally, many programs collect real-time data based on student performance, such as if they are passing individual lessons. The way data are represented tells a story itself; therefore, it is important that it is represented clearly. Lastly, the teacher-facing reports need to be actionable - streamlining the flow from identifying a need to differentiate to providing the appropriate supports to students.
Before we jump into the standards, it is important to note a subtle yet deliberate choice in our vocabulary. Typically, you hear about “data-driven decisions.” Some examples of data-driven decisions might be Netflix using customer data to determine which series it should purchase or Coca-Cola fine-tuning its marketing to drive sales. These examples give the term “data-driven decisions” a connotation that does not match up with how blended-learning teachers interact with data and their students. But, there is still something to be said about the power of a decision supported by data. With all this in mind, we prefer the term “data-informed decisions” as it reinforces that teachers, not data, have the last word in choosing what is right for their students.
My “Absolutely Delightful Racoon” Dream
In our last post, ChatGPT gave us the acronym “Really Smart Unicorn,” and DALL-E created an image of such a unicorn. Again, I turned to ChatGPT and DALL-E to help you remember our three Data and Intervention Standards: Actionable Data, Diagnostic Testing, and Teacher Resources. Could you imagine having an Absolutely Delightful Racoon as a teacher’s aide that analyzes data to help you make data-informed decisions? That would be delightful. Yet, for now, will simply remain a dream.
Actionable Data:
HPL Adaptive Software Review Standard Data.1:
The data provided to the teacher is actionable. For example, reports easily identify if a student is stuck and needs teacher support/intervention.
Software programs should intentionally represent data in a way that leads teachers to act. To give an example I hope none of you have ever experienced first-hand, I received an email from my bank titled, “A recent charge attempt requires your attention” this past Sunday night. Talk about the Sunday Scaries! Turns out, my credit card was compromised and someone had tried to pay for a Goodwill shopping spree on my dime! You better believe I clicked the big red no button in an instant.
Similarly, adaptive software programs should lead teachers to take action. For a program to meet our first Data and Intervention standard, it should clearly present student performance data that leads to meaningful intervention. One common example of this is a “stuck” alert. If a student consistently struggles with a topic (e.g., getting the majority of questions incorrect or taking a relatively long time to answer questions), the program might alert the teacher that this student might be stuck. This alert serves as a reminder for teachers to help the student personally by reteaching a topic. In some programs, such as i-Ready, teachers must dismiss these alerts before students can move on in their learning progression. At the very least, software programs must cleanly represent data such as student usage and lesson completion.
Diagnostic Assessment:
HPL Adaptive Software Review Standard Data.2:
The program provides an efficient (developmentally appropriate length) and accurate (representative of student knowledge) means of gathering baseline student performance data.
Standardized assessments offer snapshots of student performance and can inform personalized learning pathways.
My views on the saying “Test Day, Best Day” changed throughout my time teaching. During my first year, I selfishly loved test days because students were silent and otherwise occupied. Fast forward to my last semester in the classroom and you would find me borderline giddy on a test day. What an awesome opportunity to see growth and get current data to inform my teaching!
The ability to easily differentiate is one of the greatest benefits of blended learning. Unlike the conventional classrooms of yesteryear, teachers can give assignments based on an individual student’s readiness. To inform these decisions, teachers rely on baseline data from assessments. In many cases, this data comes from a standardized assessment such as NWEA MAP Growth, Renaissance STAR, or i-Ready Diagnostic. Many adaptive programs even pull data directly from these standardized tests (this webpage lists the programs NWEA Map Growth connects with). While some programs can not integrate with standardized assessments, most typically have some way of gathering this type of diagnostic data.
The common pitfall to be aware of here is over-testing. Students attend school to learn, not to be continually tested. While this data does provide crucial information for teachers, these tests can cause anxiety for many students. Not to mention, a class spent on testing is one less day spent learning. By limiting adaptive software programs and utilizing programs that integrate with standardized assessments teachers can minimize time spent testing. Implementing these strategies will preserve instruction time and student mental health. It is a win-win. Or, as I like to say, a way to feed two birds with one scone!
Teacher Resources:
HPL Adaptive Software Review Standard Intervention.3:
Useful teacher resources are easily accessible (linked to student "stuck" reports). Resources provide just-in-time supports students need to achieve grade-level proficiency or content to assist teachers in filling learning gaps.
Diagnostic Assessment, check. Adaptive Software Reporting, check. Now what?
Informed by diagnostic assessments and student performance data, it is time for a teacher to act. Teacher Resources refer to a toolbox of lesson plans and learning activities that teachers can easily employ to support students. Sadly, my “absolutely delightful raccoon” who prepares individualized data-informed lessons is a fantasy. However, some adaptive software programs provide the next best thing: a bank of ready-made learning activities! For a program to meet this standard, it must seamlessly link every lesson to activities that a teacher could use to re-teach a concept to or support a struggling student. Those activities could be mini-lesson plans teachers could use for their teacher-led small group station or independent work for students to complete before circling back with their teacher.
The Story Continues
These three standards are core criteria when evaluating adaptive software programs used in blended-learning classrooms. However, it is essential to note that every teacher, group of students, and school context is unique. These unique considerations might lead educators to prioritize certain capabilities and standards over others. However, as a generality, a program that meets these three criteria is likely to provide teachers with the data necessary to make targeted interventions.
Great teachers are looking for powerful tools to help all of their students and being able to see where they are and how to help them at a glance can empower them to help their students just when and where they need. Having actionable and diagnostic data coupled with helpful resources can be a game-changer for teachers who want to use technology meaningfully in their classrooms.
Since 2015, Higher-Powered Learning has empowered Catholic school teachers and leaders to leverage technology and related innovative research-based educational practices to meet the needs of all learners.
Subscribe to the "Higher-Powered Learning News" list to join us in this mission.
If you have any questions about Higher-Powered Learning, please contact our team at hpl@nd.edu.