To enable a richer set of immediate feedback options in the Open edX platform, the Open Learning Initiative at Stanford University collaborated with edX and the Open edX community to add two new feedback mechanisms across all of the basic problem types in the platform.
The Open Learning Initiative designs courses based on learning science research that, in turn, contribute to that research. One of our primary areas of focus is exploring how technology enables new learning pathways that are difficult or even impossible in the traditional classroom environment. We have seen significant benefits from the use of hints and feedback in existing OLI courses, and are excited about helping to bring those benefits to the Open edX platform.
One of the key takeaways from the field of cognitive science is that people learn from active engagement. Learning-by-doing, and in particular, goal-directed practice accompanied by targeted feedback, is a particularly effective form of achieving active engagement in many disciplines (Ambrose et al., 20101).
Online learning offers significant opportunities to construct environments where students can engage in this type of goal-directed practice. It is now possible to create rich, interactive activities that allow students to engage directly with concepts and receive immediate formative feedback in ways that are difficult or impractical in traditional classroom instruction.
Feedback Given Upon Answer
With the changes recently introduced to the Open edX codebase, it is now possible for instructors to write feedback that is associated with each answer option for both correct and incorrect answers.
This feedback is presented to the learner when the answer is submitted. This enables the instructor to author questions that include “distractors”, which are incorrect answers that capture common misconceptions. When a learner struggling with one of these misconceptions chooses the associated distractor, she receives immediate feedback that corrects the misconception. This is illustrated in the simple example below, where the learner has the common misconception that a tomato is a vegetable rather than a fruit.
Research data collected about learners using OLI courses show that, after correctly answering a question, they will often click on other answer choices to read the feedback associated with those choices. For example, the learner may narrow the set of answer choices down to two, but not really have any idea which choice is correct. She clicks on one, and it turns out to be the correct answer, but clicking on the other possibility and reading the corresponding feedback teaches her why that choice was incorrect.
Similarly, wherever possible, feedback for correct answers are given for the same reason – a learner may have guessed the correct answer, but not really understood why it is correct.
Learner-Requested Hints
It is also possible for learners who are having difficulty with a problem to request help in the form of one or more hints. For some problems, a single hint may be sufficient. In more difficult problems, several level of hints can be provided. This kind of help is especially important in free-form questions such as text input or numerical input. In these types of questions, learners are not able to guess at the answer and may struggle to find the path to the problem solution.
Summary
One of the promises of online learning is that it provides new pathways for learners to learn by doing. Content that might be traditionally presented as text or video can instead be presented to the learner as an activity that has them interact with the content. This can be an advanced activity like a cognitive tutor, or as simple as a multiple choice question constructed with help in the form of hints that can be requested by learners who need them, and feedback that addresses common mistakes.
Herbert Simon, Nobel Laureate and one of the founders of the field of cognitive science, said that “learning results from what the student does and thinks and only from what the student does and thinks. The teacher can advance learning only by influencing what the student does to learn.” One of the advantages of the Open edX platform is the rich set of learning activities that it provides. By extending these activities to provide hints and feedback, they become even more powerful.
To find out more about how to use these new features, check out these resources:
The hinting and feedback work detailed in this post is now available for all problems authored on edx.org. The code is present on the master branch of edx-platform, and will be included in the codebase for the upcoming Cypress named release.
Update: The hinting feature is well instrumented with plenty of analytics events for researchers! Please see our documentation on event logs for more detail.
1Ambrose, et al., 2010. How Learning Works: Seven Research-Based Principles for Smart Teaching.
The Open Learning Initiative (OLI) is a grant-funded organization that offers innovative online courses to anyone who wants to learn or teach. Our goal is to create high-quality courses and contribute original research to improve learning and transform higher education. OLI work on this feature was funded by the Bill & Melinda Gates Foundation.
This project was a success due to the hard work of many different people, from concept design to implementation to review. Thanks specifically to the following individuals:
OLI
-
Nick Parlante (primary software engineer)
-
Thomas Brennan-Marquez (secondary software engineer)
-
Ross Strader (project director)
edX
-
Sarina Canelake (project manager, engineering review)
-
Will Daly and Dan Friedman (engineering reviews)
-
Mark Sadecki (accessibility review)
-
Frances Botsford (UX support)
-
Lyla Fischer (working group member)
-
Piotr Mitros (working group member)
Stanford
-
Jane Manning (working group member)
-
John Orr (working group member)
We also thank the Open edX community for feedback on the feature at various points in the development.
