Greetings! My name is Colin Fredericks. I work at HarvardX as a Senior Project Lead, which means that most of my job is helping Harvard professors make quality courses. However, over the past few months I’ve been working, mostly in my spare time, on adding partial credit to the Open edX platform.

I really like the Open edX platform, but I was always surprised that there was no way to award partial credit. Having taught college and high school it seemed like such an obvious oversight that I kept expecting to see it on the “upcoming features” list. After three years of waiting I decided to finally take matters into my own hands.

Now, I’m not a professional programmer. (You may hear sinister music playing in the background. We call this “foreshadowing.”) I learned Applesoft BASIC in 4th grade, TI-82 programming in high school, C++ in college, Python while working at MIT, and Javascript while I was unemployed, but I had never contributed to an open-source effort before. I had never written test code. Heck, I barely knew that test code existed. (A shadow falls over the camera. More foreshadowing.) I decided that I was going to add one kind of partial credit to just one kind of problem, just keep things simple so I could handle it in a week or two. (This, too, is foreshadowing.)

That was early April. I accomplished my original goal in two days, and then the project rapidly inflated into its current state. I was convinced to expand from a single problem type to as many as I could reasonably handle, and to write up a full-fledged proposal that ended up being 16 pages long. 

While implementing this feature, I found out about this “test code” stuff, and “code quality checkers,” and other things that they (wisely) don’t tell people about in Computer Science II. I started from the edX Developer’s Guide, as best I understood it, with its pretty flowchart. I opened up a proposal ticket on JIRA, made a github repo for my own code, opened up my first pull request, did some more development, got a lot of good feedback (most of which I incorporated), learned how to rebase properly, opened a new and better pull request, and did some more coding. Finally, after about 15 weeks of development, I’m proud to say that the feature is merged into the edx-platform master branch!

Partial Credit: Feature Details

Here’s the quick rundown of what this feature currently looks like. Note that currently, authors can only enable partial credit by directly editing a problem’s OLX definition.

Multiple Choice and Dropdown problems have one type of partial credit available: “Points” style grading. The course author lists certain choices that are worth partial credit, and optionally sets how much credit they’re worth (the default is 50%). As a side effect, Dropdown problems have been upgraded so that more than one choice can be marked as correct.

Checkbox problems have two (mutually exclusive) types of partial credit available: “Halves” style, in which a perfect response gives 100% credit and each mistake cuts the credit in half, and “Every Decision Counts” style, in which each checkbox effectively turns into its own separate true/false question.

Here is an example of how a course author would enable the “Every Decision Counts” (or “EDC”) style partial credit in a Studio problem:

The OLX definition of an EDC checkbox response partial credit problem. See the documentation for more details.

This is how the problem will appear to students:

EDC checkbox problem with 3 true answers and 2 false answers. Checking one true answer and one false answer yields a total score of 0.4 out of 1 point.

Numerical problems also have two types available: “List” style, where the author lists answers that count for partial credit (for example in a math problem, partial credit answers might be ones that result from missing a negative sign or inverting a fraction), and “Close” style, that expands the given tolerance to allow for a “horseshoes and handgrenades” approach. Since the two styles are compatible, you can use both of them in the same problem if you like.

Here is how a course author can define a numerical problem with “list” style partial credit:

Here is how the problem will appear to students:

A numerical response partial credit problem. The correct answer is 93*10^6 miles; accepted for partial credit is the value in kilometers. This yields a total point value of 0.5 out of 1 point.

Finally, Custom Python problems can now return not just a point value, but an indicator of whether the problem is considered fully or only partially correct.

For more details on how to add partial credit to your problems, see the partial credit documentation.

I’m very pleased with how all this came out. The project took longer than I thought it would, but it’s also much more powerful. It provides a framework that makes it easier to add partial credit to other problem types in the future, something I really hope people will take advantage of. 

What’s coming in v2? 

Well, first, I have a v1.5 planned that cleans up some of the surrounding code, so that future contributors to the Capa module won’t have as much trouble as I did. After that – more problem types, of course! I’m also looking at making this feature available for common problems without using XML – that is, giving instructors the ability to use this feature from the Studio markdown editor. There’s some additional code generalization that could be done. Somewhere in the future there’s also the possibility of adding attempt-based grading… but let’s take things one step at a time.

I can’t say enough about how wonderful and helpful all the folks at edX were throughout this process. I came in with a smattering of Python and Javascript knowledge, and came out with much more solid skills and an understanding of what it’s like to contribute to an open-source project. Sarina Canelake, Leslie Gerhat, Piotr Mitros, Mark Sadecki, Marco Morales, Ben Patterson, Diana Huang, Mark Hoeber, and more took time from their busy days to help me better understand and polish my code. I can’t wait to see it used in the wild.

I still don’t think of myself as a professional programmer, but thanks to this work I’m getting closer.

Colin Fredericks is a Senior Project Lead at HarvardX. In the rest of his spare time he enjoys cooking, writing roleplaying games, and being friends with all the animals.