Last week I integrated the “Interactive Media for Education” (IME, aka. CLAS) video app with our school’s student information system (SIS) so that the app can update viewer lists daily for media collections that are linked to courses. The API is a simple REST service, so at first glance I thought all I needed to do was to send a request, get the data, and follow the specs correctly to parse the data and update my database, with a healthy dose of unit testing to smooth out the edge cases. Even so, I just couldn’t start coding right away even if this project is still a one-person show and I’m strapped for time. That habitual need to take long walks and imagine all the usage situations that a new feature may go through just refused to let me go, and so I walked, and I prototyped. After half a week of tinkering and thinking about it, I realize that even automated systems have a usability angle that one must consider.
Who are the “actors” in an automatic enrollment list update system? The data source I talk to is one, since it represents the designers and programmers creating that system and also the organizational culture and policy that they work in. Understanding those involved with my data source allow me to consider edge cases unwritten in the documentation. For example, I remember hearing in passing from support staff that “courses number with a letter at the end sometimes have strange enrollment lists”. Sure enough, some, not all, test courses yielded zero students. I contacted the SIS team to hash this bug out, and realized that this was not a programming bug but an organizational issue, a naming rule inconsistency between the departments and the centralized body. Not a problem I can solve now, but I got the information needed to create a work around before the service went live. Continuing this line of thoughts, I imagined how the data source would behave over time, and what it “needs”. During the course registration period before each term, the back end database of my source would be hammered, so I should be polite and stop asking it for data nightly.
Another actor is the student using the IME app. The students never see the auto provisioning service, but they are affected by it. In the first month of term when the enrollment list is in flux, tension is high, and a student who just registered late for a course from a waitlist will feel that a one day wait for a video collection to open to them is as long as two, so I force the student list to update twice a day, slowdown or not. Also, I realize that I cannot anticipate all bugs, and all human errors that either I or many other staff involved with enrollment can make, so during this period the service runs on a “kindness” mode, where new students added to courses are updated, but students dropped from courses are not applied immediately, in case that student actually lost access because of a bug or human error. Would this then jeopardize academic integrity? To address this, I make sure that the “kindness” mode is only applied to courses where videos are only shared to everyone in the course. A “one to many” content distribution relationship implies that this is just a lecture video, while “one to one” or “many to many” or “many to one” would imply coaching, group work, or assignment submissions, situations where integrity is required and “kindness” must not be misplaced.
Naturally, at this point I wish for a service where updates are PUSHED to my video app, instead of REQUESTED by my video app. My university actually do have such a service. However, integrating with this push service will take more time, because it was purpose built for the centrally-managed Learning Management System (LMS) and was not meant to be simply hooked into another app. Getting the permission and arranging the details about testing environment and move-to-production process also takes more time. With this delay, the feature might not become useful to students for at least one term. More importantly, as the number of users increase and we are developing a more enterprise-scale support model, having scalable support for user provisioning is also a crucial feature to the service managers who are evaluating the app from an operation view point right now. I decided to go with the request method to create an immediate impact to users and stakeholders, while having a conservative development schedule to ensure production quality. Meanwhile, I communicated with the LMS team to express my interest in the next iteration of their enrollment list push system, and they have committed verbally to provide an external facing API in the future.
Finally, the needs of the the support team must also be considered. An automatic enrollment service needs a mechanism to register and deregister courses from automatic user provisioning, and the support team are the ones who will do this. This process should be seamless instead of adding to their already busy workflow when first setting up a course and then supporting it on requests from instructors, otherwise it will create another source for human errors, forgetfulness under stress, and delay, especially during that hectic first month of a term when students need robustness the most. Thus, I decided that there would be no extra UI for registering and deregistering a course for auto-provisioning. These actions will be integrated into the normal support workflow that the team had always been used to. When a new course is created by an instructor, if that instructor specifies that the course is linked to SIS, it is immediately registered for auto-provisioning. If the support team receive a request to bulk import enrollment from SIS, that action will register that course for auto-provisioning. If the enrollment list is then changed manually for any reason, using any methods in the admin interface like the Add / Drop tool or the CSV import tool, the enrollment list will be deregistered, so that the automatic source will not overwrite the manual addition. Support staff also often need to use test acccounts, so I included a reserved account list in the design and exclude these accounts from being changed by the automatic updates, yet adding and dropping these accounts from courses manually also do not remove the auto-provisioning status.
Last but not least, to minimize the risk caused by having such a short development time frame, every automatic back-end features of the IME, this auto user provisioning system included, have extensive logic to check for corrupted data, data that is technically specs-valid but seems unusual (too many students to be dropped, too many blank entries, etc.). The cumulative time to update all courses, the time for each course, and the time and error detection status for each part in the process for each course, are collected, evaluated for unusual patterns, and dumped into an email report that the developer and support team can monitor for the first few months of the feature. All this monitoring and error detection may seem overbuilt for something so simple, but this extra development time is to trade for a much bigger time saving. This method allows the team to treat the first few months of live service as an extended QA testing phase, by being able to notice when things fail and exactly what course, what step it fails on, even which users whose enrollment status are being modified at the time, so that the team can manually recover from failures quickly and at the precise point before users feel any pain.