When Users Curse Your Name
Classroom observation gone wrong right
Soon after I was hired at Pearson (now Savvas), I was tasked with helping administrators register on our LMS. While you can see that flow here, it was far from the most interesting discovery I made in my time there.
During a classroom observation, students had physical copies of their books while the teacher projected the content on the smartboard.
As the teacher reviewed one of the exercises with the students, the screen suddenly refreshed to her login screen.
Immediately, the students and teachers raised their fists to the sky and shouted, “PEARSON!!!”
Correcting pain points
The teacher then walked across the classroom to her computer, logged back in, painstakingly renavigated to the section of the lesson she was in, walked back to where she was, and picked up where she left off.
When I returned to the office, I immediately informed my team about the experience. We discovered that 1.) the logout was due to a session timeout, and 2.) it was a known issue. It had not been solved, however, because the policy of logging them out after 4 hours was “more secure” than logging them out after the school day (8-10 hours).
I spoke with the UX and engineering teams to figure out the risks to adjusting this policy and their significance in comparison to our users cursing our company’s name. In the end, the visceral story won, and the time-out policy was modified.
Stories and observation are powerful
Stories like these (or other anecdotes from user testing) are powerful motivators in convincing stakeholders to take action and make decisions.
In-person user testing is invaluable.
The Failures of Mind-Blowing Content
Getting inspired
Many competitors, news sites, and other resources are using sleek, magazine-like parallax pages to show content and bring stories to life. We wanted to do the same with our graduate-level content. After all, students are up to 22 times more likely to remember a fact when it’s wrapped in a story.
Something we were proud of
We spent a lot of time creating a sleek, custom parallax articles for a series of courses. The faculty adored them and we were pretty proud of them too. So we embedded them into the university’s learning management system (LMS) using iframes and anticipated wild-eyed delight from our students.
Low engagement
Faculty soon reported that none of their students recalled the story presented in the parallax page. Diving into the Blackboard metrics, we quickly discovered that they were right – virtually none of the students had even seen our beautiful narrative.
We worked with the university to find three students willing to talk about the course with us. With probing questions, we quickly discovered that students skimmed over the iframed content was an ad.
Digging deeper
We tried several visual methods to call attention to the parallax content, but the problem persisted; students still skipped it. Faculty became frustrated, and so did we.
We did another round of testing with more students – this time allowing students to walk through their thinking a bit more. We soon realized that it wasn’t an issue with the UI, but an oversight of the graduate student persona: these busy students are juggling a full-time job, family life, and other responsibilities, and this graduate course is just another responsibility. They’re going to cut corners wherever possible, and I can’t blame them.
The parallax content wasn’t required reading for an assignment or exam, so students skipped it.
Several iterations
It didn’t matter how sleek the content was if it didn’t reach the user’s end goal of quickly completing graded assignments. The parallax articles – though interesting – weren’t relevant to the user. It was was a classic case of failing to understand our persona’s needs.
The faculty was still very attached to the content we’d created, and thought it illustrated a lot of essential principles for the course. To ensure students engaged with the content, we created an accompanying short quiz. As a result, engagement with the content increased by 87%.