In early 2009, Honda (UK) rethought the part of its Web site aimed at used-car buyers. The previous site used the common dropdown menu page design where users pick their model and price range to come up with available used cars. A new design was created to make the search experience easier and more intuitive to use, and to distinguish Honda from competitor sites.
QRCA member Jamie Hamilton, managing partner of U.K.-based research firm Nqual, worked with U.K. research agency Simpson Carpenter to conduct the qualitative research for Honda (UK). The team combined Internet-based research and application-sharing technology with the aim of unobtrusively observing and capturing natural online behavior.
The methodology is simple. Participants are given a single or set of online tasks to perform on their home computer. Immediately prior to the session, the moderator calls the participant to get a sense of how they expect they will perform the task. The participant is then left to perform the task(s) uninterrupted while the moderator, usability expert and client team remotely observe their journey. Immediately following the task(s), while the participant’s experience is fresh in the mind, an online recording of the journey is played back to them with the moderator and client teams navigating to, and asking questions about, points of interest they have tagged during the session.
Of the sessions Hamilton’s team conducted for Honda, half of the participants were Honda owners who had purchased their used car using the old Honda Web site. The other half were people who had purchased any car from another Web site. Users who had purchased their last car on the site were asked to “buy” their old car on the new site and compare the user experience.Users who had purchased their existing car from another site were asked to buy the exact same car on the Honda (UK) site and compare the experience.
“We call our approach ‘e-thnography’ because although our context is online, we have the same goal as classic ethnography - to non-intrusively observe as close to natural behavior as possible,” Hamilton says.
“Unlike standard usability testing, participants were not in an unfamiliar research facility or testing lab - they were in the comfort of their own home, at their own computer. Unquestionably, observer effects, moderator interjections and unnatural environments and procedures distort findings. When seeking to understand any natural performance - skiing, dancing, stand-up comedy - it is counterproductive to conduct the study out-of-context and continually interrupt the performer mid-flow to ask questions. It is for this reason that we capture the participants’ journeys and reactions remotely, whilst tagging points-of-interest en route. Only when they’re done do we explore events and experiences with them by reference to a shared online recording of their journey.”
Honda (UK) is getting essential feedback on the new used-car selection tool. It is making a range of usability tweaks and attaining an understanding of the higher-level preferences, motivations and dispositions of a variety of Web site visitors and users. “To give one example: amongst participants who set their own starting point for the exercise there emerged several categories of behavior based on the level of patience and method they demonstrated,” Hamilton says. “These categories employed different search engine techniques, reached the tool by various routes and landing pages, and had distinctive approaches to absorbing site information and learning the ropes. Findings such as these showed that improvements to search engine optimization, and tweaks to Web site design and usability, should be catered to different behavioral categories operating in a wider context, as well as expected user-end goals, and statistical findings.”
This technique can offer cost efficiencies over doing a study in a usability lab, which requires the renting of venues and equipment and potentially includes travel costs, Hamilton says. “Doing everything remotely, online, is also very fast. Within four days of commissioning the project, we found the participants and we were conducting our first sessions. Transcripts and recorded files were available immediately for analysis, and the process was further accelerated by being able to trial suggested improvements and theories with later participants,” he says.