Interview with AOL UK
Mark Uttley, Senior Research Manager at AOL UK, talks to the UK UPA and discusses his approach to working with the usability community.
Mark Uttley
Senior Research Manager, AOL UK
The Big Picture
UPA: What is usability for you and your organisation?
MU: I prefer to call it ‘user experience’, within which usability is a sub-category. User experience for AOL UK involves all the elements that contribute to a positive experience when using our content channels and products, from strategy through to execution.
UPA: How does usability fit with other design components?
MU: We conduct user experience research during all stages of the design process. Roughly speaking, we divide research techniques between ‘formative techniques’ that help inform the design brief, and ‘summative techniques’ that inform the design iterations that we pass through once design is underway. How much of each type of research we do depends on the importance of a project to the business, the complexity of what we are designing, and the timelines available.
UPA: What do you see as the difference between usability and market research?
MU: I think the differences between market research and usability and over-estimated. Many of the techniques used in user experience research are essentially adapted market research techniques (e.g. mood board construction, card sorting). I think it is important for user experience agencies to consider the issues normally addressed by market research agencies more often. For example, user experience research often fails to sufficiently consider branding issues. AOL UK’s content and products have to deliver our brand promise effectively, not just be easy-to-use. This requires that user research agencies have the appropriate techniques to assess whether a content channel or product is in line with our brand positioning. More often than not, the techniques for this (such as sentence completion or personification) come from market research.
UPA: What is the biggest usability myth?
MU: I think the biggest myth for usability is that it is just about ensuring simplicity and ease-of-use. If you think of it as user experience, it becomes a much bigger and more sophisticated discipline, with a wider range of techniques than the classic 10-person laboratory test.
Approach and Methods
UPA: What methods do you regard most highly?
MU: I regard formative user experience research techniques more highly than summative techniques – such as competitor benchmarking, requirements capture, and Information Architecture. A project has the best chance of being successful if research and planning are conducted at the start of the process. By doing enough work up-front, you avoid having to do drastic redesigns on hi-fidelity prototypes close to the launch date.
UPA: Which methods are the least effective?
MU: The least effective methods are last-minute usability tests conducted so close to a launch date that there is little time to make any major amendments. User research must be done throughout the design process to ensure major issues are identified early enough for there to fix them.
UPA: What is the best outcome from usability inputs you could wish for?
MU: My dream outcome for a user research project is a significant uplift in both quantitative and qualitative metrics. This means usage levels (such as reach and frequency) go up, and you hear back from users that what we have launched has a ‘wow’ factor.
UPA: What should practitioners seek in their deliverables?
MU: Deliverables should be easy-to-understand and highly actionable. Agencies should avoid using arcane HCI jargon (e.g. mental model, widget, cognitive workload), and user research reports need to be easy to scan through and absorb. Reports also need to clearly summarise the recommended ‘next steps’ with details on what needs to be done as a result of the research. Day to day design and editorial staff are simply too busy to try and decipher hard-to-understand reports.
UPA: What are the three biggest problems you have seen in usability work?
MU: The biggest problems are undoubtedly:
a) Educating internal clients about user research – due to the newness of usability as a discipline, there are varying interpretations of what it involves. Some clients believe it just involves classic usability testing prior to launch of a product, and some recognise that it can be throughout the design cycle, and incorporates a wider range of techniques. You need to work with clients to ensure they appreciate the breadth of user research available.
b) Ensuring that user research is acted on – some of the recommendations from user research can be very challenging, and it can be tempting for a project team to just take care of the easy-to-fix issues. Trying to ensure all recommendations are acted on, even the most difficult ones, requires an appropriate process.
c) Monitoring the ROI of user research – this is something that user research agencies often overlook. Most user research agencies don’t really have best practise advice on how to measure ROI, and benchmark ‘norms’ for how to assess ‘good’, ‘middling’ and ‘poor’ ROI. This should be addressed as a critical issue within the industry.
UPA: How do you look to measure usability returns?
MU: We look at a variety of metrics, but the main ones are:
a) Overall usage figures – reach, frequency of usage, and time spent within a product or channel.
b) Usage of specific features – uplift in usage of key features within a channel (e.g. a booking tool).
c) CTR, sales & registration rates – uplift in click through to partner sites, sales and registrations on partner sites.
d) Subjective metrics – pre- and post- redesign surveys can measure uplifts in metrics such as satisfaction and brand perception.
e) Changes in audience demographics – usage reporting can show whether an audience for a channel or product has broadened out of its traditional base after a redesign.
The Bottom Line
UPA: How usability was sold to you in the first place? – What was the “elevator pitch”?
MU: We didn’t need to be sold it, we know how important it is.
UPA: What makes a good agency/freelancer/employee?
MU: A good user research consultant ‘learns’ the company he is working for. By this I mean learning its most frequent user experience issues, learning how decisions are made within that company, and learning how to communicate in the right way to the right people to ensure user experience findings are understood and acted on.
UPA: What do you look for in a practitioners’ project proposal?
MU: I want to see detail in a proposal. Practitioners should meet with a project team and intensively interview them about what they want to cover with a user experience project, and what they expect to get out of it. These details should be in the proposal and signed off before a project gets underway, so everyone is ‘singing off the same song-sheet’. Of course, itemised costs, timescales and project team details also need to be in the proposal.
UPA: How do usability costs compare to other services?
MU: User experience research is expensive compared to other types of qualitative research. Consultants tend to be highly specialised, therefore have higher daily rates than other types of research agencies. And, often, respondents are interviewed separately, rather than as a group – which makes projects longer and more costly.
In a Nutshell
UPA: What are your favourite usability books/resources? Why?
MU: ‘The Elements of User experience’ by Jesse James Garrett (New Riders Publishing), because it covers in a very simple way the different elements that go into good user experience.
Alan Cooper’s ‘The Inmates are Running the Asylum: Why Technology Products Drive us Crazy and How to Restore the Sanity’, because it outlines a ‘philosophy’ that I subscribe to – technology products need to be for everyone and not just technology obsessives.
UPA: What is your favourite usability quote?
MU: "Most software needs to be spanked" - Alan Cooper.
UPA: What is your favourite famous usability finding?
MU: I love the automatic tuners that programme your TV for you, without you having to do it manually. Good usability in a nutshell – it takes the load completely off the user.