As everything in enterprise UX, usability testing provides some interesting challenges. I thought I'd share a few issues and how I've gotten around them.
The Process Is Changing, So Make Sure Users Know It
First and foremost, in enterprise testing, at least in my experience, we are never testing a UI where the users know the work process, because the basis of the UI is a major change to the process or a new process is being implemented. This is a BIG difference between B2C and enterprise UX! In B2C on an e-commerce website, you can make the assumption that your users know how to shop.
This was a big hurdle for me. I never considered that the user didn't know the process. All the testing I had ever done or seen just presented the user with tasks and off they went. It took me a couple years and major testing failures to realize this. One of my developers kept telling me, "They need context! They need context!" And I finally got what context meant. The people we were testing couldn't test the UI because they didn't know the process they were expected to follow when they were using the UI.
Normally we do an hour of testing with our employees. But now, we make our business people do a presentation about the process change or new process with reasons. I have tried sending out information to employees beforehand, but our employees are simply too busy to really have time to read a lot of information before we meet them.
Employees Don't Know What A Walkthrough Is So Teach Them
My company has employees all over the world. While at least here in the US people are somewhat familiar with the idea of consumer testing, in other countries it is a completely foreign concept. I've particularly had this problem with employees in our Asian branches. I've set up a webpage on our UX website that we send out to managers through whom we are recruiting employees and to our participants that has an explanation of:
- what a usability walkthrough/study is
- what we will ask them to do during the study
- a very short example video of a task being completed with subtitles for non-native English speakers
- who will be present in the meeting
- what we will do with the information from their work.
Right now we have the information in a couple major languages and working on more, so our employees can read this information in their native language. Our first translations were done by a couple fellow UXers and additional languages will be done by some of our developers who have participated in our studies. We want them translated by native speakers who know what we are asking our participants to do.
This particular page is prefaced for managers that they must give time to their participant employees to read the information. And we send it out as a part of the meeting invitation telling them it is required reading in preparation of the meeting. It's also the first question I ask when we start. I haven't had to do it yet, but I am always prepared to read the explanation and show the video to make sure.
Even if the participant says they have read the page, I still give a short explanation with the example that it would be like us wanting to look over their shoulder while trying to figure out a new cell phone they have just purchased.
Use Real Data in Your Prototype or Your Participants Get Stuck
I've talked to other enterprise UXers and know this is a universal problem. You must use real data in your prototypes. Don't try making it up. If the data isn't right, your participants will get stuck on the data not looking right or being in the right state. You will not be able to get them unstuck and go any further in your test. And trying to get them to imagine also doesn't work.
Get Your Participant List Upfront and Don't Allow Last Minute Substitutions
My company is made up of branches of different sizes worldwide. When we go out for participants we try to make sure that we are getting people from a small, medium and large branch and we want people who are mostly low level workers with one or two supervisors.
No matter what you do and you cannot avoid it, you will have to include managers as part of that list, even though they are not users of your application. Consider it good will. Sometimes I will use these managers as my test of the test and they are always present for the test. It's very rare to have just the participant from the branch participating in your test.
But more importantly, don't let a manager substitute someone last minute for your original participant. I've found more often than not they are also not representative of the user group I'm trying to have test the design. To avoid this I also make sure I'm the person sending out the meeting invitation. That way I'm the person getting the acceptance receipts back. If I don't have a receipt from the recipient, I always double check with them. I also keep an eye on their calendar so I see if something comes up for them.
Make Your Team Your Collaborators
As the single UXer on an agile team, it is always difficult to run usability study and not be overwhelmed by it. The trick to it is to get everyone on the team involved. Not only will you get buy-in to the tests, but you will find that others will experience your pain points and help you come up with for fixes. I've done this step by step, honestly by accident. Now it's part of my testing process and I make all my teams follow it.
How did I get here? First, my business analyst volunteered to work with business to identify managers to talk to for participants. She has now recruited the program manager to do some of that work and I've gotten my program manager to find conference rooms for our studies once I've set them up.
I usually use my design prototype for our application. But I always have to pull somewhat recent data. I went on vacation one time and in a final email asked my team to go into our legacy systems to pull data for me that I could then stuff into JSON files (JavaScript). About 5 people worked on this for me and found out how painful it is, so that I have a little subgroup working on pulling the data for me in a form I can use and/or manipulate.
In a different group, one of my developers understands how our study participants get stuck on data and he always has a development prototype ready for us to use.
Everyone in the group is required to attend one or two studies depending on how many we are running. We always run one or two in the evenings with Asia and sometimes late night with India and usually my business analyst and business owners are the ones who show up to those.
When everyone pitches in, you're looking at about 5-8 hours of work per study for yourself, mostly depending on how much work you have to do to prepare your prototype and whether you record and review the videos of each study. If I have to prepare the (HTML/JQuery) prototype and data, that is usually 2 to 3 weeks of preparatory work.
If you've been working in enterprise UX, do you have any tips?
Comments
Post a Comment