Quick and Easy UX Methods for Anyone

Speaker Notes from Presentation at Duke Tech Expo 2019

These are simply the speaker notes that go along with my presentation slides. There wasn’t a good way to include them in the slides themselves. A number with a dash at the beginning of a paragraph signals a new slide.

It’s possible that I could edit this to make more sense as a stand-alone article, including information from the slides themselves. If you are reading this and would like that, please comment on this post or reach out and let me know.

1 — Hi there, I’m Jake Geib-Rosch, from the Fuqua Software Development Services team. I’m going to spend a little bit of time discussing how we as software or website creators can talk to our users and gather useful feedback about what we’re making so we can make it better.

2 — So I’d like to start by asking a super basic question: what is it that we do anyway? The obvious answer is... ‘well, Jake, you just said we make software or something like that right? So, that.’ And I mean, yes, you’re right.

3 — But bear with me a moment as I dig a little bit deeper. What is it that we really do? We help people accomplish their goals.

Now, you’re all smart people here. You’re not here to hear me talk about obvious insights like this that sound high-minded, but really we all figured out like 6 months into our professions.

So why the hell am I even mentioning this then? Because I think it’s important to begin this discussion with some perspective. We’re all busy people. We’ve got things to do. Taxes to get done, family to attend to, self-evaluations to write, lunch to get… and after a couple years… or decades… or let’s face it, even just a couple of days… it’s easy to get lost in the details. How 3D this button looks, what shade of blue the logo is, stuff like that. We do this for a long time and for better or worse, we develop opinions. Opinions based on our experiences, and dammit, our experiences matter. And I’m not here to tell you that your experiences or opinions don’t matter. But what I am here to tell you is that in the end we need to focus on this: helping people accomplish their goals.

4 —So here’s where it gets tricky. Different people try to accomplish their goals in all sorts of weird and wacky ways. Ways we never thought about. Ways that we’re not comfortable with. Ways that we think are ridiculously inefficient. The key here is to remain humble.

And the closest thing I have to one weird trick to staying humble is to remember the usability mantra, “you are not your users.”

A quick story from my own work. The largest single platform I support is Fuqua’s intranet site called FuquaWorld. Around 2014 we did a major overhaul of the site, bringing it from the type of ten-year-old table-based design that are probably still floating around the university’s servers, and making it modern, organized, and responsive. The vast majority of the feedback was positive. There were some growing pains, but in general this site was far superior to the old site.

The thing is that our students turn over every 2 years. That’s all the longer they’re with us. At this point the site has been live something like 3 or 4 years. So to students, our shiny, new intranet is Very Old News. It’s the same thing that was here when they got here.

When I first started hearing criticism from newer students about the site, I’ll admit that it was pretty deflating. But I took some breaths and tried to understand that their point of view is not only valid, but it’s the only way that this site will get better. At this point I’m too close. I spent two years going around the building trying to find out where things should be placed on this site and why. That was a necessary part of my journey that colors how I see the project. These students never did that, but it’s critical that the site works for them.

“But these students don’t care about the project like I do!” Well, yeah, but they shouldn’t. What they care about is doing well in their courses, getting a degree, and getting a job. That’s their goal.

More important than where I placed what buttons and what text—I have to care about helping them meet their goal. And that perspective can be easy to lose.

Alright, touchy-feely-kumbaya over. Now we get to helping people meet their goals efficiently and effectively, and to do that you have to understand how they want to do that. So you have to -gasp- talk directly to them.

5 —Given that we’re constrained by time, budget, and access to users, how can we effectively get that feedback?

There are a great many techniques for gathering user feedback, but today I’m going to focus on four methods that require relatively little overhead: Surveys, Post-It Sessions, User Interviews, and Guerrilla User Testing. These all have their own plusses and minuses, and are good at gathering different feedback.

So let’s discuss these four Quick and Easy UX Methods for Anyone.

Method #1 — Surveys

6 —Alright, now time to tell you what a survey is.
Just kidding! You know what a survey is! It’s just asking people questions in a web form. Next topic!

7 —

1. One of the best features of surveys is that you can do them at any point during your process to gather feedback.
2. You can do one before you start working to understand your users and their goals.
3. You can do them after launching your site to learn about how it’s working out for them.
4. You can conduct them over time to understand how the usability of your system changes over time, or to find ideas for improvement, and prioritize work.

8 —What is it you’re trying to learn? This will help you focus your survey on your end result. Are you trying to learn where your users are getting stuck? How they prefer to work? If they’re satisfied? Are you trying to gather ideas?

This goal will help you write your questions. Focusing on this goal will help you keep your survey concise, which will drive more and better responses.

9 — Where will you find respondents? The higher-quality relationship you have with users, the better the odds are that they’ll consider helping you. Every mass email you send, and every survey you ask your users to do makes a little withdrawal from the goodwill bank. So send fewer mass emails, and personalize your requests as much as possible. Incentives also work to improve response rate, so if your survey is important, consider adding some incentives into your budget.

10 —What tool will you use? Qualtrics is common here at Duke, but Google forms is a solid option as well. These options will also affect how your data is collected and stored. Custom solutions would need you to look a bit harder at this, but if you’re making a custom survey platform, you’re definitely outside of the realm of ‘quick and easy.’

11 —Who will analyze the data? Probably you, but it’s good to have a firm understanding of this up front just in case someone else wishes to be included.

12 —A common misconception about surveys is that you can gather numbers or scores on given features of your product and that’ll tell you what you need to know in a final, objective fashion. You can’t argue with numbers, they have an exact meaning. 4 stars is objectively better than 3 stars right?

The issue here is that numbers will tell you what is occurring, sure, but it won’t give you insight into WHY it’s occurring. For that you need to ask open-ended questions. To get to the why, you have to ask someone why and listen to their response.

If you want to gather information about the what and the why, you’ll need both types of questions.

13 — It is certainly true that numbers can be useful. They’re easy to compare and they can provide real data-points that can show what’s happening. Sometimes all you need to see is what’s happening to take action on it. In that case, there are a few keys.

The first thing you should know is that the mental energy required to answer questions like these is real and it’s limited. It’s referred to as ‘cognitive load.’ In order to get people to answer more questions, the less cognitive load you need to ask of them on each question.

Another thing to keep in mind is that there’s a limit to the usefulness of a rating scales, and that limit is about 5 levels. If you’re asking your users to rate your features, how different really is a 6 and a 7? I’ve seen new Qualtrics surveys that even allow you to set a rating on a continuous scale so you can end up with decimal points. Remember, just because you can, doesn’t mean you should.

Finally, don’t ask a question if you can gather the answer from analytics. Treat your users’ goodwill as precious currency, because that’s what it is.

14 —Some qualitative question tips:

1. Testing your questions is important. But it doesn’t have to be painful. Write your questions then have your coworkers answer them. They’ll help you find questions that are unclear or that don’t quite get you the answers you’re looking for. In fact, it’s best to use your first couple of respondents as test cases as well and flagging those answers as needed in your results.
2. Keep your questions concise. People hate to read on the web. This is a usability principle as old as the field. People hate reading but love clicking and finding out what happens. So keep your questions short and to the point, but make sure they’re still clear.
3. Likewise, people don’t read long intro paragraphs. They’ll just skim and move on. So if you absolutely must write something that’s multiple sentences, be sure to bold the critical parts.
4. People will bomb out of your survey. Even well-intentioned respondents will have bosses that will come over and ask them a question right before lunchtime. So front-load or randomize your questions if you aren’t certain which are the most critical.
5. Finally, don’t lead your respondents. Avoid judgement words or phrases. Instead of, ‘how difficult did you find this task,’ ask if they found the task easy or difficult. The point here is that you want to avoid implying judgement. Instead, you want to gather the user’s judgement.

15 —

1. Plan ahead. Just shooting off a survey is never a good idea. Draft some questions and test them first. Understand your goals and make sure you have buy-in from your stakeholders first.
2. Keep your surveys as brief as possible, provide completion time up front, and show a progress indicator throughout your survey.
3. You will almost always want a mix of quantitative and qualitative questions. Because you’ll need to know what is happening and why it’s happening in order to solve your issues.
4. Ask if a respondent is willing to discuss their responses later, because no survey response is perfect, and often lead to more questions.

Method #2 — Post It Sessions

16 —A post-it session, also known as an opportunity workshop, is where you gather feedback from your respondents in a way that helps you understand at a high-level what your existing experience issues are and how to prioritize those. This is a great method for learning what your high-impact projects are, can be done relatively quickly (in about 3 - 4 hours), and can be done with either users or internal teams. If done right, these can yield a lot of useful feedback in a very short period of time.

17 —Post-it sessions really shine when you need the sort of high-level feedback that they can provide, and when you need clarity on direction. This means the sort of situation when you know you have issues... maybe even lots of them... which certainly wouldn’t apply to anyone here... and you need to know how to prioritize them effectively.

You can also just do these as a sort of gut-check exercise. It’s easy to get into the weeds on projects, especially when they’re driven from administration, and it can be very valuable to get perspective this way.

18 —First you’ll need to decide who’s going to participate. You can either schedule time with a team and do your session in a conference room, or you can go out where you can find your users and pull them aside in a public space.

If you’re doing the latter, find someone to help recruit participants while you remain at a central station with your post-its. It’s obviously best if this person is an extrovert and doesn’t mind politely asking people to help out.

19 —When conducting the actual session, it’s as easy as having your participants write their feedback on the post-its. I find that post-its provide a useful limitation of the space available.

I also like to use sharpies and differently-colored notes for positive, neutral, or negative feedback.

20 —When you’re done, take your post-its back to a whiteboard and organize them into categories. The colors of the post-its should give you a good overview of the general sentiment for those categories.

21 — Sorted Post-Its

22 —Now you’re going to re-write each feature mentioned in the feedback post-its. Use a new color and write out the feature and mark how many people mentioned it.

Next, create a 2x2 with 2 axes separating 4 quadrants. Make your x-axis Value and the y-axis effort.

Place your new post-its on this chart (being sure to consult coworkers if you aren’t sure about how to estimate effort for a given feature). You should now be able to easily tell which features are going to provide the most bang for the buck.

23 —Opportunity 2x2

24 —Post-it sessions can be an effective tool for gathering a lot of high-level information quickly. They’re open-ended, so your feedback will vary widely in quality, but given enough of it, you’ll definitely find some excellent information that you didn’t know.

Beyond the feedback itself, it provides a handy way to sort and organize the feedback into a very actionable format.
And it can be done with internal teams to help clarify business priorities, as well as your general users to understand their needs.

Method #3 — User Interviews

25 —User interviews are exactly what they sound like: you get to be the reporter and go into the field to interview the people using the thing you’re working on. You’ll be having face-to-face meetings with users, asking them questions, taking notes, making observations, and writing a report.

Of course, there’s a wide range of how in-depth a user interview can be. You could stop Gary in the hall and ask him something like, ‘hey, Gary, I was wondering if you ever use multi-select on the email list editor.’ And yes, you could technically call that an interview. And yes, it could provide some useful information. So my first point here is: try not to be intimidated.

What I’m going to talk about in this section, though, is how to conduct a systematic round of interviews that focus on goals that give you some deeper and more useful insights.

26 —Because you’re taking a deep dive with just a few people, these are best when you’re looking at the details, like when you’re starting out on a redesign, or you are digging into the details of features and behaviors.

I’ve also used this to get my arms around big projects when I changed teams and found myself responsible for the usability of a big system that I was unfamiliar with.

27 — First you’re going to write your script. There are three basic sections to it: the Introduction, Questions, and Conclusion. Are you with me so far?

1. The introduction is very important. This is where you let your interviewee know who you are, what you’re doing, why you’re doing it, and what will be done with their information. If you’re going to record any part of their session — and you’re going to take notes, so at the very least you’re recording that — this is where you let them know. This part isn’t optional. Beyond the fact that it’s ethical to let people know what you’re doing with their information that they’re offering to you, it could impede your ability to conduct further interviews if word gets out that you’re not trustworthy. A good introduction will feel natural, and establish a comfortable relationship with your interviewee.
2. For writing your questions, you’ll want to follow a lot of the same guidelines as writing questions for surveys. However, there are some important differences.
   - One, is that you will be delivering your questions verbally. So make sure that your questions feel natural to read through. Read them out loud to yourself. Read them out loud to someone else. This can make a big difference in how comfortable the entire session is.
   - Another big deal is that you can ask users to perform tasks and actually watch them do it. In this case, it’s usually good idea to include a note for them to speak out loud and walk through their thought process as they’re doing their tasks. It can be common for people to struggle with a task, or for you to watch someone do a step they forgot to mention. This unspoken feedback can be critical.
3. The conclusion is pretty straight-forward. Just let people know that you’re done and thank them for their time. Let them know if there are any next steps or if you might be following up with them.

28 —Next, you’ll need to find some participants. This can be challenging, but probably not as challenging as you think it might be. It’s usually not too difficult to find helpful folks, though scheduling can take time. Just be patient and work through your contact list. For this step I’ve found it best to just use my best judgement. Try to find people you know will be happy to help, and who represent different groups of users.

A final point I’d like to make is that you want to make sure that you’re watching the user perform tasks as closely as possible to how they normally would do them. That’s why this technique is commonly referred to as ‘contextual inquiry.’ Well, that and it sounds fancier to say. But aside from being able to sound fancy, it’s important because once you take your interviewees outside of their normal environment, their experience shifts in unpredictable ways. If you’re working on a product that they’re going to use in their office, go to their office. Watch them use their setup to do tasks. A common thing I’ve noticed when doing this, for example, is varying screen resolutions and monitor sizes. These can radically affect what’s visible at any one time, and that can make a big difference in how someone uses something.

I’d also like to touch on a pet peeve of mine when it comes to user interviews: focus groups. If you can get all your interviews done in one fell swoop, then that’s awesome, right? The problem is that people are social creatures. We respond to all sorts of dynamic social cues, and focus groups take people out of their comfort zone, place them into unfamiliar groups, and then have them answer questions by someone who had the power to call them all together to have them answer questions. Strong personalities will drive responses of those who are quieter in unfamiliar situations, and this has no reflection on how those people would normally use your product. Unless people are using your product in a group, don’t interview them in a group.

29 — Conducting User Interviews

1. There are a number of ways you can take notes. You can simply write notes on paper, or as I prefer to do, print out your script and leave spaces for notes between questions. I’ve also heard of people that use spreadsheets to take notes in, as this can save a great deal of time later on when you have to consolidate your notes since consolidating and organizing handwritten notes can be extremely time-consuming.
2. You may run across questions or terms in your script that are unclear. Here, feel free to clarify, but be aware that this isn’t desirable and reflects a gap between your understanding of the system and the user’s. This is something that can be worked out by testing your questions ahead of time.
3. It’s okay to let your users struggle a bit. It may feel bad to watch someone struggle to accomplish something, but it provides important information. The problem is that it can eventually feel so bad that it threatens the session. In that case, feel free to end the task.
4. You can ask follow-up questions, but try to be clarifying rather than leading. For example, try ‘tell me more about when you arrived at this page,’ instead of, ‘why were you upset when you arrived at this page?’
5. Finally, it’s important to be respectful of your interviewee’s time. Your participants are your most valuable resource and it’s important to treat them that way. Keep an eye on the time and let them know when time is up and offer to let them end the session with the option of continuing another time. I’ve found that if it’s something you don’t do often, and they don’t have something scheduled right away, they’ll go ahead and finish then anyway.

30 — Compiling Your Notes

Once you’ve completed your sessions you’ll need to compile your notes. This can be difficult and time-consuming to do with long-form responses. This is also something that an assistant could help out with.

Spreadsheets can help because you can list respondents with their answers to given questions in rows and columns. You can color code them and summarize the responses as well. If you took your notes directly in a spreadsheet, then that makes this all that much simpler.

31 — Creating a Slide Deck

I’ve seen large studies summarized into very brief bullet points and it’s been super effective. Stakeholders generally just want to know the high-level takeaways instead of getting into the weeds. Slide decks are good for this sort of feedback and you can provide visual examples where relevant.

32 — User Interview Takeaways

At 5–10 hours, this method takes a bit longer than the others, but sitting down with your users, asking questions, and watching them use your product is really one of the best ways to get detailed feedback.

Go to where your users are, and make sure they’re comfortable.
If you keep your participants at 5, you’ll reveal 80% of the issues with your site or application, and hopefully your questions will reveal why those issues are occurring.

Method #4 — Guerrilla Usability Testing

33 —Guerrilla Usability Testing has some elements of the post-it session and the user interview. The idea here is that you’re looking to quickly find participants (like in a post-it session) who will sit down and validate some idea by running through tasks (like in an interview). These tasks can be performed in either an existing product or a prototype mockup. And like in an interview, you’ll watch what they do, where they struggle, what they like, and you’ll take notes.

A quick note about prototypes... they’re exactly that. They don’t need to be finished, and they can take many forms, from paper sketches, wireframes that can be clicked through, or flat HTML/CSS. The point of a prototype is to test something. You want to spend as little time building it as possible so you can get it in front of people to start validating ideas. It’s okay if it’s rough.

34 —Guerrilla Usability Testing is quick and focused, so this method is good at validating concepts and gathering detailed feedback on specific features or ideas.

35 —In order to quickly find your participants, you’re going to have to go to them. So find a location that makes sense based on the users of your site. A library lobby or coffee shop are good examples. Make sure you have a place where you will be easily seen and can recruit people, as well as somewhere out of the way where you can have your computer set up and waiting.

36 —You’re also going to need to write a script. We discussed scripts and tasks previously with user interviews, but what I want to mention here is that the key for guerrilla usability testing is keeping it short and sweet. Because you’re pulling aside semi-random people, you’ll want to be sure that their time investment is minimal. This will also help you get more participants.

Otherwise, you’ll want to be sure you have informed consent in your introduction and explain how their information will be used. And at the end thank your participants for their time when you’re done.

37 —Conducting the Test

Alright, “Conducting the Test.” Bullet 1: Take Good notes... hey I’ve got a question, who’s taking detailed notes here? Anyone? Everyone? Because I can save you a bit of time... these are the same bullet points as User Interviews! I cheated and copied the slide and just changed the title!

This is simply running participants through a task as you would in a user interview with a testing component. So simply ask someone to complete their task and let them know ahead of time that you’re just here to see if the task is easy for people, so you’re going to attempt to avoid giving much help if possible. It can also be helpful to let people know that it’s the site or application that’s being tested, not them.

38 —Gathering and presenting your findings is also similar to user interviews. Search for the patterns and present your feedback in easily-digestible chunks.

There’s one area where you should deviate from the high-level bullet points, and that’s quoting users. Make sure that you record the most interesting things your users say while you’re taking notes, because nothing is more powerful in a report or presentation than pulling in direct quotes.

39 —Guerrilla Usability Testing is a great way to get several tests done in a single day so that you can get some detailed feedback and integrate it into your workflow right away. It really shines when it comes to refining specific features or task flows.

Conclusion

Slide 40

41 — Surveys offer the ability to gather feedback from a broad audience at very little cost. In large part you just set it up and hope for responses. However, you won’t be able to follow-up, and sending out too many surveys or surveys that are too long or complicated can be onerous to users, so use them wisely.

42 — Post-It Sessions provide an open-ended way of gathering general feedback on problem areas in your product. This is a great way to quickly find out what major areas of your products or services are in most need of work.

43 — User Interviews are where you dig into details and figure out why you’re having issues. This is a great way to put yourself in your users’ shoes by going to where they are and seeing how they use the product. This is the most time-consuming method discussed today, but also provides the richest feedback to work with.

44 — Guerrilla Usability Testing is a compromise between Post-It Sessions and User Interviews. This is a great way to quickly get detailed feedback, but you have to limit your scope in order to do so.

45 — Bottom Line

The bottom line is that 100% of people working on your project has opinions about the project, yet 0% are real users who don’t spend their days working on it. That’s why the usability mantra is “you are not your user.”

So get out and talk to your users, however you can.
Be up front, be honest, be helpful, and listen, and you’ll learn what you need to know. And finally, I’d just like to say…

46 — Always say thank you at the end.

Credit goes to a great many people for doing a lot of solid research and learning to back these ideas up. Specifically, off the top of my head:

Jakob Nielsen has been foundational in the field, including providing evidence behind the whole ‘you don’t need to talk to more than 5 users’ thing. He and the people he’s work with deserve mention. https://www.nngroup.com/articles/

Leah Buley wrote The UX Team of One. A lot of ideas here came from that book, and I can’t recommend it enough.