The Woes of User Testing and Guys vs. Gals
I volunteered to faciliate user testing for this new web system a colleage of mine built. Frankly, it's pretty slick, conveys tons of information and seems really intuitive to me. I took me about 5 mintues to navigate the system and finished the items on the checklist even though I had never seen it before. Thing is, I'm a programmer. Little nuggets of facts such as knowing the test database of search keys will probably be incomplete, so dont' bother with the search function probably don't go passing through the heads of the general public. Furthermore, I went to school in the dark ages where a lot of these processes had to be done on paper, so I seem to appreciate web convenience more than others.
Anyhow, I been helping out with this user testing stuff for about a week now and I must say, this brims on comical trying to sit through and seriously take notes on each victim (university student). The items are very vague because you want to track the click sequences, vocal commentary, level of frustration (that's a great one) and note overall body language. Furthermore, there's a nice demographic survey they fill out beforehand for statistical purposes. I can generalized based on this that individuals from Student Government are by far the most annoying.
Kid: Well that's a dumb question...
Me: Hmmm?
Kid: Race. Why do I need to give my race?
Me: You technically don't need to fill it out but it is for statistical purposes
Kid: Why?
Me: Because we don't want to bias the system in anyway.
Kid: That's absolute crap.
Me: Just fill out the form, white boy.
On average people get stuck on the very first task because they're expected to navigate first through this interface that built by a pompus arrogant chick who thinks the world thinks just like her. (As in, there was NO user testing before she migrated that piece, let me tell ya. It was like one day, I woke up and poof, there it was.) Therefore, the link that users need to click on isn't easily seen unless you scroll down half a mile. Even so, the font is something like 10 point...Too tiny. I already mentioned how this is the test database, so search keys aren't in place, therefore searching for the link won't work. I'm not allowed to coach in anyway except for giving encouragement. I take notes and try not to be helpful.
Kid: Goddamnit!
Me: Would you say your level of frustration is medium or high?
Oh. The evil glares.
I can say the original design of the questionnare was much more helpful, but once the research analysis people came through it totally changed. It wasn't vague enough. You can imagine how us programmers felt about that, but oh well, user testing ain't our expertise, so vague questions it is.
Anyhow, 45 mintues into this the Kid eventually is able to complete most tasks. Getting in was the hard part.
Kid: This layout is fucked up.
Me: It's hard to quantify "fucked up." Could you explain how you would improve it?
Kid: I dunno. Just make it better. Add more buttons or something.
Is should point out the Kid is about 24 according to his demographic survey. In the exit survey he refused to mention anything good about the site. Which is fine, but this illustrated the difference between the male versus female students who performed the test.
Girl: Am I doing this right?
Me: Are you accomplishing the task?
Girl: (long pause....reads the questionnare again) I guess
Me: How would the website have made it clearer that you accomplished your task?
Girl: I don't know.
(after another long pause)
Girl: Could you maybe add a bit of java script to enforce a pop up that would verify that the task has been completed? But you don't have to...it was probably just me.
First useful comment of the week followed by self deprication.
It was obvious her level of frustration was a bit high as well, but upon the exit interview, she added happily: "This is a great service!"
Was that sincere or not? I'll never know, but I did put it down in my notes for the user group to procress.
By far the funniest demographic is the know-it-all Asian kid. I can say this because I am one, but they tend to be the gunners, flying through the website in a matter of minutes. By the time I look up from taking notes from the first task they're completely though with the survey.
Nerd: DONE! (pencil slamming)
Me: Okay...this is not an intelligence test. You're not getting graded on this.
I've got two more sessions to do and then I'm off the hook. I have to say I am so glad this is not my website because I don't know how I'd handle this information.
Anyhow, I been helping out with this user testing stuff for about a week now and I must say, this brims on comical trying to sit through and seriously take notes on each victim (university student). The items are very vague because you want to track the click sequences, vocal commentary, level of frustration (that's a great one) and note overall body language. Furthermore, there's a nice demographic survey they fill out beforehand for statistical purposes. I can generalized based on this that individuals from Student Government are by far the most annoying.
Kid: Well that's a dumb question...
Me: Hmmm?
Kid: Race. Why do I need to give my race?
Me: You technically don't need to fill it out but it is for statistical purposes
Kid: Why?
Me: Because we don't want to bias the system in anyway.
Kid: That's absolute crap.
Me: Just fill out the form, white boy.
On average people get stuck on the very first task because they're expected to navigate first through this interface that built by a pompus arrogant chick who thinks the world thinks just like her. (As in, there was NO user testing before she migrated that piece, let me tell ya. It was like one day, I woke up and poof, there it was.) Therefore, the link that users need to click on isn't easily seen unless you scroll down half a mile. Even so, the font is something like 10 point...Too tiny. I already mentioned how this is the test database, so search keys aren't in place, therefore searching for the link won't work. I'm not allowed to coach in anyway except for giving encouragement. I take notes and try not to be helpful.
Kid: Goddamnit!
Me: Would you say your level of frustration is medium or high?
Oh. The evil glares.
I can say the original design of the questionnare was much more helpful, but once the research analysis people came through it totally changed. It wasn't vague enough. You can imagine how us programmers felt about that, but oh well, user testing ain't our expertise, so vague questions it is.
Anyhow, 45 mintues into this the Kid eventually is able to complete most tasks. Getting in was the hard part.
Kid: This layout is fucked up.
Me: It's hard to quantify "fucked up." Could you explain how you would improve it?
Kid: I dunno. Just make it better. Add more buttons or something.
Is should point out the Kid is about 24 according to his demographic survey. In the exit survey he refused to mention anything good about the site. Which is fine, but this illustrated the difference between the male versus female students who performed the test.
Girl: Am I doing this right?
Me: Are you accomplishing the task?
Girl: (long pause....reads the questionnare again) I guess
Me: How would the website have made it clearer that you accomplished your task?
Girl: I don't know.
(after another long pause)
Girl: Could you maybe add a bit of java script to enforce a pop up that would verify that the task has been completed? But you don't have to...it was probably just me.
First useful comment of the week followed by self deprication.
It was obvious her level of frustration was a bit high as well, but upon the exit interview, she added happily: "This is a great service!"
Was that sincere or not? I'll never know, but I did put it down in my notes for the user group to procress.
By far the funniest demographic is the know-it-all Asian kid. I can say this because I am one, but they tend to be the gunners, flying through the website in a matter of minutes. By the time I look up from taking notes from the first task they're completely though with the survey.
Nerd: DONE! (pencil slamming)
Me: Okay...this is not an intelligence test. You're not getting graded on this.
I've got two more sessions to do and then I'm off the hook. I have to say I am so glad this is not my website because I don't know how I'd handle this information.
posted by cchang at 8:07 AM
1 Comments:
that's hilarious. :)
Post a Comment
<< Home