Transcript 22_Questionnairesx - SIUE Computer Science

The Post-Study System Usability Questionnaire (Version 3)
Strongly
Agree
1
2
Strongly
Disagree
3
4
5
6
7
NA
1. Overall, I am satisfied with how easy it is to use this system.
     


2. It was simple to use this system.
     


3. I was able to complete the tasks and scenarios quickly using
this system.
     


4. I felt comfortable using this system.
     


5. It was easy to learn to use this system.
     


6. I believe I could become productive quickly using this system.
     


7. The system gave error messages that clearly told me how to fix
problems.
     


8. Whenever I made a mistake using the system, I could recover
easily and quickly.
     


9. The information (such as on-line help, on-screen messages, and
other documentation) provided with this system was clear.
     


10. It was easy to find the information I needed.
     


11. The information was effective in helping me complete the tasks
and scenarios.
     


12. The organization of information on the system screens was
clear.
     


13. The interface of this system was pleasant.
     


14. I liked using the interface of this system.
     


15. This system has all the functions and capabilities I expect it to
have.
     


16. Overall, I am satisfied with this system.
     


PSSUQ-3 Norms (Means and 99% Confidence Intervals)
Lower
Upper
Mean
Limit
Limit
1. Overall, I am satisfied with how easy it is to use this system.
2.60
2.85
3.09
2. It was simple to use this system.
2.45
2.69
2.93
3. I was able to complete the tasks and scenarios quickly using this
system.
2.86
3.16
3.45
4. I felt comfortable using this system.
2.40
2.66
2.91
5. It was easy to learn to use this system.
2.07
2.27
2.48
6. I believe I could become productive quickly using this system.
2.54
2.86
3.17
7. The system gave error messages that clearly told me how to fix
problems.
3.36
3.70
4.05
8. Whenever I made a mistake using the system, I could recover easily
and quickly.
2.93
3.21
3.49
9. The information (such as on-line help, on-screen messages, and other
documentation) provided with this system was clear.
2.65
2.96
3.27
10. It was easy to find the information I needed.
2.79
3.09
3.38
11. The information was effective in helping me complete the tasks and
scenarios.
2.46
2.74
3.01
12. The organization of information on the system screens was clear.
2.41
2.66
2.92
13. The interface of this system was pleasant.
2.06
2.28
2.49
14. I liked using the interface of this system.
2.18
2.42
2.66
15. This system has all the functions and capabilities I expect it to have.
2.51
2.79
3.07
16. Overall, I am satisfied with this system.
2.55
2.82
3.09
The After-Scenario Questionnaire (Version 1)
Strongly
Agree
1
2
Strongly
Disagree
3
4
5
6
7
NA
1. Overall, I am satisfied with the ease of completing the tasks in this
scenario.
      

2. Overall, I am satisfied with the amount of time it took to complete
the tasks in this scenario.
      

3. Overall, I am satisfied with the support information (online help,
messages, documentation) when completing the tasks.
      

The System Usability Scale (SUS)
1. I think that I would like to use this system
frequently.
2. I found the system unnecessarily complex.
3. I thought the system was easy to use.
4. I think that I would need the support of a
technical person to be able to use this
system.
5. I found the various functions in this system
were well integrated.
6. I thought there was too much
inconsistency in this system.
7. I would imagine that most people would
learn to use this system very quickly.
8. I found the system very cumbersome to
use.
9. I felt very confident using the system.
10.I needed to learn a lot of things before I
could get going with this system.
The Extremely Positive System Usability
Scale (XP-SUS)
1. I think that this is one of my all-time favorite
web sites.
2. I found the web site was really
straightforward.
3. I thought the web site was amazingly easy to
use.
4. I think that technical support services are just
not required for the web site.
5. I found the various pages on the web site
worked together very smoothly.
6. I thought the web site was consistent
throughout.
7. I would imagine anybody could use the web
site like a pro from day one.
8. I found the web site was a delight to use.
9. I felt completely confident using the web site.
10.Everything I needed to know about using the
website was there for me.
The Extremely Negative System Usability Scale
(XN-SUS)
1. I think I never want to use the web site again.
2. I found the web site to be horribly complex for
no good reason.
3. I thought the web site was very difficult to use.
4. I think that I would need a permanent hot-line to
the help desk to be able to use the web site.*
5. I found all the pages on the web site to be an
ugly mess.
6. I thought the inconsistency in the web site would
kill it.
7. I found the web site to be completely impossible
to use.
8. I found that this web site was extremely
awkward to use.
9. I felt utterly confused by the web site.
10.Absolutely nothing about the web site worked
The average score on the XP-SUS were approximately 30% lower than
the results on the SUS, while the average score on the XN-SUS were
approximately 25% higher than the results on the SUS.
People tend to only agree with statements that are close to their
attitude and disagree with all other statements.
By rephrasing items to their extreme concept, only respondents
who had passionately favorable attitudes about the usability of a
particular website would tend to agree with the extremely
phrased positive statements, resulting in a significantly lower
average score.
Similarly, only respondents who had
passionately unfavorable attitudes about
the usability of a particular website would
tend to agree with the extremely phrased
negative statements, resulting in a
significantly higher average score.
Research indicates that respondents to questionnaires tend
to favor the left side of a response scale.
My College has an excellent reputation.
My College has an excellent reputation.
Strongly
Disagree
Disagree
Undecided
Agree
Strongly Agree





Strongly Agree
Agree
Undecided
Disagree
Strongly
Disagree





More students tended to give “Strongly Agree” or “Agree” responses to the second
response option than to the first response option. Explanations vary regarding the
causes of this effect.
Possible Remedies:
• Have the survey software offer the possibility when creating questions for
the response options to be “flipped” so that some people will see them
negative to positive and others will see them positive to negative.
• Vary the response order in long surveys, particularly when using the same or
similar scales, to avoid respondents suffering from “survey fatigue”.
• Replace the radio-button format of the question’s interface with a slider that
is set by default to the middle “Undecided” setting, thus resulting in the
same effort to move the slider in either the Agree or Disagree direction.