Transcript No Slide Title - tedleath.com

Report on Evaluation Process
The decisions and issues involved in the
proposal, research, design and prototyping of
the Mood Gallery are documented on-line
beginning at;
http://www2.infm.ulst.ac.uk/~cdp8let/group/pages/gallery.htm
Original Evaluation Proposals
For evaluation and development, the Mood
Gallery team had proposed;
• Iterative questionnaires
• Iterative interview
• A final combined questionnaire and
interview
Rethink on Evaluation
Much of the project was a process of reducing
horizons to match constraints. We went
right back to the beginning - the original
brief - to devise a simpler way to gauge
whether or not we had come close to
matching it
The Brief
We concentrated on what we determined was
the core statement of the brief;
"The software product that you devise should
exemplify, amplify and facilitate harmony with
ideas.”
To further understand the brief, we applied
definitions to the more abstract words.
Definitions
• exemplify - to show or illustrate by
example.
• amplify - to expand by the use of detail or
illustration or by closer analysis.
• facilitate - to make easier.
• harmony - pleasing or congruent
arrangement of parts.
New Evaluation Questions
Taking a literal view of the brief, new and
simpler evaluation questions were devised
based directly on the definitions associated
with the brief.
Four questions were devised. Each question
allowed a rating from 1 to 5.
Evaluation Question 1
exemplify - to show or illustrate by example.
Does the software make effective use of
images to illustrate concepts? (1=not
effective, 5=very effective)
Evaluation Question 2
amplify - to expand by the use of detail or
illustration or by closer analysis.
Does the software make use of varying levels
or stages? (1=no use of levels and/or stages,
5=good use of levels and/or stages)
Evaluation Question 3
facilitate - to make easier.
Does the software make the process of
browsing an art gallery easier? (1=not as
easy, 5=much easier)
Evaluation Question 4
harmony - pleasing or congruent arrangement
of parts.
• Do the various elements of the software
work together to form a cohesive whole
which complements the gallery experience?
(1=not cohesive, 5=very cohesive)
What the Evaluators Thought
Group Averages
Question 1
Question 2
Question 3
Question 4
average 3
average 4
average 3.5
average 4
60%
80%
70%
80%
Does the Software Fit the Brief?
Overall
72.5%
Based on the 4 simple questions provided, on
a sliding scale of 20 to 100 (100 being the
fullest satisfaction of the brief) the Mood
Gallery software received a rating of 72.5
from the evaluators.
Evaluator Comments
The navigation was initially confusing - this was
partly intentional in that the viewer could, if
confused by the navigation, enter either the good
or bad mood gallery. This point was reinforced by
giving the 'floorplans' an interactive feel i.e. text
appeared when the cursor was placed over the
icon, this text clearly indicates what the icon is
and where you will go upon selection.
Evaluator Comments (cont.)
The graphics were clear and crisp, suiting the project
content - this was the intention, to try and recreate
the simplicity of viewing a 'real' exhibition, you
simply enter and browse.
The eye icon was an informative feature - this idea
was based upon the metaphor of carrying a guide,
which would give information on the art and
artists, it can be availed of whenever the viewer
wishes to obtain more information on a particular
mood canvas.
Evaluator Comments (cont.)
Can you play your own selection of music whilst
viewing the various exhibitions? - this is
accommodated in the project by providing a
default music selection upon entering a selected
gallery, if the user is displeased with this music,
then they have the option of playing their own
music to sympathise with their own mood.
Evaluator Comments (cont.)
The canvas size could have been bigger - Our point
regarding the size of the canvas is documented in
the storyboard, (frame ?) it shows how the user
can select a certain piece of mood art and 'walk
toward it to get a closer look' this is done by
bringing the canvas towards the user until it fills ¾
of the screen. This is not shown in the prototype.
Evaluator Comments (cont.)
Downloading time was too long - this could be due
to heavy traffic, the director movie is saved as a
shockwave so as to load faster.
Submit mood - too vague, needed description.
Allocating music with the mood art upon
submission.
If in a bad mood, should you not want to be cheered
up!
Most people like listening to sad songs when sad actual quote.