Transcript ASSESSMENT OF ACADEMIC WEBSITES

ASSESSMENT OF ACADEMIC
WEBSITES
Asad Khailany ,
Michael Sheppard
Abhijit Modak ,
Wafa Khorsheed,
Eastern Michigan U
Eastern Michigan U
Eastern Michigan U
Eastern Michigan U
 Currently there are no standard metrics to
measure the effectiveness of academic web
sites. In this paper we are recommending 25
metrics divided into 5 different categories to
measure and make academic web sites more
effective and productive to professors, to
students, and to public. We have used the
proposed metrics to evaluate the websites of
40 professors to provide
 Most of faculty members and students heavily utilize the




Internet for research, news, communication and
entertainment.
Internet provides many opportunities for professors to
reach out and engage students in an interactive and nonthreatening medium of communication.
Variety of web design tools such as WYSWIG (What You
See What You Get) HTML editors, Flash, Dream Weaver,
XML, Perl, Microsoft Word and many others are available
to design dynamic web pages .
There no many guidelines to design web effective
academic websites.
Currently there are no standard metrics for the
measurement of the quality of the academic web sites .
Chickering and Gamson proposed the following 7 principles for Good
Practice in Undergraduate Education:
1. Good practice encourages contact between students and faculty.
2. Good practice develops reciprocity and cooperation among students.
3. Good practice encourages active learning.
4. Good practice gives prompt feedback.
5. Good practice emphasizes time on task.
6.Good practice communicates high expectations.
7. Good practice respects diverse talents and ways of learning.
Chickering and Gamson did not proposed any
technique or method to achieve their seven
principles.
We believe today’s technology provide good
opportunities for achieving the above principles
via good quality effective academic websites. For
that we have taken these principles in
consideration, combined them with good web
design practices, and incorporated them into a
group of metrics that can be applied to measure
the effectiveness of academic web sites.
 We are proposing 25 metrics, grouped into following 5 categories:
1. First Impressions pertain to site features and observations
that a visitor would note within the first few moments they
have loaded the site. Such metrics pertain to whether there
is readily available contact information, a consistent design
throughout, download time, aesthetics or “look and feel”
and whether the purpose of the site is clear. When a visitor
has passed a judgment on these metrics, they will have
formed a definite opinion of the site before even viewing
the sites contents. While the opinion may be a strong
positive or negative one, it is not irreversible. So no extra
weighting is given to this category that would make it
count more than the others. The next proposed category is
Following are the proposed metrics for FIRST
IMPRESSION category:
1. The portal quality of the web site is good.
2. The site has essential contact information like email,
phone/fax numbers.
3. The design is consistent (headers, footers, navigation
bar, etc.)
4. Download time is short
5. It has a good look and feel – Readability, Attractiveness
6. The purpose of the site is clear? Is it obvious who owns
this site?
2. Navigation: The three metrics proposed for this category
have fairly obvious intents. They are centered on two areas
of concern that are universal to all websites, not just
academic ones: Do the internal and external links work? Is
there a map that can sufficiently direct a visitor to the
content they are looking for? The proposed metrics for this
category are:
1.The links all work
2. There are links to the home page on every page
3. There is a comprehensive site map
3. Content: The metrics of this catagory have the
closest correlation with Chickering and Gamson’s
Seven Principles. They cover a wide range of
topics and attempt to capture not only individual
features of the site but also create a sense of the
total dynamic experience for a visitor. After
evaluating the site based on the metrics in
category Content, the evaluator should have a
sense of its relevancy to a student’s course work,
how engaging the site is, the level of interactivity
it promotes, and how up-to-date is the content that
is provided.
Following are the proposed metrics for CONTENT category:
1. It has bulletins & announcements
2. There is class meeting information: date, time, location, agenda
3. Home works and assignments are posted
4. The site contains tutorials for related materials to your course
5.The site contains the solutions for past exams
6. The site has a syllabus. The syllabus contains the hot links
7. There are handouts posted on the web site. The handouts make sense within
the context of the course.
8. There is personal information about the author of the site
9. The page is dated and current
10. The site uses valuable graphics, animation or sound
11. Level of interaction is high (i.e. exercises, bulletin board feature, etc.)
12. There are synchronous collaborative communications tools for real-time
interaction(ie. chat functionality)
13. There are asynchronous collaborative communications tools (i.e threaded
discussion feature)
4. Findability: The fourth proposed category measures
how easily a viewer can locate the web site from a search
engine. The metrics in Findability ask the evaluator to
check for relevant keywords in the header of the HTML
code. It also asks the evaluator to assess whether they feel
the URL path is simple and easy to remember. Not only is
Findability concerned with whether a visitor can easily
find a professors homepage but whether they can find their
way to specific content within the site from outside the
site. This is the intent in the metric “It does NOT use
frames”. Though frames can be a convenient way for the
developer to create a clean uniform site, they also can
present the would-be visitor some daunting challenges as
they try to locate and bookmark a specific page on the
Following are the proposed metrics for
Findability category:
1. It uses intuitive keywords.
2. It does not use frames.
3. The URL is intuitive.
5. Compatibility: This category is the most self-evident of
the five metric categories. The effort behind this category
is aimed at establishing whether the site provides enough
backward compatibility with older browsers. Following are
the proposed metrics for this category:
1. Internet Explorer 4.0 and up
2. Netscape 5.0 and up
The proposed metrics attempt channel those seven
principles specifically to the online presence professors
have built to support their classroom teaching. Some of
the metrics in our evaluation are articulated to directly
support one or more of the seven principles.
We used the proposed metrics to evaluate the
quality of the websites of 40 professors from
different colleges and universities. The web sites
studied achieved an average rating of 2.59 out of
4.00 or 64.70% effective. The median score was
2.65 out of 4.00 or 66.20%. Based on our analysis
of individual characteristics of each web site, we
concluded that a web site might be considered
successful by achieving an average score of 3.00
out of 4.00 in each of the main categories.
The web sites generally scored highly on the following metrics
 The site has essential contact information like email, phone/fax
numbers.
 Download time is short
 The purpose of the site is clear. Is it obvious who owns the site?
The web sites scored weak on the following metrics:
 The portal quality of the web site is good.
 There is a comprehensive site map.

There are asynchronous collaborative communications tools (i.e.
threaded discussion feature)
 There are synchronous collaborative communications tools for realtime interaction (i.e. chat functionality)
 Level of interaction is high (i.e. exercises, bulletin board feature, etc.)
 By improving their web sites around these issues,
professors can better engage and maintain the student’s
attention.
 Unless the portal quality is improved and a comprehensive
sitemap is provided, the students may spend too much time
trying to find relevant information, which can cause the
student to discount the value of the site as a learning tool.
 The use of synchronous communication tools, such as live
chat, and asynchronous collaborative communications
tools, such as the threaded discussion feature, are
important in order to develop reciprocity and cooperation
among students and encourage contact between students
and faculty.
 The use of synchronous communication tools combined with a high
level of interaction through exercises, bulletin boards, etc., would
achieve the Chickering and Gamson principles of time on task, active
learning, and prompt feedback.
 The matrix is a flexible tool that allows a professor to use it in a manner
that meets their needs. For example, a professor may elect to use it as
a personal tool where they can rate the effectiveness of their own site.
Others may elect to use it as part of an instructor-lead evaluation,
where they ask their students for feedback on their web site
 It is the goal of every educator to build an effective, and useful website..
With our proposed metrics educators can effectively, analyze, and
improve the quality of their web sites. We used these metrics to
evaluate the websites of 40 professors. Websites cannot be regarded
“one-shot deals” that can be developed and not updated. It must be
continuously evaluated and updates. Our metric ease such evaluations
and updates. Complete research results provided in the appendix