Testing The Usability Of The Stanford University Library's Cataloging Services Website

May 7, 2002

Introduction

This report evaluates the usability of the Stanford University Libraries' Cataloging Services (CS) Department website, as it existed between June 2001 and March 2002, and suggest improvements.

The Stanford University Cataloging Services Department website is designed to be publicly viewable, but primarily serves as a resource for cataloging staff and selected staff in other parts of the library. Resources include:

Documentation for local policies and procedures
Staff contact lists
Information about current and past department projects
General information about the department and each service unit
Forms
Tools, including an accession number generator for (primarily) non-print materials (moderate use by staff outside department)
Links to offsite documentation and tools
Unicorn (OPAC) library/location/holding code tables (heavy use by staff outside department)

The site was completely redesigned in June 2001 and upgraded to its current form in October 2001. A Web Advisory Group (CS-WAG) convened in August 2001 to further develop the site, write guidelines, and to address known problems that had surfaced since the redesign, including

Difficulty with navigation and some other features in Netscape 4.7x (the default staff browser). Note: in the context of this report, Netscape refers to these versions only.
Other cross-platform inconsistencies
Lack of a site-specific search, although one could search the site using the library-wide search feature (implemented March 2002)
Lack of comprehensive site index (implemented March 2002)
Some important documents not yet converted
Incomplete or non-authoritative resources chosen for off-site resource listings

Since the redesign, all but a few legacy documents have been updated and converted to the new design, and additional documents have been added at the rate of 1-3 a week.

Description of the Problem

No formal study of staff use had been made prior to the current report and the CS-WAG and management have had no concrete information about its use. The basic site design was developed in a short time with testing from a few managers only; very few staff commented on the site when asked to do so after the launch, with the responses limited almost solely to criticism of the Netscape Javascript handling problems. While the members of the CS-WAG have devoted many hours to the site in addition to their regular duties, the usefulness of that work had not yet been fully demonstrated.

Research Questions

Do CS staff use the site in their daily work?
What barriers (such as violations of common design standards and accessibility guidelines, lack of relevant information, preference for printed reference) prevent its use by CS or other staff?
What are the most important changes that should be made to increase usability?
Have other universities' cataloging or technical services departments created more usable sites and if so, what features or designs create better usability?

Scope

This study looked at site usability primarily in terms of design, with some attention to content.

Methods

The site was evaluated through

Analysis and comparison against design and usability guidelines suggested by Nielsen (1997, March 19; 2000) and Krug (2000), which focused on home page design, interior page design, and navigation
Usability analysis by CAST's Bobby International software against the W3C's Checklist of Checkpoints for Web Content Accessibility Guidelines 1.0 and the Americans With Disabilities (ADA) Electronic and Information Technology Accessibility Standards(Section 508 guidelines)
Comparison of site against sites for equivalent departments of seven major university libraries, also using many of the Nielsen and Krug's guidelines, as well as the W3C's Checklist
Usability studies involving a small sample (5 subjects), which looked at the home page, an interior page and navigation

A Note About the Usability Studies

Five subjects looked at the site in a format suggested by Krug (2000). The number chosen was based on Nielsen's (2000, March 19) report revealing that five users will uncover 85 percent of a site problems and additional users lead to rapidly diminishing returns. All five were frequent Internet users; all but one were department members. The interviews and observations took approximately twenty minutes each (ten minutes for questions and ten for observations). They were less formal than Krug's format (no videotaping, no separate interviewer and remote observer, only one round of testing), but more importantly, were performed many months after the design was launched instead of during development. Nonetheless, results were consistent with expectations from site analysis based on Krug's and Nielsen's guidelines and revealed a number of weaknesses.

Interview script is located in Appendix B; interview/observation results, in Appendix C.

Analysis of Staff Use

Access Logs

Access logs are typically a useful way of gauging site use and logs from the library server are available, with monthly statistics by file, by root directory, and by IP address. I initially expected to use these logs determine which files were most frequently used, and by whom. When I examined the file access logs for June 2001-March 2002, access has clearly increased for many files, but in many cases, the increases were also clearly correlated to the activities of the CS-WAG - each upload of a revised page was recorded as a page access, thus falsely boosting the statistics. New files within a subdirectory all have artificially high numbers the month they were created, but show lower numbers the following months. In February 2002, a form for moving files onto the public server was added to the site and was used an average of 55 times for February and March; but since multiple pages can be moved, these numbers can serve only as a rough guide to anomalously high page accesses.

An analysis of the top 25 pages accessed, however, reveals a very consistent pattern. For the months examined, the following pages appeared on the majority of logs:

Table 1. Page Access

*Page Name (Mar. 2002)*	*Jul-01*	*Aug-01*	*Sep-01*	*Oct-01*	*Dec-01*	*Feb-02*	*Mar-02*	AVERAGE
Admin/Liaison home page	128	114	115	150	116	106	109	140
Cat. Policy: directory	76	73	103	129	99	147	109	123
Cat. Policy: Internet Cataloging	93	66	54	69	55	48	58	74
Classification Unit home page	122	112	114	142	113	109	94	134
Data Control Unit home page	123	109	124	153	122	101	99	138
Home page	476	448	390	582	502	686	726	635
Home page (non Javascript)	110	89	105	140	52	49	55	100
MARC Unit home page	133	112	118	158	122	120	98	144
Materials Control Unit home page	126	110	115	138	119	104	96	135
Metadata Unit home page	131	115	115	143	129	125	110	145
Reference Shelf: General cataloging	94	67	102	109	132	126	126	126
Reference Shelf: Web catalogs	64	67	73	94	63	69	61	81
Site Map	91	73	62	68	68	79	58	83
Unicorn docs index	74	64	80	88	86	84	76	92
Unicorn lib/loc table (default)	283	275	241	274	155	151	129	251
Unicorn lib/loc: Earth Sciences	70	80	82	116	94	118	79	107
Unicorn lib/loc: Green Library	105	116	128	125	140	144	137	149
Whom to Contact: Books	75	68	66	114	61	51	50	80

Note: NACO Hebraica Funnel (a subsite for a portion of the national name authority database project) access accounted for a significant percentage of pages within the CS directory; these data were not included in the list because they are maintained separately from the rest of the files and were not redesigned in conjunction with the rest of the site.

Other discrepancies

Like the Hebraica Funnel pages, the Unicorn library/location/holding tables were maintained separately until Oct. 2001, but were then integrated into the rest of the site. Old files remained on the server until finally removed in Feb. 2002. In some cases, forwarding addresses to new pages were provided; no forwarding was provided for most. The former (and unmaintained) home pages for the Data Control Unit, MARC Unit and older versions of the Unicorn library/location/holding tables continued to be frequently accessed until they were removed.

Total Directory Access

The most interesting data was the total access for the site for the same six months in 2000/2001. Even with an artificially high number of accesses due to the web development, the increase in page visits was dramatic:

Table 2. Directory Access

Month	July	Aug.	Sept.	Oct.	Dec.	Feb.	March	AVERAGE
Total file access (any) July-Oct, and Dec. 2000; Feb-Mar. 2001	5515	6161	6924	7425	7400	6329	8926	8113
Total file access (any) July-Oct. and Dec. 2001; Feb-Mar. 2002	24686	22692	23810	32580	39487	46799	42276	37721
% Increase:	448%	368%	439%	439%	554%	739%	474%	465%

Note: Statistics for Nov. 2001 and Jan. 2002 unavailable due to logging error.

See Appendix A for complete directory access statistics, Jan. 2000-March 2002.

IP Analysis

User's IP address analysis unfortunately provides fairly meaningless results because these statistics are available only for the library server (www-sul.stanford.edu) as a whole and no breakdown is available for the department directory.

Staff Survey

An informal survey of roughly two-thirds of the department staff revealed that most staff used the site one to two times each week. Newer staff had used it when they began to familiarize themselves with local policies and procedures, but most others referred to documents only when encountering unusual problems in their work. Some had made printouts of the most-used documents. If this survey is accurate (no reason to suspect otherwise), the tremendous increase in use is not coming from within the department. The jump is most likely the result of robots indexing the ever-increasing number of pages. This hypothesis could perhaps be verified through further analysis of IP address logs, with a focus on non-SU addresses; other directories show similar access increases.

Analysis of Site Usability Against Common Web Design Standards

Page Design

Home Page

Home page design is typically different than internal page design. The page aims to put a face on the site and allow users to find anything they might need from the site. Krug (2000, p. 97-98) lists the following a variety of tasks the home page might accommodate; a home page for a site such as the CS site might include:

Site identity and mission
Site hierarchy
Search
Timely content
Short-cuts

Figure 1. Home page - default version

In addition to the concrete needs, he lists a few abstract objectives:

Show me what I'm looking for
…and what I'm not looking for

Show me where to start

Establish credibility and trust

How does the CS home page measure against these needs?

A. Concrete needs

1. Site identity and mission

Includes a statement of purpose (an earlier version included a formal mission statement which has now been removed from site entirely). Second paragraph is "happy talk" - self-congratulatory text with minimal content (Krug, p. 47). One user faulted both paragraphs for jargon.

Figure 2. Expanded menu

2. Site hierarchy

Not evident except through menu; no textual link navigation. Users preferred a version with more obvious site hierarchy (see below).

3. Search

Prominent "Search" button brings up a popup menu and also links to a directory of searches. No search box. Search button is at bottom of menu bar; usual pattern is for search button at top.

4. Timely content

No way to tell - no "last edited" note (although a last updated on server note appears at bottom).

5. Short-cuts

Set of links in footer serves; also many items directly accessible through menu

B. Abstract needs

1. Show me what I'm looking for…

Not immediately evident - menu requires mousing to display full options; menu choices not completely obvious

2. …and what I'm not looking for

Most users of site will be seeking specific information. Hidden or surprising information is not necessarily accessible from this page, but may not exist within site for most users

3. Show me where to start

Menu provides reasonably good clue. First time users might also click on expanded description of department's purpose and goals since it is the only link in text - not a bad place to start.

4. Establish credibility and trust

Site is clearly identified as connected to main library site through label at top and through use of library rosette in upper left corner. Colors also incorporate typical Stanford colors.

Other Home Page Considerations

Nielsen (2000) argues that home page design should be different from interior pages, although style should be shared throughout. He suggests that, except for first time visitors, the home page should be a point of entry to the site's navigation scheme; contain a prominent search feature; and include news. The persistent navigation scheme found on interior pages may not be necessary for the home page and a Home button should be avoided.

Figure 3. Home page screen real estate divisions

Krug (2000) is less strict regarding navigation, and suggests that it can be unique, but can also be the same as for interior pages. Typical differences might be section descriptions, different orientation and more space for identity.

Since the CS home page includes no hierarchical listing and the menu is identical for all pages, the user has no choice but to use the menu to navigate. No quick search feature is available, although the user can reach the site search from the footer or by clicking through two pages from the menu.

The design does present information without sacrificing too much screen space to navigation or other non-content. Nielsen's (2000 p. 22) rule of thumb ideally reserves 80% for content and no more than 20% for navigation. The present design is closer to 25% navigation for a 700 x 1064 pixel window.

The Alternate Home Page

Fig.4. Alternative home page [lower half]

Should users happen to use the tab key to click on the button hidden under the menu (or if scripts are disabled), they are taken to a version of the home page with section navigation - a text version of the menu. This version was the original design and was replaced in autumn 2001 with the current version, but retained as an alternate.

All usability testers wished to click on the "What we do" link since it was the only hyperlink within the text. They wanted a minimum of text and uniformly preferred the version with section navigation. One tester felt that the entire text was too self-congratulatory and should be replaced. Another tester thought that the department header violated the hierarchy of most to least important information by not presenting the department name at the top.

Recommendations

Header: Reconfigure to give more prominence to department name (applies throughout site). Possibly move address and fax number under introductory text.
Text: Rewrite and condense; eliminate fluff
Section navigation: Add explicit section navigation; possibly add short explanatory text where necessary. Establish a single home page.
Navigation: Possibly provide slightly different menu without self-referential link

Interior Page

Krug devotes less space to specific guidelines for interior page design, but does offer some thoughts (p. 31):

Create a clear visual hierarchy on each page
Take advantage of conventions
Break pages up into clearly defined areas
Make if obvious what's clickable
Minimize noise

Figure 5. Interior directory page

1. Clear visual hierarchy

Main heading is larger than subheadings, as well as being set off by white space. Header violates this guideline (noted above).

2. Conventions

Krug (2000, p. 35) notes that conventions become conventions because they work. This interior page has a list of links in a colored box down the left side of the window; a header that tells the user whose site they are looking at; and most of the right side contains the content (in this case, a directory).

3. Divide pages into clearly recognizable areas

The page scores well on this account as well.

4. Make obvious what's clickable

Although the navigation links are not underlined, their presence in a clearly defined navigation area makes their function clear. Links in the content area follow convention for color and underlining. An earlier version used only colors in the body, and while the colors were standard, not all text throughout the site uses black as a default. While underlined links makes this directory page seem busy (item 5), no user is likely to mistake them for ordinary text. No underlining for emphasis is found anywhere in the site.

5. Minimize noise

Just by their nature as part of a non-commercial site, this page and others avoid common busyness problems such as ads and many invitations to buy. Colors are muted; headers are used in a logical way; very few exclamation points are used.

Additional Page Design Issues

In addition to the navigation/real estate division mentioned above, Nielsen (2000, p. 25-97), discusses other page design considerations, including cross-platform design - an area in which the CS site falls short. The main problem is slow loading in Netscape due to Netscape's poor Javascript implementation. Even previously viewed pages do not load quickly from the memory cache on a fast machine. Since over 90% of the users in the department use Netscape, they must wait a long time to view each page, compared to IE users. In addition, the Javascript hierarchical menus do not always display properly in Netscape (see Navigation, below) and page elements occasionally overlay upon loading. The Javascript menus also print poorly, whether from Netscape or IE, so that the printed page includes a strange and illegible navigation bar. Pages printed from Netscape often suffer from overlaid elements.

The site uses style sheets, which help maintain a unified appearance throughout, but type size is defined in fixed points, severely limiting the options for users to adjust them for their own preferences (see Disabilities Access, below). The original style sheets set type size in pixels and percentages, but was changed after a user complained about the huge type size that displayed on her preferred Netscape 600 x 800 window. Dictating a fixed type size appeared to be the only way to address the problem at the time - the fixed size displayed approximately the same with both browsers. Short of an automatic window size detector, no easy solution can be applied that will satisfy all users.

One user pointed to the displaying documents across the entire screen as a usability problem. Although the menu bar is a fixed width (100 pixels), the remainder of the window is variable, depending on screen resolution. The National Cancer Institute's Usability.gov guidelines call for no more than 50 characters per line for scanning and 100 per line for careful reading. On a 1024 x 768 screen over 140 characters can display on a line.

Link titles (Nielsen, p. 60) are generally not used. They are an easy way to provide more information about cryptic links; for most users of this site, named links such as "Treatment of 090 on Copy" are useful enough, although to a non-cataloging visitor, such link titles (and the content of the linked document) are likely to be quite opaque.

Recommendations

Platform issues: Investigate additional ways to minimize browser differences, particularly in menu display (see Navigation, below)
Printing: Provide more PDF copies of important documents for users to print; investigate ways to avoid overlaid elements
Text size: Replace all fixed typeface sizes with pixel or percentage sizes (will require additional testing with different screen resolutions, particularly in Netscape)
Line length: Consider limiting line length to 100 characters/line maximum
Link titles: Consider adding, especially for cryptic titles

Navigation

Search

Figure 6. Search menu

Nielsen (1997, July 15) divides site visitors into "search-dominant" users who go to a search box first; and "link-dominant" users, who browse first and search only when they can't find what they are looking for. Nielsen's usability studies indicate that over half fall into the former category, with only a fifth using link-dominant behavior, and the rest using both methods. He recommends:

A search button on every page
Global searching rather than scoped searching (page or section)
Boolean queries only on a secondary search page

Until the end of February 2002, the only search available was for the entire library site. While this search may have sufficed for outside visitors (and even most library users, given the specialized nature of site documents), a user could not easily search within the site itself. The button itself gives several options to search in addition to the CS site. The Cataloging Services Site Index (added at the same time as the site search) links to a comprehensive and cross-referenced document title index; Catalog links to the library OPAC search page; SUL/AIR Web Pages links to a search page for the library server; and finally, Stanford Web Pages links to the university wide search page. The number of choices may actually reduce the search usability since user must decide what scope they want.

Table 3. Sample phrase search: "special collections"

CS	49 pages
CS Index	2 titles
SUL/AIR (Verity)	473 pages
SUL/AIR (Google)	477 pages
SU	20562 pages

A more useful design would be scope options on the search page itself. Another option would be a search box within the navigation bar (site only) and a link to an advanced search page. Since the primary audience is familiar with Boolean searches, users should not have problems with such options in any configuration.

The footer for every page contains links to both the site index and the site search page. The search link is labeled as "search," which may be confusing to users, since it is the same as the button, but narrower in scope.

The current search script creates a results page that is the generic library search results page rather than one that fits in the design of the entire site. This problem was supposed to be fixed but has not been at the time this report.

Recommendations

Search button: Should link to a single page (below) with no hierarchical menu
Search page: Provide options to cover all possible scopes, retaining CS site search as the default.
Results page: Integrate design with rest of site
Search box: Possibly provide a text box in menu bar in place of button (with link to advanced search)
Site index: Integrate into main navigation menu; add link to site index in site search page

Menus

As discussed above, the Javascript menus are quirky, particularly when used with Netscape. Most categories are two or three levels deep; the hierarchical menus give users a sense of how the site is organized. My usability study (and own experience) points to a major problem: the categories aren't self-evident. More than one of the usability testers were confused by both the labels and placement of certain types of documents (one user also questioned the categorizations on an internal directory page). Problematic categories and labels include

Overview: No explicit labels identifying the "Contact List" and "Organizational Chart" as belonging to the department as a whole (confusion may be in the Overview label)
Resources: what constituted a "Resource" appeared to be confusing. The "Local Documentation" section found under "Resources" was also unclear
Home: choices exist for the department, the library and the university home pages

Breadcrumbs and Signboards

One of the most important features of a well designed website is an easy way for users to know where they are. Krug (2000, p. 72-73) recommends a variety of strategies, including:

Page names for every page
Names in the right place - identifying the content rather than some other part of the page
Prominence
Names need to match links from other pages or menus

Both Krug and Nielsen suggest the use of breadcrumbs (e.g. "Local documentation > Cataloging Policies & Procedures > Special Cataloging Notebook > Imprint" at the top of the page) - a term derived from the Hansel and Gretel story. Other possibilities include a fisheye view (a simplified hierarchical chart) with current location and path highlighted or the use of highlighted tabs.

Figure 7. Section and document
navigation

Users have various ways of identifying where they are within the CS site. Interior pages include a section navigation button that varies by where the page is located within the site hierarchy. In addition, section navigation menus are tied to these buttons (clicking on the button will lead to a section directory page).

For lengthy (multi-page) documents, additional (static) navigation menus are located below the section navigation button. Clickable links are white and small; headers are gold and larger. Unfortunately, this distinction only applies to IE. In Netscape, all document navigation is uniformly white. In addition, the links sometimes overlay the headers. Once again, Netscape's handling of Javascript and style sheets are to blame. With either browser, some multi-page documents have such lengthy menus that the menu bar extends below well below "the fold," thus defeating the purpose of providing easy document navigation.

Unit specific pages have unique headers with the unit name most prominent and a "Unit Directory" button with a hierarchical menu (all other features are consistent with the rest of the site). At this point, most units have only a few pages associated with them.

Recommendations

Navigation in the site is more difficult than it should be and several options exist:

Move persistent navigation to a top menu (possibly with tabs). If an integrated Technical Services site were created, this option would help unify the different departments' sites
Use hierarchical navigation for section navigation
Make top-level buttons of menu permanent so they will show up immediately; possibly hard-code the top-level buttons rather than rely on Javascript to create
Investigate other menu scripts that might work better with both browsers
Incorporate breadcrumbs or other location aids

Disabilities Access

The Checklist of Checkpoints for Web Content Accessibility Guidelines 1.0 provides three levels of checkpoints. Level 1 includes checkpoints a developer must satisfy to avoid making access impossible for one or more groups of users; Level 2 checkpoints includes significant barriers for easy use (developer should address); and Level 3 checkpoints are less critical barriers (developers may address). The U.S. Government Section 508 Guidelines (Electronic and Information Technology Accessibility Standards) are the ADA standards and are less detailed than the W3C guidelines.

The home page failed tests against both standards with CAST's Bobby Worldwide software.

Table 4. Home page as it appears without scripts (Section 508 test)

Welcome to SUL/AIR's Cataloging Services Department!

In support of the University's mission of teaching, learning and research, the Cataloging Services Department strives to provide intellectual access to SUL/AIR's collections by serving as a major player in the development of SUL/AIR's discovery and retrieval environment.

We are proud of the expertise of our cataloging staff. We invite you to learn from the pages within this site (see especially What We Do ) how each of our units make contributions to the faculty, students, and staff of Stanford University and provide intellectual access to SUL/AIR's rich collections.

For the W3C guidelines, failures included:

Level 1: poor readability if style sheet doesn't load; no alternate text for scripts; color used for information without alternatives; tables not fully labeled
Level 2: absolute text sizing used; some presentation hard-coded rather than through style sheet; obsolete tag language; foreground and background colors not contrasting enough; lack of general site layout description;
Level 3: language of text not identified; navigation unclear

(see Appendix D for full report)

For the Section 508 guidelines, failures included:

Poor readability without style sheets
Color used for information without alternatives
Tables not fully labeled

(see Appendix E for full report)

The same problems exist throughout the site. Some issues and suggestions for improvements have been noted above. Although no users with disabilities of the sort that would prevent them from using the site are employed in the department at this time, disabled users may exist outside the department and our own conditions may change.

Recommendations

Provide additional alternative pages for main directory pages rather than force users to rely on Javascript menus
Provide non-dynamically created document menus on first page of a multi-page document (or a stand-alone document navigation page)
Add alternate text for scripts
Check color problems and correct them
Use relative text sizing
Eliminate use of obsolete tags

Comparison to Other Equivalent Sites

The CS site scored relatively well in a comparison to seven other university's cataloging or technical services sites: Cornell, Princeton, University of Virginia, Harvard, Northwestern, University of Maryland, and Yale. Points of comparison included

Overview (scope of site; search, index, and sitemap features)
Home page (format, navigation, types of links, headers and footers)
Interior directory and document pages (design, header/footers, navigation/menus)
Overall design strengths and weaknesses

Some had better navigation, some more content, but none featured a similar consistency of design. Harvard's site was the most surprising because it was only a page of links - local documentation must be found elsewhere (perhaps on an internal server?). The sample was perhaps too small to reveal much. The main result was to underscore the usefulness of consistent design and good navigation for identifying where one was in a site and how to get to other areas.

The full report is located in Appendix F.

Recommendations

Work toward creating an integrated Technical Services website that is easily reached from the library home page.
Modify the menu layout to include persistent navigation (bar or tabs) for TS options

Conclusions

The CS site meets its goals of providing access to local documentation for policies and procedures. Informal surveys revealed that most users within the department utilize the site as they would most reference materials - on an as-needed basis. Some design features create barriers to usability and should be changed; cross-platform problems cause the majority of problems and should be addressed since Netscape will continue to be the default browser. Finally, the site compares favorably with sites created by similar departments at other universities.

This study was necessarily limited in scope. A number of questions remain unanswered at this time, including:

Do staff outside the department use the site, and if so, what information are they seeking?
Why has the rate of access increased by over 400% when users within the department rarely use the site?
Can site contributors and managers easily add or update content? If not, what changes can be made to site design and/or management?
Are documents sufficiently readable? If not, to what extent should they be edited and/or reformatted for the Web?
What documentation is still missing from site?

Resources

Bryce, A. (1996). Information tasks: toward a user-centered approach to information systems. San Diego, Calif.: Academic Press.

CAST. (2001). Bobby Worldwide. Retrieved March 6, 2002, from http://bobby.cast.org/bobby/

Frary, R.B. A brief guide to questionnaire development. Retrieved March 16, 2002, from
http://ericae.net/ft/tamu/vpiques3.htm

Hackos, J.A., & Redish, J.C. (1998). User and task analysis for interface design. 1^st ed. New York, N.Y.: John Wiley & Sons.

Krug, S., & Black R. (2000). Don't make me think: common sense approach to Web usability. 1^st ed. Indianapolis, In. : New Riders Publishing.

Lynch, P. & Horton, P. (1999). Web style guide. New Haven, Conn.: Yale University Press. (also available online: http://www.med.yale.edu/caim/manual/)

McNamara, C. (1999). Brief overview of basic methods to collect information. Retrieved March 19, 2002, from
http://www.mapnp.org/library/research/overview.htm

National Cancer Institute. Research-based Web Design & Usability Guidelines. Retrieved April 10, 2002, from http://usability.gov/

Nielson, J. (1997, July 15). "Search and you may find." In Jakob Nielsen's Alertbox Retrieved April 2, 2002, from http://www.useit.com/alertbox/9707b.html

Nielson, J. (2000). Designing Web usability. Indianapolis, In.: New Riders Publishing.

Nielson, J. (2000, March 19). "Why you only need to test with 5 users." In Jakob Nielsen's Alertbox, Retrieved March 17, 2002, from http://www.useit.com/alertbox/20000319.html

Rosenfeld, L., & Morville, P. (1998). Information architecture for the World Wide Web. 1^st ed. Sebastopol, Calif.: O'Reilly & Associates.

Trochim, W.M.K. (2001). Research methods knowledge base. Retrieved March 28, 2002, from
http://trochim.human.cornell.edu/kb/index.htm

United States. Architectural and Transportation Barriers Compliance Board. (2000). Electronic and Information Technology Accessibility Standards. Retrieved March 6, 2002, from http://www.access-board.gov/sec508/508standards.htm

W3C. (1999). Checklist of Checkpoints for Web Content Accessibility Guidelines 1.0. Retrieved March 6, 2002, from http://www.w3.org/TR/WAI-WEBCONTENT/full-checklist.html

W3C. (1999). Web Content Accessibility Guidelines 1.0. http://www.w3.org/TR/WAI-WEBCONTENT/