Intended Meaning: Benefits.
Test Users' Interpretations: Health field, money, health care is
expensive, Clinton's health plan,
hospital, don't know, benefits.
Jakob Nielsen
and
Darrell Sano
Much of the user interface work was done in a few weeks with four usability studies completed in a single week: card sorting to discover the users' view of the information space, an icon intuitiveness study, and two tests of the actual interface.
The conclusions from this project are that a uniform user interface structure can make a Web significantly easier to use and that "discount usability engineering" can be employed to base the design on user studies even when project schedules are very tight.
For security purposes, it was desirable to have separate visual identities for the internal and external pages (making it clear to users and authors when information could be accessed by outsiders). Also, the type of information available through the internal web was in many ways different in nature from the external information, meaning that a different design would be appropriate. Therefore, a separate SunWeb project was initiated in order to design Sun's internal Web.
SunWeb was designed during the summer of 1994, a period where the number of Web servers on the Internet grew at a rate of about 10,000% per year (doubling every two months). We suspect that the growth of internal Web servers inside Sun's firewall was even faster in this period. Many Sun employees had already built home pages for groups and individuals before the start of the SunWeb project and many more were coming online during the project. As another indication of the growth of the Web inside Sun, the SunWeb server was accessed by 2,649 different employees during a 34-day period of testing where it had not even been announced yet. Thus, work on SunWeb had to be completed within a few months if we were to have any hope of achieving a consistent user interface and not just glom together a set of disparate designs.
We were able to conduct four usability studies over a period of two weeks during which we tested the interface at various stages of design. We used the following sequence of studies:
Before our card sorting study, the SunWeb development group had brainstormed about possible information services to be provided over the system, resulting in a list of 51 types of information. We wrote the name of each of these services on a 3-by-4 inch notecard. In a few cases we also wrote a one-line explanation on the card.
Before each user entered the lab, the cards were scattered around the desk in random order. The users were asked to sit down at a table and sort the cards into piles according to similarity. Users were encouraged not to produce too small or too large piles but they were not requested to place any specific number of cards in each pile. After a user had sorted the cards into piles of similar cards, the user was asked to group the piles into larger groups that seemed to belong together and to invent a name for each group. These names were written on Post-It notes and placed on the table next to the groups. The users typically completed the entire process in about 30 minutes, though some took about 40 minutes.
The data from this study was lists of named groups of cards with subgroups corresponding to the original piles. Based on this information, it is possible to calculate similarity ratings for the various concepts by giving a pair of concepts one similarity point for each time both concepts occur in the same larger cluster and two points for each time they occur in the same pile. This similarity matrix can then be fed to one of the many standard statistics packages for a cluster analysis. It is also possible to use other statistical techniques such as multidimensional scaling.
We actually computed a statistical cluster analysis at the end of the project, but we initially did not have time to code the data appropriately, so our design was based on "data eyeballing" and not on formal statistics. Given our discount usability engineering approach with only four users, the statistical methods are not that reliable anyway but, as it turned out, the statistical cluster analysis was very similar to that we had constructed manually. For our manual clustering, we worked bottom- up and initially grouped concepts that most users had sorted together. We then expanded these small groups into larger clusters by adding concepts that some users had sorted with most of the concepts in the group if the grouping made sense to us. This subjective interpretation of the data is dubious if the objective "truth" is desired, but in our case we were after a coherent design, so we felt justified in applying our judgment in those cases where the user data was too sparse for a clear conclusion to be drawn on the basis of the numbers. Eyeballing the data took one person about two hours and resulted in a list of recommended groups of features as well as initial names for the groups. The names were often inspired by the labels provided by the users, but due to the verbal disagreement phenomenon [Furnas et al. 1987], the users obviously had not used exactly the same terms, so we also had to apply our judgment in choosing appropriate names.
Intended Meaning: Geographic view of the company
(branch offices in different locations).
Test Users' Interpretations: World, global view, planet, the
world, Earth.
Intended Meaning: Benefits.
Test Users' Interpretations: Health field, money, health care is
expensive, Clinton's health plan,
hospital, don't know, benefits.
Intended Meaning: Public relations (TV with commercial).
Test Users' Interpretations: TV set, video, TV, TV, TV.
Intended Meaning: Product catalog,
Test Users' Interpretations: System oriented, disk, CD, Computer, CD-ROM, CD-ROM.
Intended Meaning: Specialized tools (toolbox).
Test Users' Interpretations: Briefcase, personal info, briefcase, toolbox, briefcase.
Intended Meaning: What's new (bulletin board).
Test Users' Interpretations: Bulletin board,
bulletin board, bulletin board, laundry.
Intended Meaning: World Wide Web.
Test Users' Interpretations: Networking on a world scale, map,
location, dimensions of the planet,
networking around the world,
geography, global.
Table 1. Results of icon intuitiveness study with four users (some users gave more than one interpretation of some icons).
In other cases, users did not get exactly the correct meaning of an icon but we decided to keep it anyway. The product catalog was one such example, because the CD-ROM component of the icon was so visually powerful that most users focused on it and not on the group of system components. The underlying problem is that software is an abstract concept that is difficult to visualize. In another icon we tried to show "software" by a code listing but that also proved difficult for users to recognize. Our final decision was to keep the product catalog icon since users did recognize its components. In the final design (see Figure 4), we made the workstation screen more visually prominent by giving it a more saturated color.
With respect to the "what's new" bulletin board, one user claimed that it looked like laundry. Even so, three of the four users recognized the icon immediately and we decided to keep it. When conducting user testing with a small number of users it is important not to let results from individual users influence one's decisions unduly. Seen from one perspective, 25% of our sample had trouble with the icon, but seen from another (and more appropriate) perspective, we happened to have asked a person who did not recognize the icon. When designing based on small samples of test users, one has to apply judgment based on general usability principles and one's experience with interaction principles since the data itself is too sparse to use without interpretation.
The toolbox icon was seen by most users as a briefcase. Since briefcases have completely different connotations, this icon would have been misleading to users and we decided to redesign the it by adding a monkey wrench to emphasize the "tool" aspect. Finally, the World Wide Web icon was reasonable: some users got it almost exactly right by guessing "networking on a world scale" or some such, but many of the interpretations had to do with geography or the globe. Since we already had a globe icon to signify geography, these erroneous interpretations of the WWW icon were problematic and we decided to redesign with more emphasis on the network. We also replaced the globe with a flat map to distinguish the WWW icon further from the geography icon.
Figure 1. Card distribution to icons in the usability laboratory. In preparation for the test, the icons were printed out in 200% magnification on a color printer and Post-It tape was used to divide the table into areas for each icon. For the test, the user is given a pile of cards and asked to place each card in the area with the most appropriate icon. An observer is in the room with the user to give instructions and take notes. Other observers can watch though the one-way mirror and the session is also videotaped in case further analysis is necessary later.
We used Post-It(TM) tape to divide the desk into areas for each icon, and each of test user was then asked to distribute the cards among these areas, with each card going to the (labelled) icon most appropriate for it. At the end of the card distribution test, the users were asked to comment on the aesthetics of the icons and to list the icons they liked the most and disliked the most. Figure 1 shows the lab setup during the card distribution test. The card distribution tests took about fifteen minutes per user.
The card distribution and page walkthrough studies revealed several usability problems in our revised design, leading to additional design changes before we could release the SunWeb user interface. One of these new usability problems is discussed in further detail in the following section. In retrospect, though, we feel that we learned the most from the first two studies, the card sorting and the icon intuitiveness study. In general, we will always recommend thinking aloud studies where users perform tasks with the full user interface, but in the special case where the command structure and interaction techniques are determined by the World Wide Web viewer it was more important to study the information structure that we could actually influence.
Asking users for subjective ratings of icon aesthetics proved to be useful. Even though people have different taste and liked and disliked different icons, there were two icons that were singled out for criticism by most users: a blackboard we had used to represent education and the TV icon we had used to represent sales, marketing, and PR. Users had no trouble recognizing the TV (see Table 1) and most users easily understood that it represented promotional materials (though only a few of them recognized it as showing a "Sun TV commercial," which was one of the services that we planned to make available through SunWeb's multimedia features). Users just did not like it.
Since subjective satisfaction is at least as important as task performance for a system like SunWeb where we want to encourage people to browse and learn more about their company, we removed these most hated icons and replaced them with more attractive ones. The blackboard was replaced with a graduation cap and a diploma to represent education, and the TV set was replaced with a spotlight to represent promotional materials. Even though the American-style graduation cap is problematic from an international use perspective (not all countries use these caps), we decided to use it for this system because it is intended for internal use in Sun which is a company where most employees understand English and basic aspects of American culture.
Figure 2. Five iterations of the icon to represent a group of special-purpose applications. The first two icons use the metaphor "application toolbox," the next two are "application stores," and the last is the "application chest."
As mentioned earlier, the initial toolbox icon was interpreted as a briefcase by most users, so we opened it up and added a monkey wrench. This redesign worked and users in our card distribution study and page walkthroughs had no problems recognizing the icon as a toolbox. Unfortunately, the tool metaphor was too strong and users sorted a very large numbers of cards onto this icon with the comment that "oh, this is a tool." Essentially, almost any concept that represented an executable program was considered a tool. An example is the expense report application which should have been grouped with the travel icon but was often placed in the toolbox.
In order to use a weaker metaphor for the special-purpose applications, we next tried a shopping metaphor with an icon showing a storefront (the middle icon in Figure 2). When we conducted an icon intuitiveness study and showed this icon to a user, he immediately said "this is a circuit board." This user happened to be an engineer, but since we do have a very large number of engineers in the user population for SunWeb, we decided to take this comment seriously and redesigned the icon. This is an example where our judgment as user interface specialists was to rely on a result from a single user since we felt that this user's problems would be frequent in real use of the system.
We tried several alternative storefront and other shopping icons (see the fourth icon in Figure 2) before realizing that a successful shopping icon would interfere with one of our other interface elements: the "product catalog" icon. Therefore, we dropped the application store as the metaphor and we finally settled on the "application chest" icon shown as the last icon in Figure 2.
Figure 3. SunWeb main homepage masthead (top) and second level masthead (below).
A systematic design solution was needed to provide page identification and access to essential controls, and this was accomplished by developing "banners" (see Figure 3). Present at the top of every SunWeb page, the banners provide a category of essential global controls, including Search, Overview, and Help (C). The vast scope of information included in SunWeb required a Search engine as a prerequisite from the initial functional specification stage. An Overview in the form of a comprehensive content list was also deemed necessary. These buttons were rendered in a somewhat subdued grey, so as not to compete or overpower the other elements on the page. The subdued uniformity also visually separates functionality from the more colorful category icons found on the home page. The SunWeb identifier mark (A) is consistently located on the left half of the banner. This logotype area provides SunWeb the necessary visual distinction from the "look" of Suns' external public server. This area also functions as the link mechanism back to the main SunWeb homepage from anywhere in the system, and is reinforced by a descriptive label. On second level banners, the paging buttons (D) are highlighted when available to allow users paging capability through a series of information, for example, the homepages of members of a certain engineering team, or pages in a product catalog. This avoids the back-next selection cycle, a tedious routine for users if numerous categories of information are available. The 15 different categories of SunWeb information represented by icons are integral second level banner components (B). These icons embedded in the banner function as direct links back to that particular category homepage. In SunWeb, these 15 categories may be thought of as major "hubs," as found in airline route systems, and provide the major arteries through the information space. The ability for users to quickly return to these central homepages was a crucial navigational feature. By clicking on the icon, the user will immediately link to that category homepage, no matter how removed the user may currently be from the second level. The banner icons also function as location cues for user position within SunWeb.
In order to facilitate consistency among contributors to SunWeb, banner components were placed in a central repository, and usage encouraged by the development team. Although not mandated to adopt the banner gif files or adhere to a particular design, many content contributors felt it necessary to "fit-in", and wished to appear consistent with the rest of the top-level SunWeb environment. In this instance, a suitable design solution "given away" provided the initial momentum for widespread exposure and adoption. Banner specification drawings helped to clarify the design, and insured proper implementation. These drawings were distributed in hard copy during the development phase meetings to the various content providers.
As with the icons, the SunWeb banners relied on a minimal color palette in order to reduce the possibility of color-map flashing and unpredictable results. This also would reserve more color space for the content providers, who may feel it necessary to include an undetermined amount of continuous tone photographs, color illustrations and charts, etc., into their SunWeb pages. Subsets from a 64 color palette were adopted for all graphic elements in SunWeb. This would insure hue saturation, intensity, and value consistency across components.
Figure 4. The final home page design for SunWeb. Icons sharing design attributes, such as frontal orientation and color palette, insures a visual continuity across the image ensemble. Each icon occupies a similar amount of space within the button graphic, so as to equalize image density. Pre-defined colors are used throughout the icon imagery which represents the 15 major categories of information provided by SunWeb. This provides the necessary visual contrast with the global icon controls in the SunWeb banner, which visually remain monochromatic. Grouping and placement of the categories on the page reflects a hierarchy based on related topics and information precedence. Higher priority information is placed at the top, while categories of lesser importance towards the bottom. It is interesting to note that first iteration of the homepage design presented each icon as a separate graphic ismap. However, due to the length of time required by Mosaic to load each individual image, one at a time, the homepage was redesigned as one large ismap containing all button categories. This dramatically reduced access time for users. Textual labels in the smallest font available was positioned below the ismap (not shown), in case users turn off the load picture functionality. A design should always take into account users who are concerned with speed of access, and therefore prefer not to load any of the graphic gif files.
Mullet, Kevin, and Sano, Darrell (1994). Designing Visual Interfaces: Communication Oriented Techniques. SunSoft Press/Prentice Hall, New Jersey.
Nielsen, J. (1993). Usability Engineering. Academic Press, Boston.
