Authors

Wendy Adams- wadams@telocity.com
Jiraphan Brown- djlb@erols.com
Dorothy Rapeepun- drapeep@wam.umd.edu
Wayne Williams- wcwill@wam.umd.edu

Abstract

This experiment examined the effectiveness of three online help systems: text only, animated images only, and integrated interactive. The three help systems were implemented for the software application PhotoFinder developed by the University of Maryland HCIL (Human Computer Interaction Lab). Each help system was tested with ten different subjects. All subjects were given the same tasks to complete. We hypothesized that of the three help systems the integrated interactive help would improve a user's performance of completing a task the most. We also evaluated subjects preferences of help systems. Results showed that there was no statistically significant difference in users performance of completing a task between the three different kinds of online help.

Everyone, at one time or another, has had to reach out for help, whether to find directions when they’re lost, or to learn to use a Palm Pilot.  Using a new software application is no different and often requires extensive and well-written help facilities.  Unfortunately, most application users find an application's help system to be far from useful.  

Some of the problems the average computer user has incurred with application help systems include:

• Selection rules that are rarely present
• Help systems that are to large for the novice user
• Help systems that are organized according to system functions and not user goals
• Help systems that discuss how the system works and not how the help menu is to be used
• Help systems that rarely present a complete method and therefore explicitly forcing the user to problem solve the simplest tasks. [Web cited]

Evaluating the effectiveness of various applications’ help systems is an important and active part of research today.  Despite the time, effort, and money companies have spent discovering what does and does not work in an application’s help system, users still have problems finding the answers to their questions.

There is no doubt that today’s software products are facing the ever-increasing challenge of attempting to satisfy the needs of both novice and expert users.  For a software application to be considered a success, the help facility must serve users’ needs in an efficient manner.   Users should be able to find the information they are looking for in a matter of seconds.  The help menu should offer an intuitive, clear, how-to instruction on performing each action that the application provides to the user.  The help information provided by applications has great potential to frustrate or facilitate the user’s work.  Users typically want to do their work and can devote limited attention to learning new user interfaces.  System documentation can be a confusing collection of reference manuals, keyboard templates, user guides, and online documentation.  Often to make use of them one must already know what feature or function is appropriate for the task at hand.  Help information that is focused on the user’s task saves time and frustration.

Choosing the right kind of help facility for a software application is a difficult decision due to the variety of help systems application developers have to choose from. The two most common forms of help systems available today are either online or offline. Currently, the only available means of providing users with offline help are printed tutorials, quick reference cards, getting started notes, alphabetic command lists, and reference manuals. Text-based environments typically present help in the same manner as the information being described. In other words, text demonstrations follow text commands. On the other hand, there are a wide variety of online help systems available. It is easy to understand why online help systems are a replacement for hardcopy manuals. The following are the advantages of online help systems:

• Greater availability: Online information can provide a reliable source of information for all software packages and on all platforms.

• Easier access: Online help systems provide for efficient access to the relevant information.
• More interaction: Online both the user and the system can interact with the information.
• High accuracy: online documentation requires shorter development cycles and is more easily updated compared to hardcopy documents.
• Low cost: In general, online documentation is less expensive to store, reproduce, and distribute.
• Multimedia and AI: Online information can exploit multiple media, such as video, sound and animation, and can apply techniques from Artificial Intelligence (AI) [13].

The two most common forms of online help systems are external-application and inter-application help systems. The presentation possibilities are expanded by online graphical user interfaces (GUI) running on multimedia-capable hardware. The versatility of these new interfaces leads us to consider possible approaches to utilizing GUI interactions.

External-application help systems are provided outside the application and are either implemented as an HTML web page or as part of the application itself. External-application web page help systems are becoming more popular because PC (personal computer) users are becoming more accustomed to the World Wide Web. Besides, emerging standards are making HTML-based help systems easier to navigate, view, distribute, and link to applications. The lure of HTML's cross-platform compatibility and the ability to update the content of online help systems dynamically and distribute the information over networks has driven most of the effort to convert printed documentation to HTML. It is now feasible, but far from convenient, for the user to access an application's help system online. External-application help facilities are only convenient for those who are directly connected to the Internet and are not required to dial an Internet Service Provider every time they press the application's help button. Other disadvantages of external-application help systems are slow download speeds, current HTML standards are not equipped to handle the rigors of large scale online publications, effective online documentation requires capabilities that are not part of the broad HTML feature set, and down servers that leave the user without a usable help system. Some application designers have completely eliminated the problems of HTML web page help systems by selling the applications help facility with the application. Therefore, when the user installs the application onto their system the help facility will be installed at the same time. As a result when the user presses the applications help button they will be accessing the help facility's information on the users hard drive. Thus eliminating the problems of HTML web page help systems. As wonderful as external applications sound their disadvantages include:

• When users invoked help, they switched out of the application they needed to use to complete their task.
• When users tried to follow the instructions, they switched out of help, and another application's windows often covered their instructions.
• Because of the two problems just noted, users had to read and remember a series of instructions before they tried to perform them.
• Users often got out of step with the instructions, sometimes failed to perform instructions.
• Users are often confused by the illustrations in the instruction windows, mistaking them for functioning interface elements that they could click. [6]

Inter-application help systems are integrated within the application and possess the characteristic help facilities lack. Inter-application help systems most commonly consist of wizards, animated tutorials, non-animated tutorials, animated helpers, non-animated helpers, context sensitive, and task oriented help systems. Integrated help systems seem to be popular in systems where lessons on how to use the application are done graphically and interactively, showing the user how to perform the action. These help facilities are most commonly implemented as frames that pop up inside the application. Frames are advantageous over web pages because they are integrated within the application and the user does not have to flip back and forth between the application they are working with and a web page to follow the instructions given by the help facility. Through the use of inter-application help facilities the user also does not have to memorize or print the instructions given by the application's help system because the instructions are integrated within the user interface.

One of the biggest problems with any online help system is that users in need of assistance typically do not utilize the online help information. Subjects cited "poor quality of the help information" as a common reason for avoiding online help [2]. Not to mention the fact that the common textual form of online help tends to hinder rather than help the users performance of procedural tasks. The traditional form of online help has not captured the visual dynamic nature of modern graphical user interfaces. Many of today's applications provide text based on-line help systems to assist users in performing tasks. Developers have neglected the capabilities of using graphics and animation in their help systems. The Comparison of Still, Animated, or Non-illustrated On-line Help with Written or Spoken Instructions in a Graphical User Interface describes a study of 176 undergraduates who received on-line help instructions for completing seven computer-based tasks. Instructions for the participants were provided in either written or spoken form and with or without animation or still graphic visuals. The results revealed that visuals, either animated or still graphic, in the online help instructions enabled users to perform more tasks in less time and with fewer errors than did users who did not have visuals in the on-line help system. Surprisingly this study found an insignificant difference between the subjects' that used the still graphic and subjects' that used the animated help system. However, the data did suggest a slight advantage for those that used the animated help system [5].

Animation in computer interfaces can actually mean many different things. Animation can consist of moving an object within a scene, or the object may change its appearance as it is moved. A scene may be larger than can fit on the computer screen, and the viewpoint can be changed with animated movement by rendering in-between frames part way between the starting and ending state similar to a movie. There are numerous other types of animations as well, including one we are all familiar with-Microsoft Word's Clippie.

Interactive animation has been used to shift some of the user's cognitive load to the human perceptual system. The perceptual phenomenon of object consistency enables the user to track relationships without thinking about it, and thus easily relate an animated help system to the application itself. Ben Bederson's study, Does Animation Help Users Build Mental Maps of Spatial Information, found that animation improves users' ability to reconstruct the information space, with no penalty on task performance time. Bederson believes that his study provides strong evidence for adding animated transitions in applications with fixed data where the user navigates around the user environment [1]. In other words animation could prove to be extremely useful to computer users while searching for answers in an application's help menu.

In today's computer world, ease of use is becoming a critical part of an applications survival. While applications are becoming more complex, customers are hoping that the designers can make learning the application easier. This is why a well-designed online help system is becoming an essential part of software applications. Researchers have discovered that software application users questions can be classified into five general forms: goal questions ("What kinds of things can I do with this program?), descriptive questions ("What is this? What does this do?"), procedural questions ("How do I do this?"), interpretive questions ("Why did that happen? What does this mean?"), and navigational questions ("Where am I?") [6]. Designers of help facilities that are responsible for identifying the content and structure of menu systems should test their help facilities by my making sure the 5 user questions are answered for all application tasks. Users need to remember that Designers are subjected to various constraints when preparing online software documentation. The constraints are grouped into two categories: system constraints and user constraints. System constraints include screen size, access mechanism, memory and disk space, and lack of context. User constraints include the need for a quick solution and a variability in user sophistication.

Help systems are most effective when the information supplied is a good match for the user's background, what they are trying to accomplish, and the information is supplied in a way that is immediately useful to the user. A well-designed online help system should bring relevant information to users when they need it and guide them through the interface just as a human teacher would. It helps users with a problem as efficiently as possible without requiring the users to study a topic in depth. Currently, the designers of help systems are trying to discover one help system that will effectively help all users. Instead, they should be trying to discover a help facility that most effectively helps a group of users (elderly, novices, etc.) and applications should be shipped with all help facilities so the user can choose the one that most effectively helps them.