The Effect of Direct Annotation on Speed and Satisfaction

Introduction

The work of annotating photos and maps can be a cumbersome and error-prone task even for many professionals in the field. In the past, it was a task that was very tedious to perform and to organize. With the advent of many tasks being computerized in the past few years, the task of the annotation of graphics was sure to follow. And through development of software such as PhotoFinder (Shneiderman & Kang 2000), FotoFile (Kuchinsky / Pering / Creech / Freeze / Serra / Gwizdka 1999) and NSCS Pro (Norton 2000), the annotating task is possible through computer software. However, there is an important question; what is the best method of annotating photos? What kind of user interface design is the most efficient and rewarding to the user? We seek to conduct an experiment using three different methods of annotation.

In our experiment, we have three methods of annotating a photo: direct annotation, click & type, and textbox. Out of the three, direct annotation stands out from the rest due to the fact that direct annotation is fully based upon the concept of direct manipulation. In the textbook, "Designing the User Interface" (Shneiderman, 1998) , direct manipulation is promoted for its ease of learning, low error rate, and potential speed. In the textbook, direct manipulation is defined as follows: "The central ideas in the systems that inspire such delight are visibility of the objects and actions of interest; rapid, reversible, incremental actions; and replacement of complex command-language syntax by direct manipulation of the object of interest (Shneiderman, 1983)." To make the point, the book gives a excellent example of driving a car as direct manipulation. As the book puts it, if driving a car was a command based, a driver would need to input "Left 30 degrees" whenever a driver wishes to make a left turn; this is rather ridiculous and probably very unsafe to a driver. The book argues that this is the level of what we use in many of our applications today. Some applications have improved; for example, typing "delete the first letter of the third word" in a word processor will be very cumbersome. It will be more visually helpful and quicker if you could just click the letter you want to delete and hit the backspace, which can be done in today's word processors.

With the advantages of direct manipulation in mind, we have come up with three possible methods to annotate a photo that might be useful for comparisons. The direct manipulation method, named direct annotation, uses the list of names from which a user can click with a mouse and drag onto a photo; basically a drag and drop method to annotate names on a photo. The second method, click & type, is a variation of the first method so that a user can click on a photo and type in names, in which after the typing is done it becomes an annotation on the photo. This is a variation of direct manipulation because it is a hybrid; the part where a user can click on the photo is closely related to direct manipulation. However, the part where a user has to type in a name is a somewhat cumbersome form of direct manipulation in that if the user is annotating several people of the same name, the user has to type the same name several times. The third method is the traditional caption form in which a user has to type in names in a empty textbox provided, in which the leftmost person is typed first, and the rightmost last.

The software that we used for the experiment, "Direct Annotation: A Drag-and-Drop Strategy for Labeling Photos" (Shneiderman & Kang 2000) is a PhotoFinder prototype that is in development at the HCIL in the University of Maryland. PhotoFinder is being developed especially with important user interface design principles such as direct manipulation and user satisfaction in mind. Other software of note that deals with image annotation are FotoFile, and NSCS Pro. FotoFile is an experimental system for multimedia organization and retrieval, based upon the design goal of making multimedia content accessible to non-expert users. Search and retrieval are done in terms that are natural to the task. The system blends human and automatic annotation methods. It extends textual search, browsing, and retrieval technologies to support multimedia data types. In NSCS Pro, users can tag picture records with unlimited numbers of Descriptor keyword. Also, you can create two kinds of information labels, ID labels and Caption labels. ID labels contain the primary information like what the picture is and where the picture was taken. ID label information is saved to a database, this information can be used as a lookup for speeding up new data entry and for searching and filing. NSCS Pro also can create caption labels that can contain a large amount of free-form information to further amplify or expand on what's in the ID Label. Other photo library programs exist on the internet such as picture sharing software programs. At http://www.gatherround.com/, one could “create password-protected picture albums and invite people to see your creations”. This site offers the user methods of annotating their photos by way of clicking and typing into specific fill boxes which aim to help the user organize, arrange and edit his or her photos. Another relevant feature is the picture e-mail option which allows the user to specify one to six picture to send someone with three possible delivery styles. These styles include “Carousel, Photo Cube, and Slide Show”. The selection process involves the user to click the checkboxes of the picture(s) he or she wishes to send in the e-mail. With photo sites as such, one can sign up for a login ID and password which will better ensure protection over the network.

There have not been any previous experiments that we found that dealt with the problem of annotating photos or maps. While there have been several studies on voice annotation of documents and other forms of annotation, we believe that this experiment is one of the first that explores the annotation of graphics such as photos and maps. Other relative application programs with annotation techniques include Microsoft Word 2000, Microsoft Excel 2000, Microsoft Access 2000, and Microsoft Powerpoint 2000. Among these Microsoft Office 2000 application programs, Microsoft Word 2000 offers the user to create mailing labels with use of the mail merge function, and Microsoft Excel 2000 allows the user to change chart data labels as well as cell labels. In Microsoft Access 2000, one may apply a label on a form, report, or data access page, and see that it is automatically allocated to its destination. With Microsoft Powerpoint 2000, one may write or draw (annotate) on slides during a slide show. These methods of labeling are similar to that of clicking and typing as opposed to direct annotation. The direct annotation technique seems to be a modern method to labeling which could very well be gaining popularity, whereas, the click and type and text box type method have been dominant in the past.

The experiment tested subjects for time completion and subjective satisfaction. The factors mentioned in Dr. Shneiderman’s textbook lists the advantage of direct manipulation as: faster learning and higher retention, errors are more easily prevented, and higher satisfaction from using the system. This suggests that direct annotation method should have better time and better subjective satisfaction compared to the click & type and the textbox methods. Psychologically direct manipulation enhances the positive feelings of the user being in control, so the subjective preference should be highest. Next, click and type, which would be the in-between of the three, should place second in satisfaction due to its positive effect of direct manipulation paired with the negative effect of cumbering achieved by the repeated typing of click and type. The textbox method should generate the lowest satisfaction due to its dull nature; we’ve all seen captions that mention names from left to right before.

For the time factor, direct annotation and textbox should have the best scores for the time. They both utilize only one part of the keyboard/mouse combination, so there is no lag time for switching back and forth. However the click and type method requires the user to switch back and forth, which should decrease speed significantly.

One could assume that the greater the subject list, the greater the chances are that one would acquire a more exact measurement of the average user performance. Since this experiment did not contain such a vast group of subjects, other data can play an important role in analyzing any skewed results which may appear after the experiment comes to an end. Such pertinent data would be user comments, actions, and energy evaluated by the experiment conductor, and later compared to the statistical output from the subject list.

Department of Computer Science: Direct questions and comments to the student editorial team