Metaphors: Realistic vs. Form-based Phone Interfaces

Experiment

Introduction and Hypothesis
    There are two presentation strategies for a program whose feature set roughly mimics a "real-world" object. One approach is to provide a metaphor for that "real world" object and map the features of the program onto that metaphor, such as IBM's RealPhone. Another approach is to ignore the "real world" metaphor and simply design the program with a standard Windows 95/NT interface, which would be form-based and include the standard Windows 95/NT buttons, labels, text boxes, etc.

    We theorized that the 3D metaphor of a realistic phone would not translate well to the 2D environment of RealPhone.  For example, tasks such as picking up a receiver have no corresponding action on a computer screen.  Also, we believed that most computer users would be expecting a normal Windows interface, and that a realistic phone interface would initially confuse them.  However, we felt that users would find the graphical look of RealPhone more appealing.  Based on this theory, we hypothesized that the RealPhone interface would yield longer task completion times than the form-based interface using Windows widgets, but would rate higher on a subjective satisfaction scale.

    The independent variable in our study was interface style.  The two treatments were a real-world metaphor based on RealPhone and a Windows 95/NT form-based phone interface. The dependant variables were speed of task completion and subjective satisfaction.

Pilot Study Results
    Six people participated in our pilot study.  A number of problems became evident during the pilot study, such as:

• Most subjects were confused that they had to press "dial" before dialing the number, especially when performing the form-based phone tasks first. We changed the design to allow the subjects to enter the number first, and then press "dial."
• The task window originally obscured the instructions.  We instituted a requirement that the study be run at a resolution of 1024x768 to allow sufficient screen space.
• Subjects were having a great deal of difficulty entering 11-digit telephone numbers due to memory overload and clutter.  We limited phone numbers to a length of seven digits.

    Based on the results of our pilot study, we decided to cap all task completion times to sixty seconds.  This reduced the effect of any outliers.

Subjects
    Twenty-two undergraduate students from the University of Maryland with varying levels of computer experience participated in the experiment. Before running the experiment, each subject was asked to complete a Background Survey which asked about time spent on a computer per week, operating systems they used the most, their level of computer experience and whether or not they had ever used IBM's RealPhone.  Sixteen out of twenty-two subjects used Windows 95/NT most often.  All of our subjects were familiar with the speed dial feature.  One subject had previously used RealPhone.

Subjects estimated the number of hours they spent working a computer per week.	Subjects rated themselves in computer skills from novice (1) to expert (9)

Materials
    Hardware:  The experiment was conducted on Pentium-class computer systems running Windows 95 or Windows NT 4.0.  The monitors were set at 1024x768 resolution and were all 15".  All participants used a two-button mouse.

    Materials:

• Informed Consent Form
• Instructions
• Preliminary Background Survey
• Visual Basic Test Tool
• Task List
• Subjective Satisfaction Questionnaire
• Comparison Survey

    Interfaces:  The realistic metaphor interface was derived from IBM’s RealPhone program. The program featured a typical business phone, complete with speed dialing and frequent-call list features. We adapted the interface for use in our program.  The form-based interface was designed using standard Windows 95/NT widgets.  It was designed to have the same dimensions and feature placement as the realistic metaphor interface in order to reduce the effect of Fitts' Law.  By placing the controls in the same relative locations, we eliminated pointing time as an important factor.

RealPhone Interface	Form-based Interface

Procedures
    The experiment used a within subjects design and was balanced between the two interface treatments.  Before the subjects began the study, the experimenter specified the order in which the two interfaces would be presented.

    Subjects read and agreed to an Informed Consent Form.  They were then presented with a Background Survey to determine their approximate amount of experience with computers, Windows 95/NT, speed dial features, and IBM's RealPhone.  Subjects also read through an Instruction Screen which presented an overview of the tasks to be performed.

    Subjects were asked to complete a set of twelve tasks for each of the two interfaces.  The tasks involved clearing the display, dialing phone numbers, and speed dialing.  The two display-clearing tasks asked the subject to enter a phone number and then clear the display using the flash button.  The six dialing tasks asked the subject to enter a phone number and then open a line to dial that number.   The four speed dialing tasks asked the subject to store a phone number in the frequent dialing list, then place that number in speed dial and dial it.  The Test Tool recorded the subjects' task completion times.

    For each task, subjects were shown an instruction box in the upper left-hand corner of the screen, which detailed a task they had to complete.  After thoroughly reading the task instructions, subjects were presented with the first interface.  Once the subjects correctly performed the task, the screen displaying the interface closed and they were presented with their next task. When they completed all tasks for the first interface, the subjects were given a Subjective Satisfaction Questionnaire.  They then performed the same set of tasks on the second interface and again filled out a Subjective Satisfaction Questionnaire.  When the subjects finished with both interfaces, they were given a Comparison Survey, which asked them to compare the two interfaces.

Problems

• The layouts of the RealPhone and form-based interfaces were nearly identical.   Because of this, subjects were able to complete the second set of tasks much more quickly than the first set.  This led to a very high standard deviation in the data for some of the tasks.
  
• Tasks 1 and 2 required subjects to clear the display.  Task 3 was a dialing task, but many subjects either did not read the instructions carefully enough or expected that task 3 would also require that they clear the display after dialing.  This was probably due to poor task organization and resulted in longer task times than would otherwise have been recorded.
  
• Most subjects tried to use the keyboard to enter a phone number.  Many took a long time to realize that the mouse was the only usable input device for dialing.  Also, some subjects believed that they had to click on the number display before dialing phone numbers with the mouse.
  
• The sliders did not reset themselves in the Subjective Satisfaction Questionnaire.   When filling it out for their second interface, subjects tended to only alter their evaluation by one or two values, resulting in biased results.
  
• Subjects who lived on campus tried to dial only the five-digit extension for numbers with a 314 (campus) prefix.  Also, some subjects dialed an area code in addition to the number presented in the task because Maryland now requires that the area code be dialed for local numbers.
  
• The telephone number used in task 9 began with a "1" and was consequently confusing.  It was inconsistent with actual phone numbers, which cannot begin with "1."
  
• Papa John's phone number (277-7722), used in task 11, was familiar to some subjects who ate lots of pizza.  Subjects who did not frequent Papa John's did not have the same advantage.
  
• Certain tasks were too leading, such as task 9, which told the subject specifically how to open the frequent-call list.
  
• The RealPhone interface should not have been inside a form.  According to IBM's RealPhone website, the window border and controls "completely disrupt the visual metaphor of the application".  Unfortunately, we were unable to remove the form in Visual Basic.

Department of Computer Sciences Direct questions and comments to the student editorial team