This experiment evaluated history mechanisms from three web browsers (Netscape, Internet Explorer and PadPrints) and their effect on re-visitation of web pages. The subjects were required to build a history of web pages and t hen asked to revisit those pages in order to answer questions. Written instructions were presented to each subject outlining the tasks of the experiment. The tasks instructions were presented to the subject using a software program written in the Tcl/Tk scripting language. User satisfaction questions on the history feature were also presented by this program as the final step. In order to test the subject's memory, we presented all 20 thumbnail images on the screen using a Microsoft Visual Basic program. These thumbnail images were accompanied by tool-tips that gave the user a textual clue to the contents of the page.

We hypothesize that there will not be a statistically significant difference in the speed of performance of PadPrints compared to the history styles of Netscape and Internet Explorer, due to the learning time of the PadPrints Environment. However, we believe user satisfaction ratings will be significantly higher with PadPrints as companion to either browsers.

The experiment can best be described as three split groups.

• Independent Variables

The independent variable tested by the experiment was the History Style of the web browsers.

Table 1 shows that each subject tested only one of the three history styles. The experiment was designed this way to avoid any interaction effects.

The three web browsers offer history mechanisms that are different in structure. Netscape 4.0's history is displayed in a separate window providing a tabular text description, URL, date and statistics on visitation (refer to figure 1). Internet Explorer's history feature displays a folder and file structure representation, much like the structure used in the Windows 95 Explorer (refer to figure 2). PadPrints, very different from the other two, gives a graphical display in the form of thumbnail images of the pages that have been previously visited. The images are accompanied by URLs in the status bar region of the window. The hierarchical ordering of the thumbnails represent the visitation order (refer to figure 3).

Three treatments were given in order to test the history styles. They were:

1. Tabular Text (Netscape 4.0).
2. Hierarchical Text ( Internet Explorer 4.0).
3. Hierarchical Graphical ( PadPrints).

• Dependent Variables

Three dependent variables were analyzed in studying the different treatments :

1. Time to complete the tasks.
2. Errors.
3. Subject satisfaction ratings.

Dependent Variable 1: The time to complete the task was measured from the time the subject initiated a request for a new question until they completed the task by providing an answer. This was measured and recorded by the TCL/TK program. The mean time required by the users to locate and answer each question was then calculated for each of the treatment groups.

Dependent Variable 2: The average number of errors (incorrect answer selection) was calculated manually.

Dependent Variable 3: The subjective satisfaction level of each history style was measured, based on ten questions referring to the following areas : overall reaction, screen layout, searching and navigation. This data was collected from the questionnaire that the subjects complete after completing the tasks.

To supplement these variables, the effect of different history styles on memory was also considered. To test the subject's memory after using a particular history mechanism, he/she was presented with a five minute recall task. The program referred to as "Memory Recall" recorded the time and the subject's selection order of the web pages.

One pilot subject was experimented on each of the treatments. All data was collected and analyzed. This included the data on the three dependent variables mentioned above. Also, suggestions were taken from the pilot subjects including the layout of the screen, tasks, etc. We made the necessary changes to the experimental setup.

The pilot subject's tests revealed that some of the questions in the pre-tasks and tasks were ambiguous and difficult to understand by the subjects. Thus, the questions were reconstructed to make them clearer. A time limit was set up in the Memory Recall program as the subjects were taking too long to mark the order in which they visited the web pages. Also, changes were made to the layout of the screen, concerning the placing of the dialog boxes on the screen.

Thirty students participated in this experiment, most of whom attend the University of Maryland. The subjects consisted of 4 undergraduate and 26 graduate students with major concentration in the following areas: Software Engineering, Mechanical Engineering, Computer Science, Reliability Engineering, Medicine, etc. The subjects had varying experience with different web browsers such as Netscape, Internet Explorer, Lynx, Mosaic and two subject with no experience at all. Netscape was the most preferred browser(18 out of 30 subjects) whereas only 8 subjects preferred working with Internet Explorer. The age range of the subjects was from 18 to 55. There were 21 male and 9 female subjects. The subjects were divided randomly into three groups of 10 each for the three treatments.

The materials used for the experiment included the following:

The instructions were provided on a single page to the subjects. It included a brief introduction to the experiment and the details of what the subjects were supposed to do during the experiment.

( Instructions )

The Computer system used was a Pentium-based PC running Linux with 17" monitors running in a resolution of 1024 x 768. The experiment was run on only one terminal as the other terminals running on Linux had subtle differences that would introduce other variables.

Another PC was used for the Memory Recall program, which was running Windows NT.

• Instrument Program :

The "Instrument Program" is written in Tcl/Tk. It was used to measure and record the time that the subjects took to complete each task of the experiment. It presented the pre-task and task questions to the subjects in the form of dialog boxes on the screen. The task questions were presented with multiple choice answers for the subject to choose from. These choices were recorded by the program. The program also presented the Subjective Satisfaction Questionnaire (QUIS) after the subjects completed all the tasks.

• Memory Recall Program:

The " Memory Recall Program" was developed in Visual Basic 5.0. It displays a screen containing screen-grabs of the pages ( similar to those displayed in PadPrints) visited by the subjects, and they were required to select them in the order in which they had visited the pages while doing the pre-tasks. The program provides tool-tips referring to the web pages when the user places the cursor over any of the thumbnail images. A maximum of five minutes was allotted for this test. The program recorded the order in which the subjects marked the pages.

The experimental procedure was as follows:

1. Subject reads the disclosure and signs the consent form.
2. Subject fills out background survey form. ( Background Survey )
3. Subject reads the instructions and get doubts cleared, if any.
4. Subject then sits at the computer terminal to run the experiment.
5. Subject uses hints in the pre-tasks to build a history and familiarize themselves with the history feature.

Pre-tasks include 23 questions ( Pretask Questions ), each requiring the subject to visit or revisit a page on the web. This resulted in the addition of 20 pages on the history list of the browsers. The subjects were not timed at this phase and were allowed to seek the proctor's help. ( Instrument Program )

6. Subject answers the tasks questions.

This includes questions that the subject had to answer by revisiting some of the pages from the history list by using the history mechanism of the browser for that treatment. The subject was timed by the testing software to get the time that the subject needed to answer each question on the tasks. In this part of the experiment, the subjects were not assisted by the proctors. There was a total of 20 questions. ( Task Questions )

7. Subject fills in the Subjective Satisfaction Questionnaire which appears on the screen after the tasks are completed. ( Questionnaire )
8. Subject takes the memory recall test.

Top | Credits | Introduction | Literature Review | Experiment | Results | Discussion | Conclusions | Acknowledgements | References | Appendices

Last modified: May 11, 1998