Text vs. Graphical Links in an E-commerce Website

Experiment

Introduction and Hypothesis

This experiment explored effective ways to display links on a web page. We wished to determine what method of displaying links within a web page is the most effective. We examined the use of textual, graphical and text plus graphical links.In order to explore this topic we set up a small experiment that simulated web browsing.
The experiment simulated an online store, in which users traversed numerous links in order to reach a goal. The goal was to find the price of a specific product. Our hypothesis was that users will be able to reach the goal, in the quickest time with the lowest error rate, on the simulated online store that uses a text and graphics combination for its links

Indendant Variables

The independent variable for this experiment was the type of link used in the online store. There were three treatments for the independent variable. These treatments were the type of links: purely textual, purely graphical (iconic), or a combination of both text and graphics.

Dependant Variables

There are three dependant variables that we measured during the administration of the experiment.

The first dependant variable was the time taken to complete the task, which was measured by time-stamping each page when visited and calculating the time in seconds it took to navigate to the desired product page.
The second dependant variable was the error rate based on the number of links needed to complete the task. The error rate was calculated by taking the number of links traversed minus the minimum number of links needed to complete the task.
The third dependant variable was a survey of each subject’s ease of navigation and satisfaction with each treatment. The subjective satisfaction was measured through an online survey given to the subjects upon the completion of the experiment. The survey allowed subjects to rank each of the three treatments on a 0-9 scale of ease of navigation.

Pilot Study Results

The pilot study was performed on two subjects. It took between five and ten minutes for the subjects to complete the pilot test. During the pilot study we encountered numerous issues which needed to be solved before conducting the actual experiment.

Typographical errors in the questionnaire and survey were found. We also found that one of tasks gave a product number that did not match with the given number on the corresponding product page. These errors were easily corrected.

Another issue we encountered was that subjects did not always give the answers in the correct decimal format. For example, upon finding a price of "$120.00" for a product the subject would type in the answer "$120" and neglect the decimal portion of the price. To remedy this problem, we discarded the decimal portion of the prices in all the products within the simulated online store.

We also found that when users used the ENTER key to submit their answers with the Microsoft Explorer web browser an error occured. We eliminated this problem by allowing subjects to use only the Netscape web browser.

After conducting the pilot study, we realized that the time to load images in the web browser was also included in the measured navigation time. We wanted the experiment to concentrate on the issue of navigation, thus we decided that the administrator of the experiment would cache the images from the simulated online store before the subjects performed the tasks.

To further concentrate on the navigation time, we decided to include a button for users to click between tasks, and allow users to start searching when they are ready. Thus, timing the tasks began only after the subject was ready to begin, and focusing only on the the subjects navigation time.

Subjects

The experiment was conducted with 24 subjects, 17 male and 7 female. Each subject was given one of the six different orderings of the treatments. For each treatment the subject completed two tasks of finding a product's price, thus totaling six tasks for each subject. These subjects must have had prior experience browsing the web and at least limited computer skills.

Materials

The main materials used in this experiment were computers from WAM labs at the University of Maryland. The Netscape web browser was used to administer the experiment. All of our materials for the experiment are located online in the OTAL accounts received in class. These materials include a pre-experiment questionnaire, the simulated e-commerce site, and the post-experiment survey. The online questionnaire and survey can be found in the appendices.

Procedures and Problems

The experiments were conducted in various locations, including computer labs at the University of Maryland. When the subjects arrived at the designated experiment area, they were given instructions on how to complete their task. After completing the questionnaire and receiving their instruction, the subjects were given their task.

At the beginning of a task the subject is given the brand and model number of a product, and asked to find its price. After reading the question, the subject clicks on an button to begin the task. The subject is then shown two frames. The left frame contains the question, while the right frame contains the main store page. Each task involves starting at the main store page and navigating to a specificied product page. Upon finding the price of the product, the subject enters the price into the form in the left frame, and clicks a button to submit the answer. If the answer is incorrect, the subject is told to try again. Otherwise, the subject is then given another task.

The subjects performed two tasks in each of the three treatments. With each task, the subject was given one of the three treatments of the online store to navigate. The rotation of the different treatments was done, to offset for the possibility of subjects performing better with the last treatment type. Upon completing the six tasks, subjects filled out an untimed online survey.