One from Column A and one from Column B: Simultaneous vs. Sequential Menus

Results

Quantitative results

The graph 3.1 and chart 3.1 summarize the results: error bars on the graph indicate a range of one standard deviation from the mean.

As a reminder:

• T1 is a direct hit question,
• T2 needs one backtrack,
• T3 need two.

 

 

Question type Menu type Min Med Max Average Std dev.

T1 Sequential 5 14 36 14.62 5.94 8.4 (1.34)

Simultaneous 12 21 43 21.38 6.17 7.8 (1.48)

T2 Sequential 9 25 41 25.11 8.09 15.0 (1.41)

Simultaneous 12 29 72 29.18 8.78 10.2 (1.64)

T3 Sequential 12 37 69 39.44 12.65 22.2 (1.30)

Simultaneous 15 32 62 33.55 9.58 13.2 (1.79)

  

Figure 3.1: Results

Type 1 question

The results of the t-Test are summarized in table 3.2.

 

 

Sequential Simultaneous

Mean 14.62 21.38

Variance 35.33 38.09

Observations 45 55 Hypothesized mean difference

df 95

t Stat -5.56

$$P(T \leq t)$$ 0.00

t Critical one-tail 1.66

$$P(T \leq t)$$ 0.00

t Critical two-tail 1.99

Since $\overline{X}_{sim}$ > $\overline{X}_{seq}$, we have to reject H_A and accept H₀.

Type 2 question

The results of the t-Test are summarized in table 3.3.

 

 

Sequential Simultaneous

Mean 25.11 29.18

Variance 65.51 77.04

Observations 45 55 Hypothesized mean difference

df 97

t Stat -2.41

$$T \leq t$$ 0.01

t Critical one-tail 1.66

$$T \leq t$$ 0.02

t Critical two-tail 1.99

Once again $\overline{X}_{sim}$ > $\overline{X}_{seq}$, so we reject H_A and accept H₀.

Type 3 question

The results of the t-Test are summarized in table 3.4.

 

 

Sequential Simultaneous

Mean 39.44 33.55

Variance 160.12 91.81

Observations 45 55 Hypothesized mean difference

df 81

t Stat 2.58

$$T \leq t$$ 0.01

t Critical one-tail 1.66

$$T \leq t$$ 0.01

t Critical two-tail 1.99

A clear and significant difference appears here. $\overline{X}_{sim}$ < $\overline{X}_{seq}$ and the one-tailed $P(T \leq t) = 0.01 < 0.05$. We reject H₀ and accept H_A.

We note that the variance increased considerably for this question.

Correct answers

The answers to the questions were mostly correct. We counted right all the answers where people gave a meaningful answer while not responding specifically to the question: answer construction while the question was does construction have more of this than financial (should have answer yes).

Even if the answer was wrong, the time was still recorded: we assume that the subjects took the time and did the right page navigation even if the final answer was incorrect.

A summary of the answers are in table 3.5.

 

 

Type 11 13 14 15 Avg

Sequential 1 1 1 6 14.22

Simultaneous 1 10 14.91

Total 1 1 2 16 14.60

Subjective results

The results of the questionnaire are compiled in table 3.6 and displayed in fig 3.2.

 

 

Questions Nb Scale (1-9) Sequential Simultaneous

System 1.a frust.-satisf 5.89 6.36

1.b diffic.-easy 6.78 7.27

Helpful layout 2 never-always 5.78 6.45

Information arrang. 3.a illog.-logical 6.89 6.90

3.b confusing-not 8.56 7.27

Amount information 4 inad.-adequate 5.56 7.00

Navigation 5 diffic.-easy 6.44 7.91

Initial instr. 6 inad.-adequate 8.22 7.55

  

Figure: Subjective

Questions 3.a, 3.b and 6 suggest that the simultaneous layout is more confusing than the sequential one and that it did reflect on the screen and in our instructions.

For the rest, the users preferred the simultaneous menu.