 |
|
 |
 |
 |
|
|
Results When the subjects were performing tasks, each task completion time was recorded in seconds. The total time the subjects spent on PhotoFinder and answering thirteen subjective questions were also recorded. The recorded data is presented in the Raw Data Table in the Appendices section. The five tasks’ completion times were summed, recorded and named Total Tasks Time or TTT. The relevant data and descriptive statistics are presented in tabular and graphical forms below. Tasks Completion Times The total time is the time that the subjects spent on PhotoFinder during the experiment. TTT is the sum of times that the subjects spent on five tasks.
The chart below (Figure 1) shows the comparison between the mean of each task completion time of the three groups (Text only, Animated only, and Integrated interactive online help).
Figure 1 The chart below (Figure 2) shows that mean TTT (Total Task Time) for the online help systems are: Text only 339.8 seconds; Animated only 243.9 seconds; and Integrated 237.6 seconds.
Figure 2 ANOVA, statistically analyzed data in tabular form below, shows that the difference among the mean TTTs is not statistically significant since F < F crit (F = 1.88 < F crit = 3.35), with significance level α = 0.05. Hence the null hypothesis cannot be rejected.
To confirm ANOVA’s results, the t-test was also performed on three pairs of treatments; Text only and Animated only, Animated only and Integrated, and Text and Integrated online helps. Null hypothesis (Ho) is rejected when p < alpha, but Text only and Animated only online help: p one-tail = 0.07 > alpha = 0.05 Animated only and Integrated interactive online help: p one-tail = 0.32 > alpha = 0.05 Text only and Integrated interactive online help: p one-tail = 0.13 > alpha = 0.05 Therefore, the null hypothesis cannot be rejected. (See Sheet1 worksheet in Statistical data in Appendices.) Since Figure 1 shows that there may be some statistically significant difference among each task performance time of the three treatments, ANOVAs (single factor analysis of variances) were used to analyze the performance times. The results are:
The result of ANOVA (table below) on Task 5's performance time shows statistically significant differences. It implies that the null hypothesis can be rejected. Therefore, our hypothesis is accepted, for Task 5 (Create a slide show of the photo's in the Buildings collection.) Integrated Interactive online help has the best performance time. On the other hand, ANOVAs on Task 1, 2, 3 and 4 show no statistically significant differences. (See Sheet1 worksheet in Statistical data.)
Subjective Questionnaires Questions 1 to 4 are background questions. Only questions 5 to13 used the scoring system of from 1 (least preferred) to 9 (most preferred). Means and standard deviation are shown in tabular forms below.
The chart below (Figure 3) shows the comparison between the mean of each score given by the subjects of the three treatment groups.
Figure 3 The chart below (Figure 4) shows mean total subjective scores with a 95% confidence interval (+ and – from the mean) for the online help systems are: Text only 58.4; Animated only 63.9; and Integrated 60.
Figure 4 ANOVA, statistically analyzed data in tabular form below, shows that the difference among the means is not statistically significant since F < F crit (F = 0.88 < F crit = 3.35), with significant level α = 0.05. Hence the null hypothesis cannot be rejected.
ANOVA shows no statistical significance in any of the subjective questionnaire scores given by the three groups of subjects. The 30 subjects included 16 males and 14 females. They were distributed evenly among three treatments. About 57% of the subjects (17 out of 30) considered themselves intermediate in computer experience level, seven out of 30 considered themselves as advanced computer experience level, four subjects were in the beginner level, and three subjects did not answer the question. Nineteen subjects had English as their first language, nine did not, and two did not answer the question. There were also short-answer questions, e.g., “What were the problems during the use of PhotoFinder?” and a request for “Other comments and or suggestions.” These are included under Subjective Questionnaires in the Appendices. Microsoft Excel 2000 was used to analyze the raw data. The whole workbook is presented as Statistical data in Appendices.
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
