The Effects of Training Methods on Users' Performance: A comparison between Person-to-Person Training and Group/Classroom Training
by Bjorne Beltran, Partha Ghosh, Srinivas Kadiyala
 
 
 
 

I. Introduction and Background

Training is an integral part of every company's agenda. Approximately 50 billion dollars are spent annually, 5 billion of which is allocated to Information Technology (IT) training (Nelson, et. al., 1995). It is estimated that there will be an increase of 13.6% per year allocated for IT training according to the International Data Corp. This is due to the short shelf rate of software products and the high demand for qualified users. Software vendors keep on improving software and companies are always looking for the latest and greatest in information technologies (Ginsburg, 1997).

Because of the implications of training, it is important to have training that is effective. Studies have proven that more costly but effective training can save money that is wasted on cheap but inefficient training (Ginsburg). Unfortunately, there is no rule of thumb or "one for all" method of effective training. Methods of training have to be analyzed and studied before companies rely on them to train a competent workforce.

It is very important to have a needs analysis to determine which method works best. There are numerous factors to be considered in making training method decisions. Factors such as training objectives (what is aimed to be learned), cost, and trainee demographics are some important issues to be considered.
 

 
A. Training Objectives 

It is important to first differentiate education from training. Education is mainly theory based. It teaches a person to think. It doesn't teach a person what to do in particular situations, but it gives the person tools to figure out what to do in any situation. Training on the other hand is more specific. It attempts to give a person skills needed to perform in a particular situation or complete certain tasks. It does not cover every situation.

This distinction makes it very important that training address the particular tasks that an employee must do. It must be closely tied into work situations and tasks. Training ideally should be "custom-made" to fit into the employee's needs; not only his personal learning needs, but the particular tasks that he needs to be able to do.
 

B. Cost 

A company's budget dictates the kinds of training that may be considered. Unfortunately, the training budget is most often cut off first whenever the budget becomes tight. Training methods vary in cost. Usually methods that require human relations are more expensive than those that don't.

There are two general kinds of training methods described in this paper. First is the word of mouth (WOM) training which teaches through direct human interaction (Galletta et. al., 1995) and the second is Technology aided training which is self-learning aided with training tools.
 
1. Kinds of Training 
 
a. WOM 

This is presently the most common method of training. This is the kind of training where speech and human interaction is key. Whether the instructor is an outside contractor, inside IT professional, or a trained peer co-worker, what distinguishes this from computer based training is human interaction and feedback. This kind of training costs within the range of $175-$350 per trainee per day. The kinds of WOM training are: 
 
i. Person-to-Person 

This training involves an individual trainee paired with an individual instructor. The trainee's attention and concentration is mostly guaranteed to be in the task and in the training. The trainer can give personal attention to the trainee as well. This method can be expensive because of the individuality of it.

 ii. Group/Classroom 

This type of training is cost-efficient since it takes advantage of one trainer for multiple trainees. The benefit of structured learning is another positive aspect of this method. A consequence of this method is that since everyone is different, some trainees may get lost in the process and pacing is different. 
 
b. Technology Aided Training 

This kind of training is cost efficient. This method usually involves self-training, self-study, and self-guidance. Examples of this method are the tutorials, manuals, web tutorials, video tapes, and computer based training (CBT). CBT is growing rapidly in use because of its flexibility of integrating text, graphics, sound, animation and video (Scott & Violette, 1996). The cost of technology aided training can range anywhere from $600 to $1000, but can be used repeatedly by numerous employees. Most of the time, trainees can pace themselves according to their capabilities using these methods, but there is a lack of feedback and guidance. Trainees, if not knowledgeable and if lacking the proper educational background, may confuse themselves even more. 
 
2. Instructors 

The kinds of instructors are also considered in the budget. There are contracted outside instructors that companies may hire. The advantage to this is that they are focused on training as their task, have experience in training , and therefore can be considered experts. They may be expensive and there may be a chance that they do not know the company culture. They may not accurately customize their training to the needs of the company.

The IT division of a company may be trainers as well. This is a more cost-efficient way, and they may know the company culture. There are times when IT specialists do not want to be bothered by training duties because they'd rather be doing technical duties, or they may not be familiar with the training needs of the trainees. 

Certain chosen employees can serve as instructors. A study by Fitzgerald and Cater-Steel (1995) considers these trainers. Having these kind of instructors present lower costs, easier adaptation to the demands of the workplace, and rapport with the trainees. Since they are "one of them", trainees do not feel as intimidated. These trainers gain more responsibilities added to their present workload. Training others may be burdensome and may take them away from their initial duties.

 

C. Trainee Demographics 

People needing training can be classified in different ways. There is a distinction between novice users and expert users. They can also be classified through their educational backgrounds, or through their current employment position. Whatever way trainees are classified, they all have different needs. It is important, when choosing a training method, to identify who is to be trained. 
Novice users may be computer-shy or technology-intimidated and may need personal attention. Experts may need little attention and may be bored with basic information, and therefore dampening the desire to learn. According to Agarwal et. al., educational background information is important. They also say that for people with little education, structure in learning is important. 

Employee position is important as well. Senior management may not have the time to attend group training or may have frequent distractions. The issue of self-esteem may also be a factor. Someone high in the ranks may not want to appear stupid to his counterparts by asking a question in group training. 

Analysis from the beginning is definitely needed for training to be effective. The effectiveness of training is usually measured through user performance (Goldstein, 1987). Goldstein introduces the idea of training validity which assess the performance of trainees in relation to the criteria set by the training. "Training must always be evaluated with respect to both its immediate and long term impacts" (Carroll and Rosson, 1995). It starts from the training experience to the training outcome. The training experience includes the actual training and the immediate effects of the training based on performance. Training outcome are the long term effects of the training (Carroll and Rosson). Both of these, training experience, and training outcome, are tested in this study by testing right after training and testing retention after seven days. 
 
 

II. The Study

Due to the vast areas to consider regarding training, no one study can evaluate or analyze it as a whole. This study deals with the comparison of the two kinds of WOM training, Person-to-Person training and Group/Classroom training because they are presently the most common methods and we feel that human interaction and feedback is very important in learning.

The type of trainer used was an in-house IT/helpdesk professional because this kind of trainer is familiar with the trainees and the company's methodology and culture at the same time being an expert on the trained material. The trainees were 40 employees of the Office of the Comptroller, all of which are between the ages of 18 and 50 years old. They all have had some college education. Their employment positions vary from secretaries to other IT professionals. All were considered novice users.

Training validity will be assessed by measuring speed and accuracy. Training outcome is then done by measuring speed and accuracy after seven days. We believe that person-to-person training will result in better performances in both training validity and training outcome because of the individual attention and guidance given to the trainee and the trainees concentration and attention to the tasks being learned.

This hypothesis is based on a study by Agarwal et. al., which claimed that deep user involvement in learning provides users with excitement and motivation, allowing them to engage in a cognitive activity such as learning software operations. Trainees are also allowed to move at their own pace and the trainers can modify the training to the trainee's needs as they surface.

The Agarwal et. al. study also mentioned that users who have little or no educational background benefit more from a structured and organized way of learning. Group/Classroom training and Person-to-Person training both have structure, but Person-to-Person training has more diversity and flexibility than Group/Classroom training.
 

 

III. Materials and Methodology

We took advantage of the Office of the Comptroller's transition from Windows 3.1 platform to the Windows '95 platform. Forty employees from this office were asked to participate in the study.

The dependent variables of speed (measured by task time completion), accuracy (measured by the number of mouse clicks), and retention were measured. Retention was measured by taking speed and accuracy measurements after seven days from initial training. The T-test was used to discriminate statistically significant data. User satisfaction was also taken by giving the subjects a survey. No statistical analysis was done with the satisfaction survey.

Some elements of the experiment were controlled. All users were considered novice users since they have little or no experience in the Windows '95 platform before the office transition. The tasks that users were trained and tested on were also the tasks that are not usually done in day-to-day work. Due to experimental purposes, tasks were not tasks that trainees usually perform were given. Tasks varied from easy to difficult.

All seven tasks were the same for all subjects. The subjects were controlled to have almost  similar backgrounds. Machines were limited to be Gateway2000 G6-233. Time was limited to 45 minutes per training session. All users were given user manuals at the beginning of training and were allowed to use the manuals during testing. All users had the opportunity to walk through the tasks with the PC during training. The instructor was known and familiar to all subjects.

The forty subjects were divided into two groups. Twenty underwent person-to-person training while the other twenty underwent group/classroom training.

Person-to-Person training was 45 minutes long. The training manual was given at the beginning of the training session. The trainer guided and walked through the tasks as laid out in the manual with the user, while the user walked through the manual using the computer. This training was done at the trainee's desk and environment. The trainee was allowed to ask questions at any time. The initial testing was done right after training. The subjective satisfaction survey was given to be filled out after testing.

Due to space limitations, group/classroom training was done with 10 subjects at a time at the Administrative Computing Center. The training lasted for 45 minutes. The trainer stood and demonstrated in front with a projector which showed his computer screen. The manuals were passed out in the beginning of the session. The trainer walked through the tasks in the manual using his PC while the trainees followed using their own PC's. The trainees were allowed to ask questions at any time. Initial testing was done right after training in groups of two. Subjective satisfaction surveys were given out to be filled after testing.

Testing comprised of having users go through tasks by themselves while being timed. The mouse clicks done were also counted. The subjects were allowed to refer to their manuals at anytime.

Retention testing was done after seven days of initial testing. Subjects were asked to come back and were asked to perform the same seven tasks from part one of the test. Again, completion time and the number of mouse clicks were observed and measured.

The subjective satisfaction survey was not given again.
 

 

IV. Results

Subjects that received person-to-person training did better than the subjects who received group/classroom training in both the accuracy and speed tests, whether it was during initial testing or retention testing. The subjects that received person-to-person training had a group average of 25.9 minutes to complete all seven tasks while the group which received group/classroom training had an average of 37.65 minutes to complete all tasks. Both had a standard deviation of sd = 4.5. Refer to http://www.otal.umd.edu/SHORE98/bs17/raw_data.html for the raw data.

The total amount of mouse clicks were less for those who received person-to-person training. The average number of clicks for subjects with person to person training was 76 clicks while subjects who received group training averaged 96 clicks to complete tasks. Standard deviations are as follows: person-to-person had sd = 4.1, and group/classroom had sd = 8.2 clicks. The lesser number of clicks indicates less errors. The raw data could be viewed at http://www.otal.umd.edu/SHORE98/bs17/raw_data.html.

The results showed a statistically significant difference between the effects of person-to-person training and group/classroom training in both speed and accuracy factors. The mean and standard deviation of the treatment results are shown in the table below.
  Speed 

(Time in Minutes)

 Accuracy 

(Number of Mouse Clicks)
Person-to-Person 25.9 

(4.39)

 76 

(4.05)
Group/Classroom 37.65 

(4.99)

 96 

(8.19)
The raw data for the retention tests can be found at http://www.otal.umd.edu/SHORE98/bs17/raw_retention.html. The mean accuracy, in the number of mouse clicks, for the person-to-person training subjects had a mean of 29.3 minutes with a standard deviation of sd = 4.12. Group/classroom training subjects had a mean 29.3 and a standard deviation of sd = 6.28. These results are not as good as the performance shown during initial testing, but they still hold the property that person-to-person training subjects did better than group/classroom training subjects.

A summary of both initial speed and retention speed is shown in the graph below:
 

 
 

A graph for the summary of the accuracy tests is shown below:
 
 

 
 

A T-test analysis of data showed statistically significant differences between the two treatments of training methods for both speed and accuracy factors. A significance level of ( = 0.10 was used with 19 degrees of freedom. The critical value based on the T-table was a value of T = ± 2.086. Any value in between +2.086 and -2.086 would not have been considered statistically significant. The result of the calculations made on the data yielded, t = 7.003 for speed and t = 8.9349 for accuracy. For retention, speed had t = 6.228 and accuracy had t = 9.4097. The formula used to get this statistic was taken from (Partha's book). The blocking principle was used for this statistic.
 

 


V. Discussion

As predicted, subjects who were given person-to-person training had faster speed and greater accuracy compared to group/classroom training. Statistically significant data shows that there is significant difference between treatments.

According to a study done by Davis and Bostrom (1993), "the benefits of training are maximized when users have an opportunity to learn in an unstructured fashion". According to Agarwal et. al. however, striker is needed in guiding users so they won't get lost in the learning process which may frustrate them.

Group/classroom training offers a more traditional and formal training. There is structure and organization. There is an instructor who is knowledgeable to guide users and answer whatever questions they have. In this method however, users may become passive, even if they have the hands-on opportunity. Attention is not necessarily on the training as the user's mind may wander. Users may also not be in the same frame of mind at the same time. One may be ahead or behind another.

Person-to-person training combines the positive aspects of formal training and unstructured technology aided training and eliminates the negative learning aspects. Person-to-peroson training requires the user to have his attention on the training being given. Since it is a one-on-one form of training, the instructor's attention is also focused on the user. This makes the user concentrate on what is being learned and enables the user to discover for himself. The instructor provides some guidance and structure to the training. The user can ask the instructor questions at anytime, without the fear of appearing stupid to others.

During training, it was interesting to note that subjects who underwent person-to-person training asked more questions. There are two possibilities for this phenomenon. Trainees may be more comfortable asking if no one else is watching or observing them, providing they are comfortable with the trainer as well. More questions may also show that the trainee has more of his attention and focus in learning, and that he is interested as to what is happening.

The subjective satisfaction survey failed to bring conclusive answers. The survey questions failed to uncover any valuable information for the study. Subject preference was seen through the process of sign-ups. More subjects signed-up to receive person-to-person training than classroom/group training. We had to request other users to sign up for classroom/group training.

Person-to-person may have surpassed all the comparison testing over group/classroom training but this study does not take into consideration the fact that it is more costly in both time and money. However, additional studies have shown that in the long run, good training is better than bad training, especially in complicated and important tasks. In this study, it is demonstrated in the retention testing. Even after seven days from training, person-to-person trainees still did significantly better than their group/classroom counter parts. Person-to-person training may be more costly, but it is worth the extra money.

Since it has been claimed that CBT is a rapidly expanding training method, out of curiosity (apart from the main study) a Computer Based Training sample was developed and tested. Five subjects were gathered from the Office of the Comptroller. The mean and sd values gathered were. The raw data from the CBT study can be found at http://www.otal.umd.edu/SHORE98/bs17/raw_cbt.html.
 

VI. Conclusions and Recommendations

Statistically significant data showed that person-to-person training is more effective as shown in user performance. Person-to-person training is also a preferred choice amongst users. We suggest that this is because it combines the attention, flexibility and focus of self-training found in technology aided training; and the structure, organization, and guidance of formal training, found in classroom environments.

Person-to-person training involves a high initial cost, but this cost may be offset by the higher quality training employees receive. Other forms of training involve a lower initial cost, but result in ineffective training and unproductive employees.

If a company has a tight training budget, it may limit it's person-to-person training only to novice users who are intimidated by new technologies, and to employees who deal with extremely important tasks. It can also be limited to senior management who do not have the time to attend group training and who need personal attention.

This study was limited to two methods of training. It did not consider motivational factors of learning. It did not investigate how needs assessment are undertaken. This study did not consider the difficulty of the tasks. The tasks for all the subjects were the same and drawn from several levels of difficulty  but there was no measurement of how trainees did on a particular difficulty level.

We recommend that further studies be done comparing different training methods to find out their effectiveness in user performance. Studies comparing the different methods under the technology aided method should be studied and assessed, especially CBT.

A more thorough study may be done on the preferences of users and why they prefer a particular method. Further studies on training and its effects should be done to aid companies in their choices of methods.

VII. Appendix

VIII. References