Previous Article Next Article Net effectsOn 1 Jun 2000 in Personnel Today Whenit comes to multimedia courseware, trainers are spoilt for choice. But how canthey be sure that the programme they select will really deliver? Robert McLuhanlooks at two contrasting approaches to courseware evaluationMoreand more organisations are opting for e-learning solutions, attracted by theirability to skill up large numbers of employees at a fraction of the cost ofclassroom training. That has brought a proliferation of providers: every daymore companies enter a market that IDC estimates will be worth $11.4bn (£7.6bn)by 2003. But the quality of the material on offer is by no means even. NETg,one of the largest suppliers, characterises much of what is currently availableas entertainment, packed with games-oriented video and animation sequences thatdo little to transfer skills.Theother type to avoid is the course that simply presents subject matter as apage-turning exercise. Only programmes that genuinely engage the user can havereal value, it argues.Tohelp trainers sort the wheat from the chaff the company has come up with aWindows-based evaluation tool. The software is free and is being distributedthrough the global Information Technology Training Association (ITTA) as acontribution to creating industry-wide standards. Eventuallythe company hopes that the UK’s Institute of IT Training and other trainingorganisations around the world will follow suit.Thename of the programme is ECG, suggested by the similarity of the graphs it producesto medical electrocardiograms. The tool plots levels of engagement andinteractivity in a course by quantifying the use of simulations andopportunities for feedback. EffectivenessConversely,reliance on quizzes demanding undemanding “yes-no” or multiple choice responsesindicates a low level of engagement. The resulting graphs provide an instantvisual comparison that enables training professionals to compare the likelyeffectiveness of various courses under consideration. Tosave time the tool can be run through selected pages, on the assumption thatthey will probably be representative of the course as a whole.“Peoplelearn better when they actually have the opportunity to be engaged in anactivity rather than simply being fed a piece of information,” points out PamBurton, NETg’s director of global marketing. “We believe that is a validmeasure of how much interactivity there is.”Brewersays, “Our objective is to help educate people who are evaluatingtechnology-based training or who are trying to come up with an e-learningstrategy. We are very committed to driving standards within the industry.”Brewerconcedes that software alone will not achieve the same result as a manualevaluation. It is intended as a starting point for buyers coming to the marketrelatively unprepared, who may be vulnerable to products that are superficiallyattractive but have little value. She says, “What we are providing is one moretool in the buyer’s toolbox and certainly there will be other factors toconsider, for instance the quality of the after-sales service.”Justhow much more there is for trainers to get to grips with before making apurchase is made clear by Xebec McGraw-Hill, which recently publishedcomprehensive guidelines on the issues involved in choosing technology-basedcourseware.“E-learningcompanies are popping up every day of the week and there is very little thathelps buyers choose between them,” says senior flexible learning consultant TimDrewitt. “Organisations either do it themselves and hope for the best or elsethey rely on advice from resellers.”CriteriaWhileDrewitt welcomes any service that helps make an informed choice, he argues itshould be based on a number of criteria. Xebec McGraw-Hill’s book, QualityStandards for Evaluating Multimedia and Online Training, outlines four mainstages in checking the usefulness of a package.“Firstyou need to be sure the product matches the organisation’s needs,” Drewittexplains. “That means asking whether the target audience described by the vendoractually fits the profile of your users, and whether the course’s statedobjectives are the same as yours.“Also,will the product work on your technology platform and do in-built features suchas progress tracking actually do what you want them to?”Thesecond step is to review the content for accuracy, depth and clarity. It willbe necessary to judge whether the learning skills are appropriate to the levelsof skills of learners and to roles in the organisation.Thenthere is the question of usability. Trainers will need to know whether thecourse is easy to install, runs smoothly, and provides clear and consistentinstructions. “You want it to be intuitive, so that it is easy to spot whatwill happen when you click on something,” Drewitt says.Onlyin the final stage do the guidelines tackle the area addressed by NETg’s ECGtool, instructional design. Questions to ask at this stage would include:–Are the objectives presented at the beginning? –Is the course structured in an appropriate sequence? –Are there sufficient examples to reinforce learning points?“Youwant to know whether the students are actively involved in the learning methodsused,” explains Drewitt. “Are they given choices of learning methods and doesthe course provide realistic opportunities for practice in differentscenarios?”Non-threateningTrainerswould want to ensure feedback and guidance is offered at every stage in atimely, relevant and non-threatening manner.Guidelinessuch as these have the advantage of taking much more into account than asoftware tool can realistically accommodate. The downside is that they demand awillingness by the buyer to make conscious judgements, instead of carrying outthe process automatically.Ultimately,Drewitt says, the value of any aid will be to short-list likely contenders.Other considerations such as cost make it difficult to base a decision onquality alone.“Atthe end of the day, the best test of the quality of the product is the user,”he suggests. “If staff can’t get round the course they will soon tell you. Butif they are enjoying it, that’s a benchmark for you: when you buy a product inthe future you know your business likes this particular brand.”Ascourse evaluation becomes a growing concern, aids such as these are likely tobe increasingly available. But no single approach seems to cover every aspect apurchaser needs to know, which means buyers will rely on their understanding ofwhat makes for successful training. Comments are closed. Related posts:No related photos.