Chapter 5

"Behind the Scenes" Innovations

This chapter deals with innovations, some of them quite technical, that are not observable to the learner but that make a difference to the quality and usability of learning technology and instruction. Behind the courseware that learners interact with are the course authors, and they can benefit from tools to aid them in the complex task of producing an online course. Instructors need information on what the learners are doing; learners need to be able to find needed information. This calls for tools working in the background to perform intelligent sifting and relating of information. The continual improvement of educational software requires behind-the-scenes technology to measure how well a system is functioning for the user and technology for dealing with problems such as slow response times, confusing displays, and similar annoyances of online life. Finally, if e-learning is to grow and prosper, it requires valid ways of monitoring costs and benefits. Tools for that purpose are just beginning to emerge. The innovations discussed in this chapter range widely but they contribute to the basis on which more powerful learning technologies can be built.

Course Development Tools

"Putting a course online" often means simply converting course outlines and hand-outs into Web pages, but it can mean much more than that. It can mean a marriage of new technology and new ideas about teaching. Several Canadian innovations are aimed at helping this marriage succeed.

Carl Bereiter

Creating Online Course Content with eTrainerCB

Developed by Lucio Teles, eTrainerCB is designed to assist educators with the migration of existing course material to an online setting, as well as facilitating the creation of completely new online course content. The system uses a series of templates to assist instructors with the design and deployment of online courses. The instructional design process is embodied in a straight-forward 10-step process of course content creation. The idea is to provide a tool that requires little technical sophistication on the part of course instructors, thereby allowing them to focus on the creation of an effective online course. eTrainerCB relieves them of the need to be proficient in HTML coding, meta-tagging (the standardized method of describing course content) or web publishing.

Once the user has complete the 10-step course creation sequence, eTrainerCB can publish the results in a variety of formats. For example, the course content can be delivered via email, published to a web page, or made available via CourseReader (described in the preceding chapter).

Development of eTrainerCB is continuing. Recent advances include full compliance with international standards for content management, as well as work to faciliate interoperability with a variety of Learning Object Repositories including POOL and eduSplash.

CADRE: Supporting the Creation and Delivery of Interactive Case Studies

The CADRE (Collaborative Authoring and Design Resources) project was led by Tom Carey from the University of Waterloo. The CADRE project consisted of five teams distributed across Quebec, Ontario and Saskatchewan. The project focused on the development of software that would support the creation and delivery of interactive case studies. These case studies were targeted to designers and developers of telelearning materials.

As defined by the researchers (see http://www.curricstudies.educ.ubc.ca/wprojects/TL-NCE/TLN-CE.Overview.pdf ):

Specific objectives included the development of: instructional models to guide the design of learning activities for computer supported collaborative learning; tools for dynamically structuring the selection/customization of collaborative learning materials during usage; learner modeling techniques for designers to use in the creation of materials; software architectures to support cost-effective creation of telelearning software as a knowledge-building process; a workbench for instructional design, supporting integration of methods and media in both autonomous and collaborative learning.

The core of the technology was the development of a course design workbench. This development work entailed the evaluation of existing course design workbench, as well as the development of an implementation model for the use of the workbench.

Support for the course design workbench was provided by an extensive library of case studies, known as CLARET (Case Library Architecture for Engaging Telelearning). The case studies were drawn from subject matter experts, instructional designers, user interface experts, and learners.

The overall design took into account research results from distributed cognition and computer-supported collaborative work (CSCW), and also took into consideration the role of peer tutoring.

Chris Teplovs

Learning Object Repositories

As instruction moved "online" it became quickly apparent that the text pages, diagrams and interactive computer applets developed for a course could often be re-used in another learning context. These so-called "learning objects" became the focus of a number of collaborations to expedite their safe storage and sharing. Search engines such as Google (www.google.com) helped locate textual materials, but non-text images, videos, and those materials produced on demand, or pulled out of data-base storage systems were more difficult to index. Repositories catalogued learning objects in databases so that searching through their educational metadata could easily retrieve them.

The construction of Learning Object Repositories (LORs) became the focus of considerable investigation. Starting about 1999 TechBC (now SFU Surrey), IBM Canada, OLA, NewMIC, TelesTraining Inc, and NB Distance Education Network collaborated with TeleLearning and Canarie to build POOL (Portal of Online Objects for Learning). POOL would work like the prairie wheat pools, consolidating the harvest in local elevators until needed, then shipped via the interconnecting rail network to fill orders in far off destinations.

POOL became focused on how to build and link a variety of different repository types. Communities could develop centralized repositories or "PONDs", while individual learners or instructors could install "SPLASH," a desktop peer-to-peer learning object repository. POOL, POND and SPLASH formed a scalable network of interoperable learning object repositories (www.eduSplash.net). Aspects of POOL examined collaborative reviews of learning objects, methods for automating metatagging, and ontological mapping techniques central to the emerging Semantic Web. POOL expanded the TeleCampus database (www.telecampus.edu), and helped create the CanCore metadata profile (www.cancore.org).

Other TeleLearning efforts on learning object repositories were carried out at LICEF and by the University of Waterloo. Tom Carey became Co-Chair of the higher education learning portal MERLOT (www.merlot.org). Merlot encourages authors to post information about their learning objects, so other instructors can add comments on how they adapted them for their own lesson plans. Merlot developed an academic peer review model in hopes that tenure committees would recognize the quality of effort required to produce good learning objects.

In 2002 many of these efforts were documented in the Canadian Journal of Learning Technology Special Issue on Learning Objects. The confluence of the Canarie projects resulted in eduSourceCanada (www.edusource.ca), a Canarie e-Learning Project to build the pan-Canadian infrastructure for learning object repositories. TeleLearning collaborations continue to inspire innovations as Behind the Scenes technology for education.

Griff Richards
For more information
Hatala, M. and Richards, G. (2002)
Richards, G. (2002)
McGreal, R., Anderson, T., et al. (2002)
Porter, D., Curry., et al. (2002)

Tools for Monitoring, Search, and Research

From the earliest days of computer-assisted instruction in the 1960s, researchers have been taken with the fact that the computer enables them to keep a record of every keystroke by every learner. Surely such an abundant source of data would make possible new discoveries about learning and more powerful ways of engineering the learning process. The unsolved problem that has made this a quixotic dream is how to extract useful information from a virtual infinitude of possibilities. A similar problem faces the ordinary Internet user today, who enters descriptive terms into a search engine and is informed that there are more than a hundred thousand documents fitting the description.

In the early days, the quantity of data overwhelmed computing capacity, but that is no longer the case. The computing power is readily available, and so the problem is figuring out what to do with it. Searching for patterns in raw data is one possibility. Searching for patterns among words is another. Defining measurable variables is still another. All these routes have been pursued by Telelearning NCE researchers, and these have given rise to a number of innovations, some of which have wide implications for learning technology.

Carl Bereiter

FX-Nomino: Natural Language Processing Meets Artificial Intelligence

Nomino is a text processing system that combines natural language processing and artificial intelligence techniques. It has a wide variety of applications: a reading assistant, a dynamic and flexible hypertext navigator, a tool capable of sorting and transforming textual information, producing text summaries, natural language queries, index creation, locating key zones, and constructing networks and knowledge bases. The system is highly extensible through the creation of tools that can be customized for specific needs. For example, one such tool was originally designed to assess the quality of learning in teleconferencing systems, and was later adapted to identify expert knowledge in online discourse.

Nomino seeks to create databases based on natural language elements. Rather than using a matrix representation (as is the case with LSA, discussed below), Nomino uses a tree-like structure for categorization. The categories, however, are not fixed, and can change as additional information is added to the knowledge base. Text that is entered into the system is lexically parsed. This parsing consists of lemmatization (i.e. normalizing different forms of the same words), syntactical categorization (i.e. identifying words as names, verbs, etc.), and the identification of Nominal Complex Units. It is this latter step that is used to identify conceptually unique units.

Once the knowledge base is thus parsed, queries to it undergo similar parsing. These queries are also in natural language, which makes it easy for end users to compose them. Nomino differs from traditional search engines in that queries do not rely on the traditional "all" or "any" word matches. Both of these traditional approaches are problematic: "all" word matches tend to retrieve too few documents, whereas "any" word matches tend to retrieve too many. The scoring system used by Nomino combines "range" and "expressivity". "Range" tends to reward information that is rare, whereas "expressivity" rewards information that is semantically similar. Documents that are well-rewarded on both "range" and "expressivity" will be returned by Nomino.

Most recently, Nomino Technologies (a spin-off company) has partnered with Inktomi (a large company specializing in information retrieval) to offer a suite of applications to support client relationship management. By integrating artificial intelligence, natural language processing and information flow processing, the software suite is designed to improve Web-based customer service.

Chris Teplovs
For more information:
Nomino Technologies

< FX-Nomino  |  Mining Web Logs >

Knowledge Building Indicators for Online Learning

One of the goals of the Knowledge Forum project has always been to turn assessment over to the knowledge-building community as a constructive tool. In the traditional classroom, the teacher assesses the performance of students, and then reports the results back to the students, and hopes that they will take appropriate action to improve. In the educational research setting, researchers assess the effect of an intervention on the performance of a classroom, then report the results back to the research community and hope that the community will use them to improve education. The length of the evaluation cycle means that by the time the teacher reports back, the students have already finished the learning unit and moved on to other things. In research, teachers do not usually get results for months or years after an intervention has occurred. Thus assessment is mainly used to evaluate past work rather than to guide present and future work. What if students could assess their own performance as they were working and teachers had the tools to assess whether an experimental intervention was working out or not? The Analytic Toolkit for Knowledge Forum, introduced in 1997 after several years of prototyping, uses the power of the online environment to make ongoing formative assessment possible for teachers and students using Knowledge Forum.

The Analytic Toolkit is a set of theoretically-based analysis tools, designed to be easily accessible through a Web interface, and to provide information that can be used meaningfully by teachers and students in their efforts to become a knowledge-building community. Instead of counting hits or key presses, the Analytic Toolkit reflects the rich interactive environment of knowledge forum is reflected in a set of measures that bear on whether the community is becoming a knowledge-building community or not: Are all participants contributing to the community effort, or are some being left behind? Are participants showing how their ideas connect to other ideas in the database, or are they only making isolated, unconnected notes? Are participants reading what others have to say? Are they working across a variety of different problems and views in the database? Are they using keywords on their notes so that other members of the community will come across their work when they search by keyword? Are they editing their work (an indicator of reflection)? Sequences of actions by participants (which notes led this student to make a breakthrough?) and the interactions between them (who's reading whose notes, who's working with whom) can also be examined. The reports can be used as is, they can be exported for more in-depth analysis in a spreadsheet or statistical package, or they can be saved back to the database for archived documentation on the progress of the community.

Of course, statistical, quantitative analyses are not sufficient to determine the quality of a database or of an individual's contributions, but they are certainly part of the picture. There are several directions in which quantitative analysis may be taken further. The project has only begun to examine patterns of performance, and to provide comparative statistics to different sites that are struggling to develop as knowledge-building communities. Also, the project is beginning to incorporate recent gains in content analysis and the automated assessment of writing samples, through the use of Latent Semantic Analysis (described later in this chapter). A third direction is to make graphical analyses of performance available at the click of a button to the users of Knowledge Forum.

The power of a computer to automatically monitor a user's activities can be frightening. But if assessment is put in the hands of the participants themselves, rather than being applied by an outside authority, then this power can become a constructive force. It can be used by participants individually and as a community to accomplish their own goals in a more effective way. The Analytic Toolkit is a pioneering step in this direction.

For more information:
Analytic Toolkit for Knowledge Forum, handbook

Mining Web Logs for Learning Patterns

Data mining is a set of computer-automated statistical techniques used to discover patterns in large sets of data. It is often referred to as a process of knowledge discovery. In science, early successes included the automated classification of faint objects from the 2nd Palomar Observatory Sky Survey, detection of earthquakes and weather phenomena from satellite photography, and searches of large bio-chemical databases for molecular and genetic relationships. In business, data mining is used to discover predictive patterns in a welter of economic indicators, to aim marketing according to patterns of online shopping, and to detect the departures from regularity that may signal fraud. In each case, the techniques of data mining are used to extract meaningful patterns for the classification of objects and events. The colourful language of "drilling down", "rolling up", and "slicing and dicing" has become part of the analysis vocabulary, and gives a sense that gold (or perhaps a gourmet meal) is waiting to be found at any moment.

In online learning, the gold is a better understanding of how people learn, which patterns of online activity are the most productive, and how the interactions of students can be understood to advance online pedagogy. Many courses are delivered by Web servers, which keep a log of every request they receive. In the mid 90's, the Database Group at Simon Fraser University, led by Jiawei Han, turned its attention to mining these Web logs, and particularly the logs kept by the Virtual U course-delivery system (see Chapter 3). Existing data mining products from the big vendors were disappointing in that they were slow, and could only discover rather low-level information about Web use, partly because the information contained in a Web log is too limited. The idea of WebPageMiner was to add information about the particular Web site or course or application, in order to boost the raw data to a more meaningful level, and then to apply the fast methods of data mining developed in DBMiner to the Web context.

With the addition of context information they were able to meaningfully classify Web page hits according to the educational activity they represented. This led to some interesting conclusions about changing patterns of use over time, but deep insights about learning styles were not yet forthcoming. One of the project's conclusions was that even with the added information, the Web log was still not a rich enough source of data. This work has since moved on to consider the content of postings, including some leading edge work on the classification and mining of multimedia data. If rich data like these can be gathered from online sources, then the techniques of data mining may yet have a lot to offer, and lead us closer to the discovery of gold.

Jud Burtis
For more information:
Fayyad, U., , Haussler, D., , and Stolorz, P. (1996)
Brachman, R., Khabaza, T., Kloesgen, W., Piatetsky-Shapiro, G., and Simoudis, E. (1996)
Xin, M. & Fisher, B. (1998)
Zaiane, O., Xin, M. & Han, J. (1998)

Latent Semantic Analysis: Automating Content Analysis of Online Discourse

Until recently, automated analysis of online discourse was impractical if not impossible. Tedious and time-consuming manual procedures were required to perform all but the most superficial assessments of discourse. Recent advances, largely by the Scientific Applications of Latent Semantic Analysis (SALSA) group at the University of Colorado at Boulder, suggest that fully automated assessment of discourse may be possible. The work of the SALSA group is not limited to assessment. They are pursuing investigations of Latent Semantic Analysis (LSA) that range from modeling second-language acquisition to the use of LSA as a way to generate summaries of text. Their interest in automated assessment, however, has focused on the assessment of essays and student-generated summaries of textbook materials. The application of LSA to the characterization of discourse generated in online settings represents a new test of LSA's capabilities.

Latent semantic analysis (LSA) is a statistical technique used to extract the deep meaning of patterns of words in specific contexts of use. The technique is performed by applying methods from linear algebra (matrix decomposition and dimension reduction) to matrices that represent usage patterns of terms in documents. It is also a theory about knowledge acquisition in human beings. Current applications of LSA include indexing and information retrieval, cross-language retrieval, assessment of text coherence, automated grading of essays, educational text selection, and summary scoring and revision.

LSA represents both a statistical technique and a model of human knowledge acquisition. Landauer and Dumais (1997) propose LSA as a model that could provide a solution to the question of how do individuals know so much given as little information as they get? This problem is variously known as Plato's Problem, the "Problem of Induction", the "poverty of the stimulus", or "the problem of the expert". (Plato's solution was that individuals possess innate knowledge and only need some stimulation to reveal it.)

LSA provides a high-dimensional (yet still reduced in dimensions as compared with "reality") representation of the associations between words and the documents containing those words. The final output from LSA is a series of measures that describe the relationships between words, documents, or words-and-documents.

The application of LSA-based techniques to the assessment of online discourse is a new one, being pursued by the present author and other researchers in the Institute for Knowledge Innovation and Technology at the University of Toronto. Whereas LSA-based techniques have been successfully applied to the assessment of student essays, there are a number of unique problems associated with the assessment of pieces of text from an online discourse environment. Perhaps the most dramatic differences between student-generated essays and student-generated online notes is the length of the text. In a typical essay setting, students are asked to write a composition that is several hundred words in length. In contrast, the average note length in a Knowledge Forum database is typically less than 200 words. Another important difference is the breadth of material that students cover. Notes in Knowledge Forum typically arise from students' investigations of phenomena and pursuit of interests that may stray from a strictly defined course of study.

Chris Teplovs
Fro more information:
LSA@CU Boulder

Quality of Service

Imagine yourself back in 1995. You wish to put a course online. What tools are available to you? How many students can you handle with those tools? What will their experience be like? Will it be easy to use? Frustrating? Wonderful? Now imagine that you aren't putting a single class online. Your eventual goal is to be able to put the entire university online, with all or part of each course having an online component. What will the experience be like for the students? In 1995, it would have been quite horrible. Modems were slower, with few people having broadband connections. Web pages containing graphics often took minutes to download. Servers were easily overloaded when large numbers of students tried to access the materials at the same time. Designing course materials has to take this kind of problem into account, and the problem still exists today, even with the technological advances of the past few years. Quality of Service (QoS) refers to a set of innovative programs designed to test various components of online courses before students try to use them. These programs test for server overloads, for length of time downloading pages, etc. The idea here is to provide the online learning environment designer with a set of tools that would allow them to test for a variety of problems before the online courses are exposed to students. With these tools, we can design robust, easy to use courses that enhance the students' online learning experience.

< Quality of Service  |  Improvements in Streaming >

"QoSim": Testing the Learning Environment

Against the background of 1995, it wasn't easy to see how large numbers of students could be accommodated with existing online resources such as modems and servers. It was common for sites to sustain crashes and other problems related to unexpectedly high demand. Other common problems of that era, some of which continue to this day, include slow-to-load Web pages, system slowdowns as more people come online, broken links, and so forth. As the design of systems such as Virtual U progressed, it was realized that some system for testing the systems being produced was necessary before unleashing these products on an unsuspecting student population. The systems had to be robust, reliable and pleasant for the students to use. QoSim was designed to be a simulator for evaluating the quality of service obtainable under different network architectures and user interaction conditions, and designed to enable course designers to eliminate as many of the bothersome problems as possible.

QoSim works by simulating extreme conditions of use. For example, in one set of tests, the performance of web-servers (Linux and NT) was tested for load delay characteristics by having multiple simulated clients randomly demand patterns of files. The data thus obtained was used to tune the parameters of a software server model. The model was then used to evaluate the scalability and performance of different architectures for load-balancing in web-server clusters. Such load-balancing is critical to the performance of web-servers and impacts on the installation costs to the institution, and to the quality of the user experience.

In general then, QoSim evaluates the quality of service obtainable with different course parameters, network configurations, technologies and network protocols. It allows the determination of the location of bottlenecks in the system performance, the prediction of the system capacity and the prediction of the effect of the implementation of new technologies. It also allows for the prediction of the effect on the system performance of adding more multimedia content, such as image downloads, and streaming video lectures, etc. to on-line courses. This greatly streamlines the design process for the environment side of the online course experience _ the actual hardware and software on which the courses will run. With the speed at which courses are going online, a trial and error approach to delivery of service to students is just not good enough, and QoSim allows for a much smoother transition to new levels of demand for these services.

For more information:
Harasim, L. (2000)

Improvements in the Flexible Transmission of Streaming Audio and Video.

Although streaming video is now common on the Web, in 1995 the technology was in its infancy and Telelearning NCE researchers contributed important innovations that helped to make it more widely accessible. Streaming refers to a process by which media information (audio or video) is sent over the Internet in compressed form. This information is then reassembled in real time by the user's computer at the other end. The result is that the media information is sent as a continuous stream and played as it arrives at the user's computer. Typically, streamed media is buffered on first arrival. In buffering, a certain number of seconds of information (often 30 seconds) is stored before presentation. This buffering allows for delays on the Internet, so that the user is presented with a continuous stream of information without interruption. At least that's the theory. In practice, there are often interruptions at peak demand times.

Streaming audio refers to the presentation of an audio stream of information. In streaming audio, no images are presented. Streaming audio uses less bandwidth than other streaming media, and is therefore the least problematic to transmit. Streaming video refers to the presentation of a sequence of images in streamed form. The rapidity with which these can be reassembled and presented gives the quality of motion. Lower quality video streams give a lower quality video _ the images can have a jerky quality. Higher rates of transmission smooth this out, giving a higher quality, but require higher bandwidth. Streaming video requires much more bandwidth than streaming audio. Streaming media refers to presentation including both streaming video and the accompanying audio. This is the most demanding of all in terms of bandwidth. Some use the terms streaming video and streaming media as synonyms, and that is its usage in the context of this project.

Streaming audio, and video are of use to educators as they can be used to present recordings of discussions, lectures, demonstrations, dramatic presentations, etc. In 1995, streaming was in its infancy, and very few presentations were available, often of poor quality. Thus, the initiative on the flexible transmission of streaming audio and video was very important to the development of course materials.

The principal investigators for this project were Aude Dufresne, Université de Montréal; Michel Duguay, Université Laval; Dan Ionescu, U. of Ottawa; Ze-Nian Li, SFU; Wo-Shun Luk, SFU; Emil Petriu, U. of Ottawa; and Samuel Pierre, LICEF, Tele-Université. They proceeded by first creating streaming media "files" for transmission over the Internet. They then tested the capacity of the LearningAid environment to deliver streaming video to many different asynchronous clients simultaneously. They used such information to study how to optimally schedule the transmission of streaming video as a function of the decoder buffer-size at the receiving end. This resulted in an algorithm for increased network carrying capacity and reduced delays in the network. At this time (May, 2002), a demonstration of the streaming video work is available at http://jupiter.cs.sfu.ca/, including a Flash animation streaming video presentation.

Use of a Movement Distracter: Helping Students to Push the Right Buttons

When the learner interacts with a complex visual environment such as a web page, there are features that can go unnoticed. How can you direct the learner's attention to the presence on the page of various buttons and icons for desirable functions? In 1995 there were no existing techniques for measuring the level of engagement of the student with the asynchronous environment, and no way of ensuring that they would "find" all of the features built into the learning environment. Therefore, Lyn Bartram, in close consultation with the project leader Tom Calvert (University of B.C.) and in collaboration with Dr. Colin Ware (University of New Hampshire) set out to investigate the level of learner engagement with the learning environment.

A series of experiments were carried out to determine the role of movement in catching the attention of a student. The features of movement that were investigated included the amplitude of movement, the frequency of movement, and the shape of the movement path. These results were then used to create a distracter of variable intensity. The distracter was then used in experiments to investigate the hypothesis that the strength of the distracter required to catch the attention of a learner engaged in a learning task provides a measure of engagement. With this information, it is then possible to compare the engagement of the student in different learning environments.

One result of this research relates to the use of movement in icons on web pages as agents to attract attention to their presence. Failure to detect these icons was less than 2% in any location on the page compared with a failure rate of up to 25% in remote locations on the page for static icons. This relatively high rate of attention getting was maintained even when the learner was engaged in attentionally demanding tasks, suggesting that movement is effective over a wide range of locations, types, and amplitudes. In general, slow linear motion was suggested as the best attention getting compromise and that animated banners and popping images were not comfortable visual elements, confirming what our intuitions suggest.

Don Philip
For more information:
Bartram, L., Ware, C. and Calvert, T. (2001)

Models and Tools for Cost-benefit Analysis

The emergence of new telelearning technologies has prompted schools and organizations to invest heavily in online learning. There are a number of rationales for implementing online education programs, including increased access to courses, improved quality of learning and reduced costs. However, in order to justify long-term investments in online learning it is important that the impact of telelearning be systematically evaluated. Institutions need to determine if their investment will yield significant benefit compared to more traditional methods of instruction. To address this issue, the TL-NCE set out to develop and apply a cost-benefit model for assessing telelearning using the ACTIONS methodology developed Tony Bates. The goal of the project was to examine the costs and benefits of using telelearning technologies in different contexts, and develop models for assessing the institutional impact of different levels of commitment to telelearning (Bartolic & Bates, 1999). The project examined cost structure and outputs/achievements of selected projects within the telelearning network.

ACTIONS

The ACTIONS methodology defines cost and benefit criteria for selecting and applying technology to online learning (Bartolic & Bates,1999). Cost factors evaluated within the ACTIONS framework include capital costs for buying equipment and materials; recurrent costs that occur on an ongoing basis, such as computer support costs; production costs, required for the development of the program; and delivery costs, associated with the teaching of the course material. Also, costs can be fixed or variable. Fixed costs are not affected by the number of students and include costs for subject experts, Internet and design specialists, server costs, departmental overheads, copyright clearances and international tutors. Variable costs differ based on student numbers and therefore include mainly personnel costs for instructors, teaching assistants, and the like.

Benefit and limitation factors that are assessed within the ACTIONS methodology include performance-driven benefits, such as student/teacher satisfaction, learning outcomes and return on investment; value-driven benefits, including increased access, flexibility and user-friendliness; and societal benefits, which include environmental factors such as reduced traffic to campus, and the potential to tap into new markets (Bartolic & Bates, 1999).

TL-NCE Case Study Findings

Using ACTIONS, two case studies describing online courses at different universities were conducted within the TL-NCE project. Each study, using common standards and methodology, sought to define widely applicable benefits and limitations of using online learning in addition to reporting findings related to the particular case being investigated (Bartolic-Zlomislic & Bates, 1999; Bartolic-Zlomislic & Brett, 1999). Key findings from the studies included;

Access: The online courses provided increased access to international experts and a diverse community of students as well as a more flexible work schedule in terms of time and place. However, access was dependent on the availability and stability of the technology; unstable technology was found to disrupt the learning process and courses availability was limited to students who were able to access the necessary technology.

Teaching and learning: Teaching and learning functions were improved in a number of ways; students were able to learning computer skills in addition to the regular course content and were given more opportunity to practice their writing and time-management skills. The studies also found that the written work produced was of a higher quality compared to face-to-face courses. However, there were also drawbacks to online teaching and learning. Students on campus were found to prefer face-to-face classes, finding online discussion too slow and expressing concern about the permanency of their online postings.

Interaction and user friendliness: Telelearning technologies were found to offer increased opportunities for interactions between students and instructors, however the possibility of miscommunication when interacting online and the lack of face-to face contact was a concern for students.

Novelty: The novelty of online course delivery required that some course time be allocated to helping students learn how to navigate through the course, detracting from time spent on course material.

Speed: Online learning enabled instructors to developed revised courses quickly and easily.

Overall, the studies identified the benefits and limitations of using telelearning technologies. However, the studies concluded that there is no single answer to the cost-benefit question. The final decision on whether to invest in online learning depends on the values and goals of the institution in question.

Maria Mylopoulos