Hirumi
Get Atsusi a Life: Six Tactics for Optimizing Time Spent Online
SUMMARY. A frequent concern raised by distance educators is that elearning takes more time to facilitate than traditional classroom instruction. The simple fact that it takes more time to read and write than to speak and listen warrants consideration. To establish viable e-learning programs, we need to optimize the amount of time educators spend online. This article posits five tactics for optimizing time spent facilitating the e-learning process and one tactic for optimizing time spent developing e-learning materials. Together, the tactics applied within the context of an overall systematic instructional design process yield replicable results. The investment in systematic design is thought worthwhile because the materials are reusable and allow instructors to focus their attention on facilitating, rather than directing and clarifying, the e-learning process. [Article copies available for a fee from The Haworth Document Delivery Service: 1-800-HAWORTH. E-mail address: <docdelivery@haworthpress. com> Website: <http://www.HaworthPress.com> 2003 by The Haworth Press, Inc. All rights reserved.]
ATSUSI HIRUMI is Assistant Professor and Program Chair, Instructional Technology Department, University of HoustonClear Lake, Houston, TX 77058 (E-mail: [e-mail]).
[Haworth co-indexing entry note]: "Get a Life: Six Tactics for Optimizing Time Spent Online." Hirumi, Atsusi. Co-published simultaneously in Computers in the Schools (The Haworth Press, Inc.) Vol. 20, No. 3, 2003, pp. 73-101; and: Distance Education: What Works Well (ed: Michael Corry, and Chih-Hsiung Tu) The Haworth Press, Inc., 2003, pp. 73-101. Single or multiple copies of this article are available for a fee from The Haworth Document Delivery Service [1-800-HAWORTH, 9:00 a.m. - 5:00 p.m. (EST). E-mail address: [e-mail]].
http://www.haworthpress.com/store/product.asp?sku=J025 2003 by The Haworth Press, Inc. All rights reserved. 10.1300/J025v20n03_09
73
74
DISTANCE EDUCATION: WHAT WORKS WELL
KEYWORDS. E-learning, Web-based instruction, interactivity, online learning, Web-based education, systematic design Do you feel that e-learning takes too much time: too much time to design high-quality e-learning materials and too much time to manage the e-learning process? Modern e-learning management systems (LMS), such as CourseInfo, WebCT, Centra, and WebEx, make it easier to facilitate telecommunications and to post educational materials online. However, LMSs neither transform traditional teaching materials into effective e-learning resources, nor prevent educators from being inundated by e-mail and bulletin board postings. Ease of use does not necessarily translate into the development of innovative environments that use the potential of emerging technologies to stimulate collaborative and individual learning. On the contrary, an LMS can lead to a false sense of efficiency and an increased workload. For teachers, ease of use may mean that the LMS organizes information in familiar ways, using terms and features common in traditional classroom settings. Administrators may think that with an LMS, a user's manual, and some training, educators should be able to author marketable e-learning materials and offer meaningful e-learning experiences with little to no additional time or resources. Under such conditions, educators have little choice but to do what they know best. In most cases, that means applying teacher-directed instructional methods and materials. The problem is that traditional classroom teaching methods and materials are often insufficient or inappropriate for facilitating e-learning. Lecture notes, overhead transparencies (converted into PowerPoint slides), handouts, and course syllabi are based on an oral teaching tradition. They rely heavily on an instructor and students to speak and listen to each other to convey information, clarify expectations, define procedures, explain requirements, establish policies, complete assignments, and otherwise derive meaning from and construct knowledge of the subject matter in real time. Simply translating traditional classroom materials into an electronic format does not ensure that essential interactions will occur. During e-learning, opportunities for synchronous interactions are relatively limited. Most communications occur asynchronously through reading and writing. Key interactions that occur verbally and spontaneously in traditional classroom settings must be written and planned as an integral part of e-learning. The effort necessary to generate, disseminate, read, interpret, and otherwise manage written communications and inadequate course design are believed to be the two primary reasons why it takes so much time to teach online. So what can educators do to optimize time spent online? If educators are given access or time to generate quality e-learning materials, will it still take more time to facilitate and manage e-learning than traditional classroom in-
Atsusi Hirumi
75
struction? What should educators do with limited time and resources? This paper explores these questions and posits several potential answers. Specifically, I describe five tactics for optimizing time spent facilitating the e-learning process and one tactic for optimizing time spent producing e-learning materials. Even with such tactics, it is important to recognize that the design and development of high quality instructional materials take considerable time and effort (for any learning environment). I conclude by discussing key factors to consider when determining if the tactics yield a sufficient return on investment.
SIX OPTIMIZING TACTICS Align and Publish Objectives and Assessments
Learning objectives and student assessments are well-known components of the instructional process. Few, however, seem to approach these two vital elements in a systematic fashion. As a student, have you ever taken a test and wondered where some of the questions came from? Have you ever doubted what would be on a test or what the teacher was looking for in a class project or assignment? Have you ever spent a lot of time and effort on an assignment only to receive a bad grade, not because you didn't have the skills or knowledge, but because you just did not know what the instructor wanted? Teachers often spend time before, during, or after class clarifying expectations. Questions such as "Is this what you want?" and "What will be on the test?" are common in traditional classroom settings. During e-learning, such questions can easily overwhelm distance educators. Instructors may have to spend considerable time online clarifying expectations if (a) the objectives and performance criteria are not explicit, and (b) the behaviors specified in course objectives or practiced in class do not match the behaviors required to successfully complete assignments and examinations. The alignment of objectives and assessment criteria is fundamental to highquality instruction (Berge, 2002; Dick, Carey, & Carey, 2001). To optimize learning, students should be informed of what they are expected to know and be able to do, and they should be evaluated using assessment items and published performance criteria that match the behaviors specified in the objectives. If an objective states that students will be able to list key concepts, assessments should ask students to list key concepts. If an objective states that learners will be able to compare cases, assessment should ask learners to compare cases. In traditional classroom settings, educators can readily address questions regarding unclear or contradictory expectations. During e-learning, the communications necessary to clarify misaligned or poorly communicated objectives and criteria can take a lot of time, particularly if students submit unsatisfactory assignments that require considerable feedback and remediation. Worse yet, in-
76
DISTANCE EDUCATION: WHAT WORKS WELL
congruent instructional elements may result in significant misunderstandings that are left unattended because learners are too confused or otherwise reluctant to initiate communications with the instructor. If the elements are aligned, you can optimize time online by preventing unnecessary logistical questions, by referring learners to published objectives and criteria if asked to clarify expectations, and by using published criteria to prepare timely and detailed feedback. The development of a design evaluation chart can help ensure alignment between objectives and assessments. Table 1 depicts a two-column chart prepared for an instructional unit on criterion-referenced testing. Column one lists the objectives that are to be addressed by the unit. Column two specifies the assessment item(s) or assessment criteria that are to assess learners' achievement of the objectives. Alignment is achieved by matching the behaviors specified in each column. Note that to limit the size of Table 1 for publication only a limited set of objectives related to the unit is depicted. Column two indicates that three types of assessments are to measure achievement of the specified unit objectives. A portfolio assessment rubric is to assess learner achievement of the terminal objective. A conventional multiple-choice test and a product checklist are to measure learners' achievement of the enabling objectives. Note that the assessment items and performance criteria listed in column two match the behaviors specified in the corresponding learning objectives in column one. Of course, alignment is only useful if the objectives are valid. Before you begin aligning elements, ask yourself, "What is the course supposed to do?" List the learning goals and objectives for the entire course and assure that they are appropriate and are written in clear and measurable terms. Use the development of an e-learning course as an opportunity to reassess your goals and objectives. If you are unsure or do not have clearly defined objectives, consider some form of task analysis (cf., Jonassen, Tessmer, & Hannum, 1999) to identify essential skills and knowledge. Then generate, cluster, and sequence objectives into instructional units based on your analysis and prepare a design evaluation chart to align assessment criteria and items with the objectives.
Align Instructional Events to Support Objectives and Assessments
To facilitate e-learning, assignments and activities (i.e., instructional events) should also be aligned to support the achievement of specified objectives (Berge, 2002; Smith & Ragan, 1999). Students should be given opportunities to practice and otherwise develop the skills and knowledge specified in course and lesson objectives. To align instructional events with learning objectives, classify targeted objectives according to a particular learning taxonomy (e.g., Gagn, 1977). Then apply the assignments, activities, and instructional events that have been found to facilitate achievement of the desired outcomes. The methods used to teach concepts should differ from the methods used to teach a
Atsusi Hirumi
77
TABLE 1. Sample Design Evaluation Chart for Instructional Unit on CriterionReferenced Testing
1. Objective Assessment Item Given a set of instructional objectives, Descriptors for exemplary performance level for assessment rubric will be used to evalugenerate conventional and/or perforate criterion-referenced test instruments. mance-based criterion-referenced 1. Goal-Centered Criteria tests that are congruent with targeted a. Matches behavior, including the goals: learner, context, and published action and concepts, prescribed in assessment criteria. objectives. b. Meets conditions specified in the objectives. 2. Learner-Centered Criteria a. Tailored to learners in terms of vocabulary, language levels, developmental levels, motivational and interest levels, experiences, backgrounds and special needs. b. Free of gender, ethnic, or cultural biases. 3. Context-Centered Criteria a. Realistic or authentic to actual performance setting as possible. b. Feasible and suited to resources available in learning setting. 4. Assessment-Centered Criteria (for conventional test items) a. Applies correct grammar, spelling, and punctuation. b. Stem clearly formulates problem. c. Stem contains task and most information, keeping answers/options short. d. Stem includes only required information and is written in clear, positive manner. e. Includes only one correct, defensible, best answer. Foils are plausible and do not contain unintentional clues. Given a set of instructional situations, 1. Learner assessments are created after recognize when learner assessments ______ during the _____ phase of the are generated during systematic desystematic design process. (a) establishing instructional strategy, sign process. design (b) defining objectives, design (c) defining objectives, development (d) conducting goal analysis, analysis (e) selecting media, development Given a set of instructional situations, 1. Criterion-referenced tests can be used to recall purposes of criterion-referenced measure students' entry-level behaviors. testing (CRT). a. true b. false 2. Criterion-referenced tests can be used to monitor students' progress toward learning objectives. a. true b. false
2.
3.
78
DISTANCE EDUCATION: WHAT WORKS WELL TABLE 1 (continued)
4.
Objective Given a set of instructional situations, distinguish two forms of CRT.
1.
5.
Given a set of instructional situations, identify criteria for generating quality conventional assessment items.
1.
2.
6.
Given a set of instructional situations, contrast conventional and performance-based assessments.
1.
7.
Given the results of a goal and subordinate skills analysis and a set of objectives, generate and align assessment items by generating a design evaluation chart.
1.
Assessment Item The two basic forms of criterion-referenced tests are conventional and ______________. a. norm-referenced b. fill-in-the-blank c. product checklists d. portfolio assessments e. performance-based assessments An important goal-centered criterion for generating quality assessment instruments is that the items: a. match the behavior prescribed in objectives. b. are tailored to learners in terms of vocabulary and language. c. are realistic to actual performance setting. d. apply correct grammar. e. are performance-based assessments. An important assessment-centered criterion for generating quality conventional test items is that the foils: (a) contain most of the information. (b) include only required information. (c) meet the conditions specified in the objectives. (d) are all plausible but consist of one best answer. (e) clearly formulate the problem. In contrast to conventional CRT, performance-based CRT: a. are used to sort and rank students. b. are one dimensional and episodic. c. measure students' acquisition of skills and knowledge. d. present explicit and published performance criteria. e. contain criteria that are prescribed by the instructor. Product checklist items with two possible levels of performance: (a) yes, and (b) no, and an opportunity to comment. a. assessment method clearly defined and appropriate for context and objectives. b. evaluation chart properly formatted. c. specified skills consistent with those identified in goal and subordinate skills analyses. d. objectives consistent with specified skills. e. assessment items congruent with objectives.
Atsusi Hirumi
79
procedure that, in turn, should differ from the methods to teach complex problem solving, and so on. Smith and Ragan (1999) classify alternative instructional events that have been found to facilitate achievement of various learning outcomes (Table 2). Use the table to assure that you are using appropriate instructional events to facilitate e-learning and promote the achievement of targeted objectives. If an instructional unit addresses a range of learning objectives, place emphasis on the terminal objective (or the primary targeted learning outcome) when selecting an appropriate set of instructional events. Adding a column to the design evaluation chart depicted in Table 1 will help ensure alignment between objectives, assessments, and instructional events (Table 3). Column three describes the instructional events that will be used to help learners achieve the specified objectives. Again, alignment is achieved by matching the behaviors specified in each column. Column 3 notes that the learners are to generate criterion-referenced tests and complete reading assignments, a quiz, and a performance checklist to achieve the enabling and terminal objectives. In addition, grounded instructional events, or activities "rooted in established theory and research in human learning" (Hannifin, Hannifin, Land & Oliver, 1997, p. 102) are planned to facilitate e-learning. In this particular case, grounded events associated with the development of problem-solving skills (i.e., presentation of problem, discussion of problem space, identification of relevant principles and practice) are to be integrated within the unit. Note that to limit the size of Table 3 for publication only a limited set of objectives and assessments related to the unit is depicted. Failure to align instructional events with specified learning objectives may result in insufficient or inappropriate learning. Students may find it difficult to successfully complete assignments and exams that, in turn, may require considerable feedback and remediation. The alignment of grounded instructional events is fundamental to high-quality instruction and particularly important to e-learning because the instructor may not be readily available nor have the time necessary to correct flaws in design. For further details on the design and sequencing of instructional events to formulate a comprehensive e-learning strategy, refer to Hirumi (2002a).
Analyze and Balance Interactions
Web courses that contain many interactions can be more complicated to complete than relatively linear programs (Gilbert & Moore, 1998). For novice distance learners, such complexity may lead to feelings of helplessness and confusion and eventually cause one to drop out. Dissatisfaction may also occur if learners perceive online interactions as meaningless busy work. Even experienced distance learners may find that the overuse or misuse of interactions results in frustration, boredom, and cognitive overload (Berge, 1999). Too many interactions may also overwhelm the instructor. A common concern ex-
80
DISTANCE EDUCATION: WHAT WORKS WELL TABLE 2. Grounded Instructional Events
Outcome Grounded Events 1. Associational Techniques a. Mnemonic & metaphoric devices b. Instructor- or learner-generated images (e.g., graphs) and rehearsal (e.g., drill and practice) Organizational Techniques a. Clustering and chunking into categories (e.g., periodic table) b. Expository and narrative structures (e.g., chronologies, cause and effect, problem solutions, contrasts) c. Graphic and advanced organizers (e.g., concept tree or information linking new to prior knowledge) Elaboration Techniques a. Write meaningful sentences (e.g., sentences using elements of periodic table) b. Devise rule (e.g., describe why elements are organized in rows and columns) Inquiry approach (e.g., exploratory learning beginning with examples and non-examples) Expository approach (typically begins with an explanation of a concept and its key attributes) Attribute isolation (points out the critical attributes of a concept) Concept trees (graphic representations that illustrate subordinate and superordinate concepts) Analogies (supplied by instructor or generated by learners) and mnemonics Imagery (a mental image of concrete concepts, such as pictures, graphs, tables, and maps) Learn to determine if the procedure is required. Learn to list the steps in a procedure. Learn to complete the steps in a procedure. Learn to elaborate sequence, starting with simple epitome to more complex versions of same rule. Learn to check appropriateness of completed procedure.
1.
Verbal Information Names, labels, facts, or a collection of propositions
2.
3.
2.
Concepts Specific objects, symbols, or events, grouped on the basis of shared characteristics and which can be referenced by a particular name or symbol
1. 2. 3. 4. 5. 6.
3.
Rules Relational rules or principals and procedural rules or procedures
1. 2. 3. 4. 5.
Atsusi Hirumi
Outcome 4. Problem Solving Learned principles, procedures, verbal information and cognitive strategies combined in a unique way within a domain to solve original problems 1. 2. Grounded Events
81
3.
4.
Present problem (Case studies, simulations, limiting the number of rules that must be used.) Discuss problem space (Review directions and information about desired goal state. Define relationship between variables in current and goal state. Analyze relationships and discern patterns. Define knowns and unknowns. Determine information requirements. Break down problem into intermediate states.) Identify appropriate principles (Guide questions or direct statements on how to select and apply appropriate principles.) Practice (Present multiple representations of the problem. Recommend techniques for limiting alternative approaches to problem resolution. Provide clues about solution or intermediate solutions. Recommend strategies for acquiring information. Outline approaches for problem resolution. Establish criteria for evaluating the appropriateness of alternative solutions.) Discovery and guided discovery (Involves more direct instruction than discovery, helping learners ascertain particular strategies through the application of questioning strategies.) Observation (Observe a model demonstrating the use of the strategy by paired, cooperative learners, expert demonstration, and symbolic visual or textual representation by fictional character.) Guided participation (Work with learners to determine characteristics of learning task, identify strategies to facilitate the task, and determine effective methods for employing the strategy.) Direct instruction (Identify utility of the strategy. Provide overview of steps and their relation to strategy. Demonstrate the strategy. Illustrate examples and non-examples. Practice application of the strategy across gradually more difficult situations. Provide corrective feedback. Give explicit encouragement and guidance to transfer strategy to separate but appropriate context.)
5.
Cognitive Strategies Internally organized skills whose function is to regulate and monitor the utilization of concepts and rules
1.
2.
3.
4.
82
DISTANCE EDUCATION: WHAT WORKS WELL TABLE 2 (continued)
Outcome Grounded Events 1. 2. Demonstrate desired behaviors representative of target attitude by a respected role. Practice desired behavior associated with the desired attitudeanother powerful tool in attitude formation and change (e.g., role playing and group discussions). Provide reinforcement for the desired behavior (increases probability of the behavior recurring). Communicate persuasive messages from highly credible sources. Create dissonance by persuading learner to perform an important behavior that is counter-dissonant to the person's own attitude. Attitude change may result. Massed versus spaced practice (Massed practice engages learners in one or a few intensive periods of practice. Spaced practice exposes learners to short practice sessions distributed over time.) Whole versus parts practice (Whole practice is advisable if the task is simple, not meaningful in parts, made up of simultaneously performed parts, and has highly dependent parts, and if the learner is able to remember long sequences, has long attention spans, and is highly skilled.) Progressive parts practice (If learners may have difficulties putting the parts together into a meaningful and well-executed whole.) Backwards chaining (Where learners are exposed to and practice the last step and work their way to the first step.)
6.
Attitudes Choice behaviors that do not necessarily determine specific actions but do make certain classes of action more or less probable
3. 4. 5.
7.
Psychomotor Skills Coordinated muscular movements that may be difficult to distinguish from intellectual skills
1.
2.
3.
4.
Note. From Instructional Design (2nd ed.) by P. L. Smith & T. J. Ragan (1999).
pressed by distance educators is that it takes far more time to grade assignments and manage communications during e-learning than it does in traditional settings. For both learners and educators, too many interactions may impede, rather than facilitate learning. Educators often ask students to complete many assignments to demonstrate mastery of important skills and concepts. While their intentions are good, the
Atsusi Hirumi
83
TABLE 3. Extended Design Evaluation Chart Used to Help Ensure Alignment of Instructional Strategy
Objective 1. Given a set of instructional objectives, generate conventional and/or performance-based cyber-tests that are congruent with targeted objectives (learner and context). Assessment Item Major categories for descriptors to be included in the assessment rubric that will be used to evaluate criterionreferenced test (CRT) instruments. Descriptor specific to each category listed under Table 1. Instructional Strategy Assignment #6 (Part II)Generate assessment instrument. At the end of the instructional unit, ask learners to generate a conventional CRT, an assessment rubric, and/or a checklist. Assignment description to provide clues about the general form of the solution. Publish criteria for evaluating solutions. Learners to respond to the posted questions and post additional questions or comments about the information covered in this unit in the proper location on the course bulletin board system. Related questions should facilitate discussion of problem space. Read Dick, Carey, & Carey (2001), Chapter 7"Developing Assessment Instruments." Chapter 7 addresses: 1. When learner assessments are addressed during systematic design process 2. Purposes of criterionreferenced testing 3. Methods for generating conventional criterionreferenced testing (CRT) instruments 4. Criteria for evaluating the quality of conventional CRT Read Unit 6.0 supplemental materials on Learner Assessment Methods. Supplemental reading addresses: 1. Alternative testing methods (compares and contrasts norm-referenced, conventional, criterionreferenced, and performance-based) 2. Purposes of performancebased CRT (portfolio and checklists)
1. 2. 3. 4.
Goal-Centered Criteria Learner-Centered Criteria Context-Centered Criteria Assessment-Centered Criteria
2. Given a set of instructional situations, recognize when learner assessments are generated during systematic design process.
Learner assessments are created after _______ during the _____ phase of the systematic design process. 1. establishing instructional strategy, design 2. defining objectives, design 3. defining objectives, development 4. conducting goal analysis, analysis 5. selecting media, development
3. Given a set of instructional situations, recall purposes of criterion-referenced testing (CRT).
Criterion-referenced tests can be used to measure students' entry-level behaviors. 1. true 2. false Criterion-referenced tests can be used to monitor students' progress toward specified learning objectives. 1. true 2. false
84
DISTANCE EDUCATION: WHAT WORKS WELL TABLE 3 (continued)
Objective Assessment Item The two basic forms of crite4. Given a set of instructional situations, rion-referenced tests are condistinguish two forms ventional and ___________. 1. norm-referenced of CRT. 2. fill-in-the-blank 3. product checklists 4. portfolio assessments 5. performance-based assessments
Instructional Strategy 3. Key components of portfolio assessment methods 4. Key components of performance checklists 5. Methods for generating performance-based CRT (assessment rubrics and checklists) 6. Criteria for evaluating the quality of performancebased CRT Textbook and supplemental reading
5. Given a set of instructional situations, characterize key components of portfolio assessments.
Analytic performance assessments are based on 1. an overall impression of work samples. 2. different dimensions or components of work. 3. the conditions specified in the objectives. 4. varying performance levels or scales. 5. key learner characteristics and contextual factors.
Portfolio assessments should include three key components, including 1. conventional tests, narrative, and objectives. 2. narrative, criteria, and assessment rubrics. 3. work samples, test instruments, and contracts. 4. narrative, work samples, and performance criteria. 5. work samples, analytic and holistic assessment rubrics. 6. Given the results of a Product checklist items with two goal and subordinate possible levels of performance: skills analysis and a (a) yes, and (b) no, and an opportunity to comment. set of instructional objectives, generate 1. Assessment method clearly defined and appropriate for and align assesscontext and objectives ment items by gener2. Evaluation chart properly ating a design formatted evaluation chart. 3. Specified skills consistent with those identified in goal and subordinate skills analyses 4. Objectives consistent with specified skills 5. Assessment items congruent with objectives
Assignment #6 (Part I)Generate design evaluation chart that will be used to create assessment instruments. Description of assignment provides clues about the general form of the solution and criteria for evaluating alternative solutions. Learners to respond to the posted questions and post additional questions or comments about the information covered in this unit in the proper location on the course bulletin board system. Related questions should facilitate discussion of problem space.
Atsusi Hirumi
85
consequences in terms of time management must be considered. In traditional classroom settings, students simply hand in assignments that the instructor stores until she or he is ready to grade. Review and evaluation take time, but written feedback may be brief because the instructor can supplement written comments with verbal explanations. During e-learning, the instructor must acknowledge receipt of assignments; save and store files; print out, review and assess each document; generate and distribute detailed feedback; and assure learners understand the feedback. That's at least five time-consuming tasks for every assignment. If you have 30 students and require them to complete 10 assignments, you may have to manage as many as 1,500 interactions per course. Considering the time and effort necessary to complete each interaction, it is easy to see why distance educators may feel overwhelmed during the delivery of an online course. The fact that written communication takes considerably more time to prepare and manage than oral communication further accentuates the importance of analyzing and balancing planned e-learning interactions. In another article, I detail systematic methods for analyzing planned elearning interactions (Hirumi, 2002b). In short, you may be able to determine if you have too many or too few interactions by examining the frequency and nature of planned learner-human and learner-nonhuman interactions (Table 4). In this article, I describe one specific tactic for balancing learner-instructor interactions that requires teamwork with a just evaluation of individual student contributions. It takes considerably less time to manage and evaluate 6 group assignments, rather than 30 individual assignmentstime you could spend on other tasks or providing more detailed and timely feedback. The challenge lies in designing group projects so that all participants learn and contribute equitably. Group work can be difficult enough when students meet regularly in class. At a distance, the basic challenges remain the same, but the methods used to overcome them may differ. Successful group work requires effective planning and communications. Group members must contribute to the completion of assignments and a fair and accurate assessment must be made of each member's efforts. Specific tools and techniques are necessary to facilitate such vital group interactions online. Job aids and published procedures for using e-mail, bulletin board systems and chat rooms can help ensure that distance learners have the skills necessary to telecommunicate. Training and information on (virtual) teamwork and group problem-solving skills may also help learners communicate with one another. To assess each person's contribution, you can ask learners to complete the assignment on their own before working with teammates to produce a final "group" assignment and ask group members to evaluate one another's work. The basic design of individual and group assignments in an online course illustrates how each of these techniques may be applied to optimize time spent online.
86
DISTANCE EDUCATION: WHAT WORKS WELL TABLE 4. Sample Analysis of Planned E-Learning Interactions
Interaction Quantity 21 Quality Design Decision
1. LearnerContent
1. One lesson overview page Interface very important to that provides description of test prior to official course delivery and links to information about introduction, task, process, resources, evaluation, and conclusion. 2. Detailed descriptions of how to complete each of the 10 tasks associated with the process. 3. Links to 7 resources. 4. Two detailed evaluation rubrics. 5. Description of how to prepare and submit journal entry. 1. Ask learner to post message. 2. Review and provide feedback on topic. 3. Review and provide feedback on problem statement. 4. Provide guidance on writing final report. 5. Provide guidance on preparing debriefing. 6. Assess and provide feedback on final report. 7. Assess and provide feedback on debriefing. 8. Review and provide feedback on journal entries. Far too many interactions to manage (need to review and revise by grouping two or more interactions, grouping students, eliminating or further automating interactions)
2. LearnerInstructor
8
3. LearnerLearner
5
1. Share short description of pre- May be too much (need reviously seen or written reports. view with particular attention 2. Share and discuss problem during testing) statements. 3. Share and discuss purpose statements. 4. Conduct peer reviews of reports. 5. Participate and share comments on debriefings. 1. Contact librarian. 2. Contact other professors. Need to ensure librarian is prepared; need to ensure ready access to other professors
4. LearnerOther 5. LearnerEnvironment
2
3
Need to ensure ready access 1. Go to library. 2. Acquire and read textbook. to library resource and textbook 3. Acquire and read journal articles.
Note. From "A framework for analyzing, designing and sequencing planned eLearning interactions" by A. Hirumi, 2002, Quarterly Review of Distance Education, 3(2), 141-160.
Atsusi Hirumi
87
As soon as students register for a course, they are sent a welcome letter that provides directions for accessing the course and an online course orientation. During the orientation, students are asked to read through the course syllabus and complete a series of short activities to demonstrate their ability to use various telecommunications technologies. Students sign and submit a checklist to verify that they have completed the activities and are aware of important course policies and procedures. The checklist also serves as a contract to help clarify roles and responsibilities and ensure learners have the required prerequisite skills and knowledge. If learners are not able to successfully complete the activities, they are required to either attend an optional face-to-face orientation held on campus during the first week of class, contact the instructor to determine an appropriate course of action, particularly if they cannot attend the face-to-face orientation, or they may be administratively dropped from the course. During the first week, the goals are to ensure that learners (a) have a good understanding of course requirements and expectations, (b) can locate and interpret relevant policies and procedures, (c) are confident in their ability to use various tools and course features, and (d) can identify challenges associated with and discuss strategies for facilitating virtual teamwork. In addition to the online orientation materials, learners are asked to read several online articles on facilitating virtual teams and are reminded of the importance of interpersonal and virtual teamwork skills. If a problem should arise, group members are told to first attempt to work through issues their on own by identifying problems with process, not people, before contacting the instructor for help. Throughout the course, students are required to finish a series of both individual and group assignments to demonstrate achievement of course objectives. For each instructional unit, students are directed to complete all required readings, a multiple-choice quiz covering key concepts contained in the readings, and an individual draft of the unit assignment. Although the unit assignment is actually a group project, each member is tasked with completing his or her own draft of the assignment to help ensure that everyone is learning to apply newly acquired skills and knowledge. Then group members are to share their experiences and work with team members to complete a group version of the assignment using e-mail, a private bulletin board forum, chat, and whatever form of communications the team prefers. The team posts one copy of their teamwork. If a team posts a draft of their work one week before the official due date, feedback is given based on published performance criteria from the instructor so that the team can revise their work before submitting it for a grade. Each member is responsible for posting a copy of his or her individual draft, along with a short description of personal contributions, and for completing a teamwork evaluation for each member of the group. To minimize time spent online, individual drafts are evaluated based on a plus and minus system. Detailed review and feedback are reserved for the group version of the assign-
88
DISTANCE EDUCATION: WHAT WORKS WELL
ment. Teamwork evaluations consist of five measures: quality, timeliness, interpersonal skill, attitude, and quantity of contribution. Scores from each member of the team are averaged to derive a final score for each individual. The combination of individual and group assignments optimizes time spent online by limiting the number of assignments that must be evaluated and managed as an integral part of e-learning while also providing a fair assessment of each learner's work.
Create Feedback Templates
Feedback is vital to e-learning. At minimum, feedback is essential for closing message loops (Yacci, 2000; Northrup & Rasmussen, 2000), informing learners that communications are complete (Berge, 1999; Liaw & Huang, 2000; & Weller, 1988, as cited by Northrup, 2001). Feedback may also be used to (a) increase response rates or accuracy, (b) reinforce correct responses to prior stimuli, or (c) change erroneous responses (Kulhavy & Wager, 1993). During e-learning, telecommunications expand feedback options. Electronic mail, bulletin board systems, chat rooms, and audio and video conferencing may be used to provide immediate and delayed feedback and present learners with guidance, lesson sequence advisement, motivational messages, critical comparisons, and information about answer correctness and timeliness (Hoska, 1993). Without feedback, instruction may become "passing on content as it if were dogmatic truth, and the cycle of knowledge acquisition, critical evaluation, and knowledge validation that is important for the development of higher order thinking skills, is nonexistent" (Shale & Garrison, 1990, p. 29). From a learner's perspective, it is often the ability and commitment to provide immediate, detailed, and appropriate feedback that distinguishes a good online distance educator from a bad one. Given the importance of feedback, how can we generate timely and detailed feedback in a relatively quick and efficient manner? You could limit enrollments or increase the number of people who provide the feedback, but, unfortunately, few of us are afforded such luxuries. You could also group students and ask them to complete group assignments, thereby reducing the total number of papers you have to review. No matter how many students you have or how they are arranged, templates may be created to optimize the time you spend generating feedback. An effective feedback template consists of four primary components: (a) assessment criteria, (b) confirmatory feedback (c) corrective feedback, and (d) personalized messages. An example of the four components is depicted in Appendix 1. Assessment criteria are derived from the design evaluation chart (see Table 1). To demonstrate achievement of the unit's terminal objective, students are to generate criterion-referenced assessment instruments. Figure 1 lists descriptors that characterize exemplary, proficient, and unsatisfactory criterion-refer-
Atsusi Hirumi
89
FIGURE 1. Graphic Illustrating the Distinction Between a Horizontal and Vertical Prototype
Different Features
Scenario
Horizontal Prototype
Functionality
Vertical Prototype
Full System
enced assessment instruments. The assessment criteria are published to help guide student effort, to assess student work, and to provide both confirmatory and corrective feedback. Confirmatory feedback lets students know what they did correctly. Specific characteristics of student work listed in the assessment criteria may be highlighted in bold font to confirm exemplary performance. Additional comments provide further confirmatory feedback and highlight particular areas of proficient performance. Anecdotal reports from students indicate that starting with confirmatory feedback (letting students know the good aspects of their work) helps sustain their motivation. Corrective feedback identifies areas that could or should be improved and provides students with insights on how to revise their work. Again, specific criteria may be highlighted in bold, but in this case, at the proficient and unsatisfactory levels to identify areas for improvement. In many cases, additional comments may be required to provide sufficient corrective feedback. When confronted with a difficult assignment, students often make common errors. To generate detailed corrective feedback, I typically scan and select what appears to be the most inadequate work sample and grade it first. I assess the students' work and prepare a list of corrective comments. I then select what
90
DISTANCE EDUCATION: WHAT WORKS WELL
appears to be a model assignment and grade it to generate a list of positive confirmatory comments. I use both lists to create an initial feedback template that consists of the published assessment criteria and a list of confirmatory and corrective comments. The template saves me considerable time by allowing me to copy and paste appropriate comments rather than generating them each time I grade an assignment. As I grade other students' assignments, I add to the list of comments and continuously increase the comprehensiveness of the template. Another way of using data on common errors is to identify apparent flaws in design and revise materials to prevent the recurrence of errors. Creating errorfree environments, however, is not necessarily an effective e-learning strategy. Students can learn a lot from making and correcting mistakes, and the resulting cognitive dissonance may increase persistence and motivation. As a general rule of thumb, if 30% or more of the students make the same error on an assignment, I use the data to revise my instructional materials. If less than 30% make a particular error, I save my corrective feedback statements as an integral part of my feedback template for use in future classes. When using feedback templates, it is important to demonstrate to students that you are evaluating each assignment individually. If students within a class receive virtually identical feedback, they may begin to think that you are rushing through their assignment and not spending adequate time properly assessing their work. Personalized messages integrated with the confirmatory and corrective feedback statements refer to unique aspects of student work and demonstrate that you are taking the time to properly read and review each assignment. I inform students at the beginning of each course that I often use templates to assess assignments and provide feedback. I let them know that they may see similar comments if they compare feedback given to different students or teams of students on group projects. I tell them that feedback templates are the best way I've found to provide timely and detailed feedback and ask them to be certain to e-mail me if they have any questions concerning my feedback or feel that my comments are inappropriate. Student course satisfaction surveys suggest that the use of feedback templates is acceptable as long as students are informed of their use and receive detailed and appropriate feedback that includes personalized messages in a timely manner.
Establish Telecommunication Protocols
The use of interactive technologies does not ensure that meaningful interactions will occur. Telecommunications protocols that detail guidelines and performance requirements for posting bulletin board messages, chatting, and completing instructional units can increase the value of each experience and optimize time spent online.
Atsusi Hirumi
91
Electronic Bulletin Board (BBS) Protocol
When properly integrated, an electronic bulletin board system (BBS) can facilitate e-learning, allowing students to generate and access a continuous dialog on specific topics when desired. In addition, students have time to read, think about, and construct thoughtful responses to posted comments and save entire discussions that can be organized over time. Simply providing access to a BBS, however, does not ensure that meaningful interactions will occur. To optimize the value of a BBS, students must actively participate in and take responsibility for the discussions. They must have a clear purpose and the skills necessary to use the system. The discussions must also be well organized and easy to follow. Anything less and learners may perceive BBS postings as meaningless busy work. Students and instructors may become overwhelmed by a plethora of postings that take considerable time to read and interpret. A protocol that I use to facilitate BBS discussions includes thought-provoking questions, organized discussion forums, published performance criteria, and student-generated summaries. Initially, I post a description of the BBS activity that describes the purpose and rationale for participating in the discussions. The description also includes guidelines and performance criteria for posting messages, a series of questions used to initiate the discussions, and an example of what I consider to be an appropriate posting that demonstrates critical thinking. The questions are organized by topic into various discussion forums so that students are participating in one to three discussions per unit. During the first week of class, students are asked to review the BBS description and to rank the questions in priority order based on interest. I then randomly select students and assign one question based on prioritized list, to each student to facilitate discussion. The assignments are posted, along with due dates for each summary, and students are reminded to review the performance criteria before initiating the discussions (Table 5). During each instructional unit, students are to post responses to the original questions and comment on other classmates' postings as they complete assigned readings and activities. Students are again reminded to extend discussions, rather then repeat what others may have already posted. I also scan the postings, offer comments, and ask students to elaborate on particularly insightful or interesting remarks. However, I do not read every posting. The student assigned to a particular question is responsible for facilitating and managing each discussion. I focus my attention on reviewing, assessing, and responding to the summary that is to be posted no later than one week after the class has finished a particular unit. In this manner, I can save time spent online, while still correcting misconceptions and addressing any key issues that students may have missed during each discussion.
92
DISTANCE EDUCATION: WHAT WORKS WELL
TABLE 5. Performance Criteria Published for Facilitating Bulletin Board Communications
Distinguished (90-100 pts) Facilitates Thread (50 pts) 1. Selects and actively facilitates one thread by posting multiple comments and soliciting additional comments 2. Researches topic and posts related findings 3. Summarizes one thread, identifying trends and patterns, noting discrepancies, and addressing original question 4. Posts summary in appropriate location and by specified due date Actively Participates in Threads (50 pts) 1. Actively participates (posts more than two comments) in at least one thread per unit other than his or her own 2. Posts comments that contain original idea or thought, building and elaborating on (rather than repeating) other messages 3. Posts all comments during appropriate times (when the majority of the class is working on the related unit and assignment) Facilitates Thread 1. Selects and facilitates one thread 2. Summarizes one thread addressing original question 3. Posts summary in appropriate location and by specified due date Actively Participates in Threads 1. Participates (posts 1 comment) in at least one thread per unit other than his or her own 2. Posts comments that demonstrate some original and critical thinking 3. Posts comments during appropriate times (when the majority of the class is working on the related unit and assignment) Facilitates Thread 1. Fails to select, facilitate, and/or summarize one thread 2. Posts summary in inappropriate location and/or past specified due date 3. Fails to participate in at least one thread per unit Actively Participates in Threads 1. Comments fail to demonstrate original or critical thinking 2. Comments not posted during appropriate times (when the majority of the class is working on the related unit and assignment)
Proficient (80-89 pts)
Unsatisfactory (< 50 pts)
Chat Protocol
Chat is an often used mode of communication for facilitating e-learning. The problem is that chat sessions with over six participants can be very difficult to follow. You also confine one of the benefits of e-learning if you require students to be at a certain location at a specific time and date. However, there are some advantages to chat. For instance, real-time, interactive dialog may be established between multiple users and conversations may be saved for review at a later time at relatively low cost.
Atsusi Hirumi
93
Typically, I use chat for virtual "office hours." Students are informed that if they log on to chat at a specific time and date, they can chat with me about course assignments and activities. I do inform students that, if no one shows up during the initial 15 minutes of the specified start time, I will log off to avoid unnecessary time waiting to see if anyone will chime in. In my experience, three to five students participate in the optional sessions. If instruction warrants a required chat with an entire class, I've found one useful method for facilitating the session. Similar to the BBS protocol (discussed above), I begin by posting a description of the scheduled chat sessions that includes a purpose statement, netiquette for communicating during each session (Table 6), and a list of planned events and expected outcomes. At the beginning of the chat session, the entire class meets at the specified time and I quickly review the purpose, remind everyone to follow the posted netiquette, and assure that everyone understands the planned sequence of events and expected outcomes. Then I divide the class into small groups and ask each group to select a facilitator and a presenter. The small groups go into different chat rooms and the facilitator ensures that the group meets the targeted outcomes within the specified time frame. I go in and out of the various chat rooms to monitor progress and address any questions that may arise. After the small groups have finished chatting, everyone meets again in one room and I lead a discussion with the designated presenters to go over the results of each small group while others lurk and make note of any additional questions. Finally, at the end of the chat session, I do open the discussion to the entire group to address any lingering questions or comments. Depending on the purpose and relative importance of the chat session, I may ask each individual student to reflect on and post a short description of the insights gained during the
TABLE 6. Specific Netiquette for Facilitating Chat Sessions
1. When the instructor is present, key in on his or her comments and please follow his or her directions. 2. If you are directing your comment to a particular individual or responding to an individual's posting, type the person's name at the beginning of your message (e.g., John: I agree 100%, chat can be very useful if everyone adheres to specified protocols). 3. Keep messages fairly short. If you have a relatively long message, type ". . ." at the end of a few lines. That means that more is to come. Please do not post any new messages. If you see ". . ." typed at the end of message, hold your comments until the author has finished his or her comment. 4. Use abbreviations and emoticons whenever possible. 5. Do not worry about spelling. Everyone is typing as fast as he or she can during chat sessions. Note. A link is provided to additional information and examples of emoticons.
94
DISTANCE EDUCATION: WHAT WORKS WELL
chat session. Limiting most of the chat session to five to six active discussants, plus applying the prescribed netiquette and asking learners to reflect on the session can increase the educational value of the chat session while decreasing time spent online.
Unit/Lesson Protocol
Students can become confused and experience difficulties regulating their learning if requirements for completing a lesson or unit are located in different places. Online courses are typically made up of several components. Information and directions are often presented in a syllabus, course documents, assignments, calendar, e-mail, and bulletin board postings. As a result, students may not be able to readily discern the requirements necessary to successfully complete instruction or may be presented with contradictory directions or criteria if the instructor does not take the time to assure that everything is consistent. Instructors, in turn, may have to spend considerable time explaining procedures or remediating instruction that may have been missed or misunderstood. Revising and updating courses may also take considerable time when requirements are located in different areas and documents. An effective tactic for organizing unit or lesson requirements is to generate an overview page. Appendix 2 illustrates the components of a sample unit overview page. Note that all assignments and activities necessary to complete the instructional unit are either presented or provided as a link from the overview page. In this manner, students can go to one location and get a good idea of what they must do to manage their time. Students also know that if they finish each of the events listed in the overview, they will have completed the unit.
Generate Prototypes to Formatively Evaluate Materials
Up to this point, I've discussed tactics for optimizing the time educators spend online facilitating the e-learning process. Each of the aforementioned tactics, however, may require you to spend additional time preparing e-learning materials prior to delivery. The final tactic focuses on decreasing the amount of time spent designing and developing materials. Educators can spend a lot of time developing e-learning materials, only to find that significant revisions are necessary after initial implementation. Basic changes in navigation, structure, page layout, font styles, and instructional design may require considerable time and resources if they have to be applied to an entire course. Rather than developing all of the instructional units associated with a course, a more efficient method for creating e-learning materials is to first generate, test, and revise a horizontal and vertical prototype. The revised prototypes may then be used as a template for building the rest of your course.
Atsusi Hirumi
95
A horizontal prototype presents students with a wide range of program features that are not yet fully functional. In contrast, vertical prototypes present students with relatively few program features that are fully functional (Figure 1). For an online course, a horizontal prototype may consist of a course home page that provides access to most or all course features (e.g., syllabus, instructional units, calendar, assignments, telecommunications tools) but each feature may not be entirely operational. A vertical prototype may consist of one fully functional instructional unit, including objectives, content information, graphics, assignments, activities, tools, and assessments. With a vertical and horizontal prototype, you can test most key features of the user interface and the basic instructional design before going into full-scale production, preventing the need for costly revisions after significant portions of the e-learning program have been programmed. The vertical prototype may then be used as a template for all other instructional units as well as a basis for establishing standards for navigation, font size, font style, the use of headings, placement, and size of graphics.
WHAT'S THE PAY OFF?
The tactics discussed in this article require time and expertisevital resources that may be spent on other important tasks. It takes time and expertise to analyze an instructional situation; generate, cluster, and sequence objectives; prepare and design evaluation charts; and align objectives, assessments, and instructional events. It takes time and expertise to analyze and balance interactions, create feedback templates, establish telecommunications protocols, and generate and test horizontal and vertical prototypes. To further compl...