Without sufficient resources, students can be overwhelmed by this broad range of topics. With sufficient resources and an interface to tame the potential flood of resources, they can better comprehend the class topics. Resources that benefit students include
This paper describes the design and creation of such a web of resources "on the fly," while the course is being taught. The design and creation of this web touches on many issues, including mechanisms for convincing students to use the web; the balance between online hypertext and printed documents; problems and benefits of live update (the collection of resources changes frequently, as each class period generates new resources); the roles of audio and video; and, most importantly, student reactions.
During the summer of 1994, I used Mosaic and the World-Wide-Web to provide a variety of resources for an introductory survey course in Computer Science. Because of time restrictions, much of the development, design, and creation of these resources had to be done "on the fly," as the course was being taught. This paper presents my experiences in the design and implementation of an interactive networked support system for that class and touches on the things I learned by preparing the system, what students thought about electronic resources, and what I will change in the future. Although this paper highlights experiences with a particular class, it is likely that the methodology and system used in this class can apply to a variety of classes and educational situations.
As computers continue to play an increasing role in academia, business, and everyday life, more and more students are choosing to take an introductory course in Computer Science. Students are attracted to the course because they believe it is useful, interesting, or necessary to know more about computing. These students come from a variety of backgrounds ranging from art history and humanities to physical sciences and engineering. Therefore, they bring a wide range of experiences, needs, expectations, and learning styles to the class. Both the variety of students and advances in the field of Computer Science suggest a new type of introductory course which surveys the field of computer science (e.g., [Bie90,Bro94,DH94,SG94]) rather than teaching only programming or computer applications. This new introduction may cover algorithm design, analysis, and implementation; programming; human-computer interaction; digital-circuit design and computer architecture; artificial intelligence; networking design, protocols and applications; history and social issues; and a wide variety of other topics. This new course better serves the needs of a broader variety of students (particularly nonscientists, who should learn about computing but need not learn a significant amount of programming). Such a course has a real place in a liberal arts education as it teaches a new (and, to many of these students, very different) way of thinking, rather than teaching technical skills.
Because the course presents a broad variety of topics to many different types of learners, it must accommodate the different learning styles that both the topics and students require. Personal experience shows that some students learn best from reading the book, some from class lectures, some from experimenting using to a fixed experimentation plan, some from experimenting freely, some from talking to other students in the class, some from teaching topics to other students in the class, and some from still other methods. And learning varies from topic to topic. For example, McKeachie [McK86] suggests over twenty different teaching mechanisms and corresponding benefits, situations, and learning styles.
To accommodate these needs, the designer (instructor) of a course should attempt to provide a variety of learning resources. However, care must be taken to emphasize and rank resources so that students are not overwhelmed by too many resources. The World-Wide-Web (WWW) provides an excellent platform for developing, organizing, and disseminating a wide variety of resources, including class notes and outlines, longer textual resources that resemble traditional textbooks, interactive nonlinear tutorials, student questions and comments, and even simulations of individual class sessions. The WWW also allows instructors to prioritize resources and students to reorganize the resources in the way that fits them best.
This is not to say that a WWW-based web of resources should be used instead of lecture, discussion, laboratory, and homework; the web-based resources are intended to supplement and not replace traditional learning methods. Nor is it to say that students should be presented with an unorganized web; as [LR91] suggests, while hypertext benefits some students, other students learn best according to fixed, linear, lesson plans.
In addition to their role in providing resources for the class, Mosaic, the WWW, and HTML (the markup language used for creating WWW documents) provide valuable topics for discussion in an introductory Computer Science class. For example: students can study hypertext by writing documents in HTML; think about user interfaces while evaluating the interface to Mosaic; comment on, evaluate, and discuss networked hypertext documents (e.g., the electronic version of Bruce Sterling's The Hacker Crackdown [Ste92]); and discuss the implications of the networked resources available through the WWW. HTML can also serve as the basis for discussion of logical formatting, of the need to make documents both human- and computer-readable, and of computer-based translation and compilation.
I am not alone in attempting to make class resources available through the WWW. The University of Texas at Austin maintains an extensive guide to a variety of web-based class resources [UTA94]. This collection shows that classes and instructors use the web in many different ways. Some classes provide just a few notes or an introductory syllabus. Some use the web as a dissemination mechanism for student essays. Some use the web to personalize the class, providing student pictures and home pages as part of the web of resources.
I am also not alone in incorporating HTML authoring into the course syllabus. Many of the courses that provide web-based resources also require students to author a home page or to use the web to search for other resources. [Weg94] details an introductory course that incorporated HTML and Mosaic and allowed students to create HTML-based final projects.
The course met three times weekly for a 65 minute session which consisted of both lecture and discussion. Students also attended a weekly one-hour lab. Students were evaluated according to their work on two open-book exams, a variety of long (8-15 hour) and short (10 minute-1 hour) homework assignments, a final project, a set of notes on a topic chosen by the student, and class participation. The class included an optional weekly question-and-answer session conducted by the instructor and 9-12 hours of informal tutorials with undergraduate tutors. Students were encouraged to ask questions of the instructor and tutors via electronic mail and in person.
Although all sessions of the class were audiotaped and videotaped, I decided not to make the video and audio available, either on tape or in digital format. In part, this decision was motivated by my belief that students should attend class because they benefit from interaction in class. In part, this was because the resources were not available to distribute the audio and video electronically. In part, this was because educational research suggests that plain video does not provide significant learning benefits [Lev91, MSM92] and there was not time to put these resources into a comprehensive interface, such as the proceedings interface described in [Reb+95].
Other resources provided for the students included a textbook [SG94], sample programs and homework solutions from past classes (created by both students and instructors) that students could use in developing solutions to related homeworks, a collection of animated sorting and searching algorithms, appropriate photocopies of key articles, and software related to the course (e.g., Microsoft Excel and a circuit design program).
Experience shows that students print many of the text-based resources they use [Mook94,Win94]. Hence documents should be designed for printing and reading on paper as well as for onscreen viewing. For this course, I tended to err towards designing documents for paper viewing. However, some principles of design that apply to electronic documents also improve the readability of printed copies. For example, I tried to preface every document with a simple table of contents that contained links to the individual sections (and, similarly, a "return to table of contents" at the end of each section). While the electronic purpose of such a table of contents is navigational, research shows that such preliminary information improves students' comprehension of textual materials [Kru+89]. In addition, onscreen marks, such as horizontal rules, also help with reading printed versions of the text and provided visual cues for major breaks in handouts (e.g., the switch to a new topic in a class outline).
Because it was important for students to know when documents changed, I included the change date at the top of each document (after the title). This allows students to immediately determine whether a document has changed. Finally, because students were reading some documents on paper and some onscreen, the links to external documents were also described explicitly (as in the list of references for this paper).
At the end of the term, students were given a survey to determine their reactions to both the electronic resources for the course and to the various course topics [Reb93b]. This survey included much more specific questions than traditional course evaluation forms and asked the students to comment on many of the components of the web and segments of the course. Although it would have been preferable to conduct this survey using the Mosaic forms feature, most students were still using NCSA Mosaic for the Mac version 1.0.3 [NCSA93], which does not include this feature. In addition, many students prefer to write, rather than type, answers to questions. Hence, while the survey was prepared as an HTML document, it was presented to the students on paper, and most of the students returned it using paper (one student used an electronic version that he modified by hand). The response rate for this survey was high; over 90% (23 of 25) of the students in the class filled out survey forms.
As expected, students said they were not able to use all the resources or features provided for them. Although a goal of the design of the web was to provide documents that students could search and use "on demand," most students worked in more of a "print and read" mode. Many students indicated that they would prefer to have the resources chosen for them.
With so much time available, it is essential to limit the resources we will use or we will never finish any assignment. At times, I felt guilty for not conscientiously printing out transcripts, blackboards, etc., etc. However, to use all the resources would lead to being at your computer 24 hours a day.Others have also found that students prefer to print documents [Mook94, Win94] and, given this predilection, it seems best to provide more important documents to students in printed form as this helps the students prioritize class resources.
Although I had some concern that students would use the online resources as an excuse to miss class, only two students made a regular habit of missing class. Other students also missed the occasional class. However, the number of attendees was not out of keeping with normal attendance patterns. For those who missed the occasional class, the online resources provided a backup. While most students used only outlines for review of missed classes, at least one student (other than the two who regularly missed class) used class transcriptions for reviewing the occasional missed class.
In spite of the problems mentioned above, it was useful to provide this web of resources. Different students did, in fact, find different resources useful and the students who took the time to use the resources electronically commented that electronic access was particularly helpful. I also found the creation of the web stimulated my thinking and forced me to reevaluate many components of the course.
The following sections present extended discussion of the motivation for, design of, reactions to, and recommendations for future use of individual class components.
I also used the outlines to present administrative information (upcoming homework due dates, lab sessions, revisions to homework assignments). These pieces of "administrivia" were included in the outlines so that students could quickly determine changes to the schedule and remind themselves of deadlines. These outlines were an extremely successful component of the class web. According to the survey, all the students used the outlines in at least one of the two forms (printed or HTML). Students did not object to the lack of adherence to class outlines (although three did comment that it was an occasional problem).
[The outlines were] good for looking through later and getting the basic points and as a reference during class. Were also very accurate--if had been more they would have been forced. The professor allowed class to move naturally in digressing, but still stuck to basic points.
Although the outlines were useful for both students and instructors, they were among the most time-consuming resources to prepare. Some of this problem may be alleviated when they are reused in future classes, although reusing outlines eliminates their benefit of forcing the professor to rethink the organization, content, and presentation of topics. Another problem with the outlines was that they discouraged students from using the book, and gave the students a somewhat less positive view of the book.
These electronic blackboards provided somewhat less writing space than a traditional blackboard and were harder to draw on than a traditional blackboard (at least for this instructor). However, I find it much faster to type than it is to write, and my typing is generally much more readable than my reading. Additionally, the electronic blackboard provides a better history of the class which could be used both during class (e.g., "as we saw twenty minutes ago É") and as a review after class. The use of blackboards for history and review were the primary motivations for using electronic blackboards. The availability of the electronic blackboards also allowed students to pay closer attention to class and less time scribbling down notes.
[I]t was nice to know that we didn't have to spend the lecture hour frantically writing notes, since it's all [available online.]
In addition to providing the original HyperCard-based blackboards, I also provided the content of the blackboards in HTML format. In designing the HTML version of the blackboards, I attempted to strike a balance between mimicking the original format and accommodating the uses and limitations of HTML. Each pair of blackboards was treated as one "page" (separated from other pages by a horizontal rule) and annotated with the times I started writing on the blackboard and stopped writing the blackboard. The blackboards had no table of contents, although they did include links to related documents (in particular, the class outline and transcription). In general, I found the HTML versions of the blackboards easier to read, print, and understand.
Students were generally satisfied with the electronic blackboards, and all said they used the electronic blackboards (even though I did not mention their availability after the first week of class). Although the HTML format appeared to provide better printouts and was often better edited than the HyperCard version, the majority of students used only the HyperCard stacks. I would assume that they did not use the HTML versions of the blackboards because they did not know they existed or because they wanted to see the blackboards in the same form that they saw them onscreen.
Because of the limitations of the computer screen, and my decision to make four blackboards visible at any one time, I could only fit a limited amount of text on one blackboard. Many students criticized the design of the stacks, in part because of the limited text that would fit on screen, in part because the blackboard stacks did not fit on the smaller student monitors. It may be that a less sophisticated interface (e.g., using a word-processor for the blackboard) would suffice.
Student comments suggested that the electronic blackboards were only useful as references for those who had attended class.
The blackboards were good review for a class that I had been to, but not one that I'd missed--I don't think I would change that.Students also used the electronic blackboards as references when studying for exams (and during the open-book exams). I observed many students with stacks of paper containing the text of the blackboards.
There were further drawbacks to the blackboards. Because I was typing, rather than writing, I tended to write more than I would have otherwise. Students noticed this and suggested that I might edit them after class and add a table of contents or links within the blackboards.
The blackboards generally served the purpose of traditional blackboards -- providing students with something to refer to during class that gave a close history of the class. One student observed that the electronic blackboards included what they felt was extraneous information but acknowledge that such information also appears frequently on traditional blackboards,
A little less of the class comments etc. that were not useful. [...] Then again, if you use like a regular blackboard and still take notes, they are helpful.
Many students felt that the outlines and blackboards worked well together and obviated the need for other resources.
I didn't use the transcriptions because most of the information was already in the class outlines and the blackboards.
In general, students prepared useful and interesting summaries, although the occasional summary was wildly inaccurate and required significant revision. Because I had not provided an HTML standard for them to use, their documents had inconsistent formats and required some time to put into a reasonably consistent form. However, most attempted to mimic the form I used for most class documents, with a table of contents at the beginning and a link to the course home page at the end.
Although students were told about this assignment on the first day of class, almost all the students waited until the last day of class to turn in their notes (and some asked for extensions beyond that date). Hence, the resources were not used for this session of the course, but will be used in the next session.
However, the transcriptions did provide a separate benefit: they gave me the opportunity to think about how I presented various topics and how to incorporate descriptions of actions into textual commentary. While I don't plan to use transcriptions again, I do plan to experiment with these transcriptions to see whether they can be put to other uses (e.g., generating better class notes for the next class session).
For example, in an experiment to construct a circuit for a four-bit adder, the document describes what a four-bit adder does and reviews binary notation and then asks the student
I used one of these structured experiments in class, with students discussing each step with each other, assigned one as a take-home exercise, asked the students to write their own exercise, and left some as optional exercises that students could follow if they felt they needed more information or experience on a particular topic. Students generally found the experiences useful.
[B]y using the examples of comparators and MUX's I was able to develop a good knowledge of [circuitry].These interactive examples could be improved with links to answers to the individual steps of the example, but time limitations prevented such comprehensive answers (in part because many of them required graphics). In future sessions of the course, I expect to incorporate more of these interactive experiments.
Because the class was relatively small, I was able to act as the primary resource for answering many questions, The tutors for the class occasionally sent me collections of questions that they had been asked (most often the questions that they were unable to answer). I had also hoped to use the class transcriptions to provide additional questions and answers (particularly from special tutorial question and answer sessions held every week), but was unable to present these questions and answers in a timely fashion. However, the electronic blackboards from those sessions did provide a form of question and answer repository which was most useful for students who attended the question and answer sessions.
After collecting questions on a topic or assignment, I regularly summarized questions and answers and provided them to students in three ways:
Providing a collection of questions and answers proved quite successful. However, it is not clear that providing them through the web was necessary. Only five students reported that they used the listing of questions, and only three reported that they used the search feature (one who used it suggested that he could usually tell which documents were of interest, so it was unnecessary). Nonetheless, all students reported that they appreciated receiving a regular collection of questions and answers. Two comments summarize the general reactions and suggest that expectations for questions and answers were largely met.
The "recent questions and their answers" from other students really helped. Often you have a similar question, but don't feel like asking 10,000 questions.
I liked them; sometimes I found the answers to questions in the [mail messages] which saved me from having to ask them.
I plan to continue this policy of recording and distributing questions and answers (which I have also used in the past, although somewhat less formally). I expect to maintain all three distribution mechanisms, as I expect that some students can be better served by indices and WAIS. For others considering this idea, I would suggest appointing a tutor to record in-class questions and answers and to put them in HTML format immediately after each class.
It is likely that the majority of students would have been just as well served by only printed documents with the occasional e-mailed list of questions and answers. However, there were a few students in the class who made ample use of the web of class resources and clearly benefited from hypertext access to these resources. Such students suggest that although a web of resources may not benefit the typical student, they are valuable in supporting a wide variety of learners. Additionally, the electronic blackboards seemed to provide some use to many students, although I would guess that students used them in quite different ways.
Of course, there are other reasons for creating a web of resources for a course. As suggested earlier, creating the documents and the links between the documents forces one to rethink topics and their relationships. This reevaluation can be quite useful. In addition, by creating such a web of resources, an instructor can provide information and ideas for others teaching the class (either locally or at another institution) and provide self-study resources for students interested in learning topics outside of class.
For those interested in pursuing a similar developing a similar system, I would recommend restricting the electronic documents to traditional, class outlines, and questions and answers. If resources are available for using and showing electronic blackboards (whether special software or word-processors), they provide a useful resource that students appreciate and that helps both students who like to take notes and students who do not like to take notes. However, I would recommend evaluating the blackboards after each class to correct errors and to evaluate whether too much or too little is being written on the boards.
Now that these resources have been collected, there are many possibilities for reusing and modifying them. One future direction is the use of these materials in a coherent class simulation that includes class outline, transcriptions, blackboards, audio, video, and even interactive experiments. Such an interface might be an extension of an interactive talk from an electronic conference proceedings (e.g., [GMM93, Reb+95]) or might take a separate direction. It is also likely that these resources can provide additional help for select students in another instructor's version of the course. Both will be investigated in the coming months.
This work was partially supported by the Dartmouth College Computing Venture fund and the Dartmouth College Experimental Visualization Laboratory.
[Bie90] A. W. Biermann. Great Ideas in Computer Science: A Gentle Introduction. MIT Press: Boston, MA, 1990.
[B-L94] T. Berners-Lee. "Style guide for online hypertext." CERN - European Particle Physics Laboratory. URL http://info.cern.ch/hypertext/WWW/Provider/Style/Overview.html, 1994.
[Bro94] J. G. Brookshear. Computer Science: An Overview, Fourth Edition. The Benjamin/Cummings Publishing Company: Redwood City, CA, 1994.
[Buf94] J. F. K. Buford (contributing editor). Multimedia Systems. ACM Press: New York, NY, 1994.
[Bush45] Vannevar Bush. "As we may think." Atlantic Monthly, 1945.
[Cla90] Claris Corporation. HyperCard, version 2.1, 1990.
[Coh94] J. K. Cohen. "Elements of HTML Style." University of California at Irvine, UCI Bookstore. URL http://bookweb.cwis.uci.edu:8042/Staff/StyleGuide.html.
[DH94] R. Decker and S. Hirshfield. The Analytical Engine, An Introduction to Computer Science Using HyperCard 2.1, 2nd Edition. PWS Publishing Company: Boston, MA, 1994.
[GMM93] P. A. Gloor, F. Makedon, and J. W. Matthews (editors). Parallel Computation: Practical Implementation of Algorithms and Machines. Telos/Springer-Verlag, 1993.
[Kru+89] D. Krug, B. George, S. A. Hannon, and J. A. Glover. "The effect of outlines and headings on readers' recall of text." Contemporary Educational Psychology 14, pp. 111-123, 1989.
[Lev91] S. R. Levin. "The effects of interactive video enhanced earthquake lessons on achievement of seventh grade earth science students." Journal of Computer-Based Instruction 18(4), pp. 125-129, 1991.
[LR91] A. Lanza and T. Roselli. "Effects of the hypertextual approach versus the structured approach on students' achievement." Journal of Computer-Based Instruction 18(2), pp. 48-50, 1991.
[Maz93] E. Mazur. Peer Instruction: A User's Manual. Harvard University Physics Department: Boston, MA, 1993.
[McK86] W. J. McKeachie. Teaching Tips: A Guidebook for the Beginning College Teacher. D. C. Heath and Company: Lexington, MA, 1986.
[MSM92] Z. Mevarech, N. Shir, and N. Movshovitz-Hadar. "Is more always better? The separate and combined effects of a computer and video programme on mathematics learning." British Journal of Educational Psychology 62, pp. 106-116, 1992.
[Mook94] D. Mook. Teaching introductory physics. WISP Lecture, May 1994, Dartmouth College.
[OT94] T. Ottmann and I. Tomek (editors). Proceedings of Ed-Media '94: World Conference on Educational Multimedia and HyperMedia. Vancouver, BC, Canada; June 25-30, 1994.
[NCSA93] National Center for Supercomputing Applications. NCSA Mosaic for the Macintosh, version 1.0.3, 1993.
[Reb94a] S. A. Rebelsky. Online resources for CS4 94X-Concepts in Computing. Dartmouth College Department of Computer Science, URL http://www.cs.dartmouth.edu/~samr/CS4.94X/home.html, 1994.
[Reb94b] S. A. Rebelsky. "Summary report on CS4 94X: Course Design and Student Reactions." Dartmouth College, Department of Computer Science, URL http://www.cs.dartmouth.edu/~samr/CS4.94X/summary.html, 1994
[Reb94c] S. A. Rebelsky (editor). "Student notes on topics in introductory computer science." Dartmouth College, Department of Computer Science, URL http://www.cs.dartmouth.edu/~samr/CS4.94X/ Notes/index.html, 1994.
[Reb+95] S. A. Rebelsky, F. Makedon, J. Matthews, C. Owen, L. Bright, K. Harker, N. Toth, and P. T. Metaxas. "Building multimedia proceedings: the roles of video in interactive electronic conference proceedings." Submitted to ACM Transactions on Information Systems special issue on Digital Video in Multimedia Systems, 1995.
[SG94] G. M. Schneider and J. Gersting. An Invitation to Computer Science. West Educational Publishing: St. Paul, MN, 1994.
[Ste92] B. Sterling (translated to HTML by B. O'Sullivan). The Hacker Crackdown: Law and Disorder on the Electronic Frontier. URL http://www.scrg.cs.tcd.ie/scrg/u/bos/hacker/hacker.html. Printed version published by Bantam Books, 1992.
[UTA94] University of Texas at Austin. "WWW for instructional use." University of Texas at Austin, General Libraries and Computation Center, URL http://wwwhost.cc.utexas.edu/world/instruction/index.html. 1994.
[Weg94] P. Wegner. "Computer Literacy for Undergraduates." Brown University Technical Report CS-93-21, June 1994. URL ftp://wilma.cs.brown.edu/pub/techreports/94/cs94-21.ps.Z.
[Win94] P. J. Windley. "Using WWW to augment classroom instruction." Laboratory for Applied Logic, Brigham Young University, URL http://lal.cs.byu.edu/people/windley/using.www.to.teach.html, 1994.
samr@cs.dartmouth.edu.