Interactions

Research from the distant past regains relevance! From 1998 to 2000 and 2017 to 2020, I focused on higher education. Suddenly, some of the early work is again relevant. I’ll describe it here, along with some thoughts arising from conversations with educators, administrators, and my daughters.

Forced by Covid-19 into functioning remotely, the education field quite sensibly seized upon familiar and reliable “digital iron horse” substitutions. Zoom was widely adopted for class meetings in primary and secondary schools that had access to technology, and more broadly for university lectures and final exams in TA-monitored breakout rooms. The limitations of the webcam wave, however, soon steered attention to asynchronous approaches. Mark Guzdial reviewed their uses for remote learning but said that he hopes not to need them again [1].

Twenty years ago, a group I was part of at Microsoft Research deployed streaming-media prototypes to explore real-time and asynchronous remote education. Students and faculty had little time to prepare. Our systems were used in lecture courses at the University of Washington and MIT, and in multisession internal Microsoft training courses, with the results published in CHI, CSCW, and beyond. This was pre-wireless, running on PCs less powerful than your phone. The human side, though, has not changed as much.

Remote live lectures and audience feedback

Studio audiences were devised for early radio comedies and dramas because performers feed off audience energy (studio audiences were also used to record laugh tracks). In March, late-night television hosts had to adjust to the disappearance of live studio audiences. Classroom instructors faced the same challenge.

In “Evolving Use of a System for Education at a Distance” [2], my colleagues and I describe Flatland, a system to explore numerous tools for student feedback. It included video of the presenter, slides including interactive forms, student questions that they could vote to prioritize, a chat panel, Too Fast/Too Slow and Clear/Confusing buttons, hand-raising functionality, and a list of attendees (Figure 1). We reported on feature use and non-use, but high-level observations are more significant today.

Figure 1. Flatland, presenter view. A reminder of what interfaces looked like in 1999!

Here are some key findings:

• Teaching style matters. The most effective classroom lecturers can have difficulty online. One great instructor’s classroom approach was to race through material and monitor students closely to see where to slow down. Unfortunately, students did not use the Too Fast button. With no feedback telling him when to brake, he finished lectures in half the allotted time. He knew that few students had kept up but could not adjust. Less extroverted lecturers who prepare methodically can fare better.


• Video is great for connecting but not for feedback. Flatland did not have student webcams. Today, seeing everyone at the beginning of a class can be wonderful, although once class starts, video must be managed carefully to avoid distraction. Surprisingly, video is not good for passive feedback. In a separate controlled study, four-person groups carried out engaging tasks either face to face or connected by high-resolution video. Task performance was similar, but when connected by video, people’s faces were far less expressive. Deadpan images won’t help instructors gauge student reactions.


• Establish social conventions for student feedback and interaction. Feedback tools have a learning curve to reach consistent use. For example, an instructor who uses the chat channel to greet early class arrivals creates an often unmet expectation that chat will be watched during the lecture. Will a hand icon be used in voting initiated by the instructor, or initiated by students to signal a desire to comment or speak? Can students verbally jump in? Appropriate social conventions must be designed and communicated.

Recorded lectures and flipped classes

We also built asynchronous video systems to support a “flipped classroom” approach, in which students watch a lecture before the class meets and spend class time discussing or working with the material presented. We went further, enabling students to interact before the class (Figure 2). A student watching a prerecorded lecture and slide presentation could contribute to a time-indexed discussion: Questions and comments by previous viewers scroll by in sync with the lecture. Versions of the Microsoft Research Annotation System were used at MIT and the University of Washington, and in internal training courses [3].

Figure 2. Viewers could access the table of contents, discussion, or private notes (lower left). The topic of the comment by a previous viewer that is closest to the current point in the video replay is highlighted, with the full comment text below the slide.

Some key findings from this project:

• On-demand viewing with pre-class discussion created work for faculty. Instructors had to record lectures without an audience (though today some instructors reuse last year’s lectures without raising eyebrows). When the annotation system succeeded in fostering extensive discussion, faculty had to read it and could struggle to use the subsequent class period effectively: The discussion was over. One professor complained that to use it again, he would have to design different uses for class time.

Students generally liked having discussion and personal notes indexed to the video. We also devised advanced approaches for varying playback speed (faster or slower) while maintaining comprehensibility. Faster playback could increase learning by focusing attention. Other advantages: Students can choose when to watch a lecture, and with remote classes they save commuting and walking time.

• Procrastination. Prerecorded lectures benefit students who watch the video but adversely impact those who do not and thus can’t follow the class discussion. This effect was amplified by our Discussion feature, which also penalized the many who watched the lecture at the last hour, leaving no time to participate in the online discussion. To address this, we modified the system to support the insertion of group tasks [4] and exercises/quizzes [5] during playback, which could be assigned for completion days before the class. Most students preferred questions that popped up unannounced during the lecture.


• One size does not fit all. A crucial discovery was that each instructor wanted custom features based on their course content and use. Our out-of-the-box system was not flexible enough. An MIT film professor needed two video windows. We were asked to create an annotation playlist capability to group comments together. Terminology was tripped over: After a programming language course had minimal Discussion activity, a student explained, “I had questions, but I didn’t want to discuss C.”


• Shorter lectures? Long lectures are motivated by the cost of converging an instructor and students in a room, but long lectures are usually less appealing and effective than short lectures. Remote education makes the breaking of long lectures into segments more practical.

Next steps

Here I described video-based systems that most students (but few faculty) said they would sign up for again. Having to respond to Covid-19 is terrible, but it is also an opportunity to reflect on pedagogy and technology. From chalkboard to overhead projector to projected slides—what’s next? A visionary lecture system [6] built by David Kellermann at the University of New South Wales creates a learning community in large courses that include remote students. This crisis is also forcing a close look at assessment. Can we assure integrity in high-stakes remote exams without intrusive video systems? Should we move more rapidly to other means of achieving and measuring learning outcomes?

Education is more than learning. Engagement and motivation, key concerns today, are socially constructed. Whether motivated by competition or collaboration with peers, by discussions between classes, or by a smile from a teacher, students lose such incentives and drivers when the school bubble is replaced by the bubble of home. Seeing my daughters’ athletic awards and graduation ceremonies via video rather than a banquet or assembly is a reminder of the creative challenge in designing effective technology to supplement the natural real-time and face-to-face interactions that people have relied on for millions of years. 

Endnotes

1. Guzdial, M. How I’m lecturing during emergency remote teaching. Blog post, Apr. 6, 2020. 

2. White, S.A., Gupta. A., Grudin, J., Chesley, H., Kimberly, G., and Sanocki, E. Evolving use of a system for education at a distance. Proc. of HICSS 2000. https://doi.org/10.1109/HICSS.2000.926729 

3. Bargeron, D. and Grudin, J. As users grow more savvy: Experiences with a multimedia annotation tool. Proc. of HICSS 2004. https://doi.org/10.1109/HICSS.2004.1265260 

4. LeeTiernan, S. and Grudin, J. Fostering engagement in asynchronous learning through collaborative multimedia annotation. Proc. INTERACT 2001, 472–479. http://www.jonathangrudin.com/wp-content/uploads/2017/03/MRASengagement.pdf

5. LeeTiernan, S. and Grudin, J. Supporting engagement in asynchronous education. CHI '03 Extended Abstracts on Human Factors in Computing Systems. 2003, 888–889. https://doi.org/10.1145/765891.766051 

6. Kellermann, D. Transforming the learning experience (video). Campus Connection Summit 2020.

Jonathan Grudin