Online Course Research

This past week, we read a series of papers on analyzing the effectiveness of online courses. Sarah Haavind sums it up best, "Rarely is it possible to compare circumstances where variables such as class size, student background, curriculum and even the level of enthusiasm of the instructor are effectively controlled." The individual studies themselves are not irrefutable. Studies of classes are sociological in nature and the research methods are not purely scientific. This leaves the conclusions open to interpretation and makes the results from different studies difficult to compare. These issues are especially pronounced in meta-analysis (studies of other studies). I would rather see the research focus on identifying students that will excel in online courses and how to make online courses more effective.

Issues raised in the studies:

• it is difficult and time-consuming to codify and quantify comments from discussion boards.
• differences between instructor experience, training, and teaching approaches (e.g., "craft-oriented practices" (Saba, 2001)
  
• variations among student achievement and characteristics
• normal "ebb and flow" class variations
• "Courses are participatory experiences shaped by the learners themselves, making each individual offering a unique entity" (Sener - pg 1). Examples include media attribute theory, social presence theory, content analysis, systems dynamics and discourse analysis
  
• studies include different subjects
  
• varying student demographic data and prior knowledge
• varying student technical skills
• difficult and impractical to assign learners randomly to classes for studies (Sener - pg 1)
  
• varying student and teacher knowledge of collaboration techniques
• varying teacher knowledge of online moderation techniques
• varying perceptions about online and f2f learning
  
• varying assessments. The Rice report (pg 432) cites Bernard et al. (2004) who found that studies that involved researcher-made tests favored distance learning over face-to-face, while studies using teacher-made tests favored face-to-face classrooms over distance learning.
• "various alternative approaches to evaluating online learning within its own frame of reference" (Sener - pg 3)
• studies use different technology tools and online pedagogies
• the technology tools and pedagogy may shift over time and between studies
  
• varying degrees of learner support
  
• what was the social environment like for students (this may not be visible by the teacher)
• varying degrees of course material quality