Dr. Ryan Baker has taught massive open online courses for as many as 50,000 students—but the insights he shares here can benefit any online educator.
Associate Professor and Director of the Penn Center for Learning Analytics, University of Pennsylvania in Philadelphia
PhD and MS in Human-Computer Interaction, BS in Computer Science
How does it feel to teach an online course with thousands of students? Many of today’s professors may find themselves tasked with teaching one of these MOOCs, or massive open online courses—perhaps without much warning. That was what happened to Dr. Ryan Baker when he launched the course Big Data and Education several years ago, while at Teachers College, Columbia University. “I never would have imagined in a million years that 50,000 people would show up,” he says. “But I think it hit at just the right moment, when MOOCs were coming into awareness.”
Did he find that ironic? “It’s a bit funny that I taught a class on Big Data to a group of students that enabled me to collect Big Data,” says Baker. “But it’s not ironic. It was one of the goals. A reason I wanted to teach the course was indeed because it was a great way to do research. The other goal was to raise awareness for educators and educational developers. But I expected maybe 300 students!”
Today, Baker continues to expand upon this research as the director for the Penn Center for Learning Analytics at the University of Pennsylvania, where he is currently working on developing automated detectors that examine students’ interactions within educational software and then make inferences about how students think, behave, and learn—and just how motivated and engaged they truly are. He also researches students’ use of educational games, intelligent tutors, and other kinds of educational software, using that information, too, to improve our understanding of how students learn best when online.
“I’m fascinated about what we can know about learners from their behavior [on educational software]—their emotions, motivation, and a broad spectrum about each person.”— Ryan Baker, PhD
Course: Big Data and Education
Description: Online and software-based learning tools have been used increasingly in education. This movement has resulted in an explosion of data, which can now be used to improve educational effectiveness and support basic research on learning. In this course, you will learn how and when to use key methods for educational data mining and learning analytics on this data. You will examine the methods being developed by researchers in the educational data mining, learning analytics, learning-at-scale, student modeling, and artificial intelligence communities. You’ll also gain experience with standard data mining methods frequently applied to educational data. You will learn how to apply these methods and when to apply them, as well as their strengths and weaknesses for different applications.
See resources shared by Ryan Baker, PhDSee materials
Baker’s guide to making MOOCs more engaging
Course Hero recently caught up with this MOOC master to learn more about his groundbreaking work—and to gather his advice for those overseeing MOOCs of their own, whether it is for the first or fortieth time.
Course Hero: What got you interested in learning analytics?
Dr. Ryan Baker: When I was a graduate student at Carnegie Mellon in 2003, I had been getting up early for several weeks to observe middle-school math students. One day, it’d been snowing, and I was tired of getting up so early. I thought to myself, “There’s got to be a better way of doing this research so it’s not so laborious.”
Other researchers were using online data to study a variety of topics, and I’d heard about machine learning—the art of using data to have a machine figure out human reasoning. That’s when my focus turned to machine learning.
I built the first model that could tell whether a student was genuinely trying to get the correct answer during problem-solving. I think it was the first model that could discern whether or not a student who was using a computer to learn was truly engaged.
MOOCs are one area of my research, but I’ve done even more in online homework systems, intelligent tutors, and games. I’m fascinated about what we can know about learners from their behavior—their emotions, motivation, and a broad spectrum about each person.
What surprised you most when you first began teaching MOOCs?
Early on, I noticed that one guy seemed to love writing inflammatory things, and I had to engage in those online discussions to deal with the issue and to get students back on track. It’s not as big a problem today as it used to be. I am not sure why. It might be that being rude on the Internet isn’t such a novelty anymore.
Baker’s Reading List of His MOOC Research
Interested in learning more? Here, Baker offers five of his resources worth clicking on now:
- Gardner, J., Y. Yang, R. Baker, and C. Brooks. “Modeling and Experimental Design for MOOC Dropout Prediction: A Replication Perspective.” Proceedings of the 13th International Conference on Educational Data Mining, 2019.
- Aleven, Vincent, and Ryan Baker, et al. “Integrating MOOCs and Intelligent Tutoring Systems: edX, GIFT, and CTAT.” Fourth Annual ACM Conference on Learning at Scale, 2018.
- Wang, Yuan, and Ryan Baker. “Grit and Intention: Why Do Learners Complete MOOCs?” The International Review of Research in Open and Distributed Learning, 2018.
- Wang, Yuan, Ryan Baker, and Luc Paquette. “Behavioral Predictors of MOOC Post-Course Development.” Proceedings of the Workshop on Integrated Learning Analytics of MOOC Post-Course Development, 2017.
- Joksimovic, S., R. Baker, et al. “Automated Identification of Verbally Abusive Behaviors in Online Discussions.” Proceedings of the 3rd Workshop on Abusive Language Online, 2019.
Another thing to know is that a lot of people don’t go into MOOCs with the goal of completing. The goals vary more than you might expect in a traditional online course. Sometimes people took the course to watch one specific video, and that was it. Some took it to network—and frankly that’s fine, if that’s what they wanted to gain from it. Others took the entire MOOC and got into all aspects of it.
Some of the challenges you have written about for K–12 education include gaming the system, boredom, and carelessness. Are these also a problem in MOOCs?
Most of today’s MOOCs provide limited learning support and capture limited data. They don’t include rich enough problems for us to tell much about these behaviors, so that is an area of potential improvement that my colleagues and I are focused on.
For example, many MOOCs just feature videos to watch and some simple quizzes. But you can’t tell much from video viewing. If someone just lets the video play, you can’t tell whether they are engaged or distracted. Many MOOCs may not allow you to click ahead to get through the video quicker, but anyone with a brain and a pulse knows that you can just let the video run and come back when it’s over.
One thing we do see is “answer harvesting.” That’s when a student will create two accounts; one to go through the test the first time and get all the answers and the other to go back and redo it so the student gets a score of 100.
What solutions do you see to these unique challenges of MOOCs?
Unless you create rich enough challenges, such as simulations and multistep problem-solving, to see what the students are doing at every step of the learning activity, you can’t learn very much. Developing richer problem-solving has been shown to create better learning outcomes anyway—and it can also help us with our research.
One way we addressed this in our Big Data and Education MOOC was by creating learning activities that are divided into steps so that students have to show each step of the process. Students do a step and then we check the answer to see if they did that step right. We also offer hints and feedback for each step, some of which take students back to the videos. By watching if they view those videos when prompted, we can tell if the students are paying attention to our hints and feedback.
Based on your research and experience, what can educators do to improve engagement in MOOCs?
Here are a few things that I’d recommend:
- Try to create personalized learning systems by putting in rich problem-solving, and then adapt to the learner’s behavior. If you can determine what’s causing the learner to struggle and where in the process the learner is struggling, you can intervene at that appropriate stage to offer support. This is the case in the previous example of the program with step-by-step check-ins, for example. In another course, I worked individually with students who were struggling, according to the performance data: I wrote them emails and offered personal invitations to come to office hours. I think it made the students feel cared about.
- Be prepared to redirect attention when online learners distract each other (or themselves). This is especially important in discussion forums, where one comment can take the students completely off course. If a student brings up a problem with the content or system, respond right away and then fix the issue as quickly as you can. But if a conversation is going completely off topic, one useful approach can be to participate in other discussion threads to promote more constructive conversations. Students can act very differently in online courses than they would in a live course, so it’s valuable to keep an eye on discussions so students don’t get too distracted.
- Tell students why they shouldn’t try to trick you. I say, “Hey, just so you know, I can see all your data. You’ll get full credit if you take the course seriously. If you game the system, I’ll be able to see it, and you won’t get full credit.” This is even easier and more effective in live courses with an online learning system component!
What effect do you hope your research will have on future MOOCs?
The important thing for researchers like me is to get the right data in place—and to make sure the models are verified to be equally accurate for all learners. I hope that our efforts will produce MOOCs that take advantage of the full pedagogy that online and computer-based learning can offer.
MOOCs are a great way to learn a lot of things, and I think we can—and will—make them better.