As one of the students surveyed commented, ‘It is like my 24/7 TA'
Thousands of students are doubtless relying on ChatGPT to prepare for finals this time of year. Instructors less initiated into this ever-evolving, new AI tool might be curious — what exactly are their students doing with it?
Probably quite a lot of things, according to a recent study conducted by a group of BYU professors who had that very question. When ChatGPT burst onto the scene in early 2023, the professors noticed all kinds of confusion on social media about who should be using it and why. They decided to ask 455 BYU students to share the prompts they’d fed ChatGPT and the instructions their teachers gave them about the new tech.
The results showed that students were taking advantage of the tool’s interactive, iterative nature to converse with ChatGPT as they might with an instructor. “As one of the students commented, ‘It is like my 24/7 TA,’” said BYU computer science professor Amanda Hughes, a co-author of the paper, which was published in the proceedings of 57th Hawaii International Conference on System Sciences. “I thought that was a really interesting description of the technology.”
From the survey data, the professors grouped students’ use of ChatGPT into four broad categories:
- retrieving basic information (“When did this happen?”, “How does this work?”)
- generating content, such as writing a piece of code or outlining a paper
- revising, such as correcting or improving an essay in a designated way
- evaluating, such as assessing the quality of a resume.
They also found that students used ChatGPT for things like study support, which involved having the tool create questions their professor might put on an exam and then quiz them, or for giving advice.
“The thing ChatGPT does that is obviously very different from previous tools like Google is that it allows you to have a contextual conversation, where if ChatGPT gives you a response that isn’t quite right, you can give it more information and ask it to rephrase with the new information in mind,” Hughes said. “With Google, you can ask it a different question, but it doesn’t maintain the context of what you originally asked.”
This feature makes the tool more like a human teacher, capable of adapting to students’ specific needs. For example, one student talked about how it could be used as a virtual teacher: “I’m a very shy person, but I felt like I could ask all the follow-up questions I needed to understand when I was having a conversation with ChatGPT.” Of course, substituting ChatGPT for a human instructor or even a Google search has some inevitable drawbacks for student learning.
“One of the things I suspect is that tools like ChatGPT may make us content with less accurate and less satisfying answers because it just gives the answer without forcing us to look at other sources,” Hughes said. “My theory is that over time this might erode students’ ability to discern when they need to keep looking for a better answer.”
Hughes is optimistic overall, though, about ChatGPT as a teaching tool. Not only can AI expedite tedious daily tasks like composing emails, it may also prepare students to learn more advanced skills in a class than they otherwise might.
“For example, in my field of computer programming, there’s a huge learning curve to get to a point where you can build something useful that isn’t just a pet program you created for class,” she said. “If ChatGPT can help students write code more quickly, it may allow students to imagine bigger and better solutions for a program instead of getting caught up in all the small details of how to make something work.”
Instructors’ collective ambivalence about ChatGPT was apparent in the survey responses, with students reporting that some teachers prohibited ChatGPT altogether, while others encouraged using it and gave advice on how to best take advantage of the tool. Based on discussions she’s had with other instructors, Hughes believes the future of AI in the classroom likely looks like teaching students how to use the technology to supplement their work.
“If they don’t know how to use tools like ChatGPT, they’re going to be at a disadvantage because other people in their industries will be using them and are going to be much more effective as a result.”
The paper was additionally co-authored by Ryan Schuetzler, Justin Giboney, Taylor Wells, Benjamin Richardson, Tom Meservy, Cole Sutton, Clay Posey and Jacob Steffen.
