On a recent morning, Cheryl Drakeford, a third-grade teacher at First Avenue Elementary School in Newark, posed a challenging math question on her classroom whiteboard: “What fraction of the letters in the word mathematician are consonants?”
Ms. Drakeford knew that “cuisine” might be an unfamiliar word for some students. So he suggested they ask Khanmigo, a new tutoring bot that uses artificial intelligence, for help.
She paused for a minute while about 15 school children dutifully typed out a single question – “What are the dishes?” – in their math software. Then he asked third graders to share the answer with the tutoring bot.
“Consonants are letters of the alphabet that are not vowels,” a student read aloud. “The vowels are a, e, i, o and u. Consonants are all other letters.”
Tech industry hype and doomsday predictions about AI-enhanced chatbots like ChatGPT forced many schools this year to block or limit the use of the devices in classrooms. Newark Public Schools is taking a different approach. It is also one of the first school systems in the United States to pilot test KhanMigo, an automated learning aid developed by Khan Academy, an education nonprofit whose online lessons are used by hundreds of districts.
Newark has essentially volunteered to be a guinea pig for public schools across the country trying to separate the practical uses of the new AI-assisted tutoring bot from its marketing promises.
Proponents argue that classroom chatbots could democratize the idea of tutoring by automatically customizing students’ responses, allowing them to work on lessons at their own pace. Critics warn that the bots, which are trained on vast databases of texts, can fabricate false information to appear credible – making them a risky bet for schools.
Officials in Newark, New Jersey’s largest district, said they are carefully testing the tutoring bot at three schools. Their findings could affect districts across the United States that are testing AI tools this summer for the upcoming school year.
“It’s important to introduce our students to this, because it’s not going away,” timothy nelegersaid the director of educational technology at Newark Public Schools about the AI-assisted technology. “But we need to find out how it works, the risks, the good and the bad.”
Khan Academy is one of a handful of online learning companies that have created new tutoring bots based on language models developed by OpenAI, the research lab behind ChatGPT. The Khan Academy, whose high-level tech donors include Google, the Bill & Melinda Gates Foundation and the Elon Musk Foundation, received access to the AI model last year.
Designed specifically for schools, tutoring bots often lead students through the sequential steps needed to solve a problem.
When Khan Academy began looking for districts to test its experimental Tutorbot this spring, Newark volunteered. Several local elementary schools were already using the education organization’s online math lessons as a way to track students’ mastery of concepts such as grouping numbers. And the AI tool will be free for those schools during the initial pilot-testing phase.
District officials said they want to see if Khanmigo can increase student engagement and math learning. Schools like First Avenue, which has many children from low-income families, were also eager to give their students an early opportunity to try out the new AI-assisted learning aid.
Districts such as Newark that use Khan Academy’s online lessons, analytics and other school services—not including KhanMigo—pay an annual fee of $10 per student. Participating districts that want to pilot Minego for the upcoming school year will have to pay an additional fee of $60 per student, the nonprofit said, noting that the computing cost for the AI model was “significant.”
Newark students began using Khan’s automated learning aids in May. Reviews so far have been mixed.
One recent morning, sixth-grade students at First Avenue Elementary were working on a statistics assignment that involved developing their own consumer surveys. His teacher Tito Rodríguez suggested the students begin by asking Khanmigo two background questions: What is a survey? What makes a question statistical?
Mr. Rodriguez described the bot as a useful “co-educator” that allowed him to devote extra time to children who needed guidance, while enabling more self-driven students to move forward.
“Now they don’t have to wait for Mr. Rodriguez,” he said. “They can ask Khanmigo.”
Down the hall in Ms. Drakeford’s math class, the bot’s responses to students sometimes sounded less like suggestions and more like direct answers.
When students asked Khanmigo a variation question posted on the classroom white board, the bot replied that the word “mathematician” had 13 letters and seven of those letters were consonants. This means the consonant fraction was seven out of 13, Bott wrote, or 7/13.
“It’s our biggest concern, that there’s a lot of thought work going on through Khanmigo,” he said. Alan Usherenko, the district’s special assistant for schools including First Avenue in Newark’s North Ward. The district didn’t want the bot to take students through a problem step by step, he said, adding, “We want them to know how to approach a problem, use their critical thinking skills.”
In an email, Khan Academy said that students often require initial assistance to move through the problem-solving steps, and that practicing can help them learn to move through the steps automatically without assistance.
The group said the tutoring bot was designed to help students solve problems, not give them answers. But in the case of a different problem in Newark, the organization said, Khanmigo helped “too much, too fast.”
“Our engineering team corrected the AI a few weeks ago so that this question is no longer answered,” Khan Academy said in an email on Tuesday.
On Wednesday, a reporter asked Khanmigo the same different question. In student mode, the tutoring bot explained the steps and then provided a direct answer: “The fraction of consonants in the word ‘mathematician’ is 7/13.”
In teacher mode, which is designed to take teachers through the problems and answers, the bot provided a different – incorrect – response. Khanmigo mistakenly said that the word “mathematician” had eight consonants. This caused the bot to give the wrong answer: “8 consonants / 14 total letters = 8/14”
In an email, Khan Academy said it had fixed the problem within its “Tutor Me: Math and Science” section for students, noting that the reporter had asked a question in a different part of the site. “As far as the teacher giving wrong answers,” the email said, “sometimes Khanmigo makes mistakes.”
Still, Mr. Usherenko said he remains hopeful. The district suggested to Khan Academy that, rather than relying on students to ask KhanMigo the right questions, it would be more helpful if the bot asked students open-ended questions and analyzed their responses.
Mr. Usherenko said of Khanmigo, “It’s not yet where I want it to be.” “But when it addresses students’ misconceptions, it will be a game changer.”
Khan Academy said the tutoring bot often asks students open-ended questions and the group is working on getting an AI model to accurately identify misconceptions. The nonprofit said it continues to improve Khanmigo with feedback from school districts.
It remains to be seen whether schools will be able to afford AI-assisted tutorbots.
Khan Academy said it would offer the discount for districts where more than half of its students qualify for free or reduced-price lunch. Still, financial constraints suggest that AI-enhanced classroom chatbots are unlikely to democratize tuition any time soon.
Newark’s ed tech director, Mr. Nellegar, said his district was looking for outside funding to help cover the cost of Khanmigo this fall.
“The long-term cost of AI is a concern for us,” he said.
