New Report Maps Strengths and Risks of AI Learning Tools

A new large-scale evaluation of artificial intelligence tools in schools has produced a mixed verdict. The report, conducted by the nonprofit Instruction Partners, examines sixteen specific AI products to determine how they impact teaching and student achievement. While some tools show promise in supporting learning, the analysis warns that others are described as isolating, boring, and potentially damaging to children's educational development.

This effort addresses a growing gap in the education sector. As AI tools become increasingly common in classrooms, there is limited comprehensive information about their instructional quality. Previous reviews by organizations like EdReports and Common Sense Media have focused on safety or curriculum alignment, but this new project is one of the first to systematically weigh the pedagogical value of these technologies against their risks.

Mixed Results in Classroom Observations

The team analyzed twenty tools by interviewing teachers, students, and district leaders, and by observing technology in action across sixteen school systems. The findings highlight that not all AI is created equal. Emily Freitag, co-founder of Instruction Partners, emphasized the need for nuanced decision-making to identify where technology offers real value versus where it creates risk. The profiles include user feedback, instructional capability reviews, and available research evidence.

Teachers reported that some tools felt more like a hindrance than a help. A specific concern raised by educators was the interaction between general-purpose chatbots and search engines. In observed instances, students would ask a question to an AI tool, then use a search engine to find the answer, and finally input that answer back into the AI. This workflow suggests that the technology may not be facilitating genuine learning but rather acting as a shortcut for information retrieval.

Challenges in Assessing Quality

Evaluating AI tools is more complex than reviewing traditional curriculum. Lewis Ferebee, CEO of EdReports, noted that the rapid evolution of these products makes it difficult to establish consistent quality standards. While organizations like Common Sense Media rate tools for safety and security, they do not assess instructional quality. This lack of comprehensive data means schools often make decisions without clear evidence of how well these tools support teaching and learning.

The report categorizes tools into three types: general-purpose chatbots, education-focused multi-application tools, and discrete instructional tools. Each profile explains the roles of the student, AI, teacher, and school leaders. The goal is to cut through marketing language and help administrators make informed choices. However, the fast-paced nature of AI development means that these assessments may quickly become outdated as features and capabilities change.

Implications for School Decision Making

For schools, the findings suggest a cautious approach to adopting AI. The report serves as a resource for leaders to understand the specific strengths and limitations of each tool. It highlights that while some products may enhance engagement, others risk disengaging students. The trade-off is that relying on AI without proper evaluation can lead to significant gaps in learning, particularly when general-purpose chatbots are used in place of targeted instructional support.

The analysis from Instruction Partners provides a necessary counterpoint to the rapid adoption of AI in education. By documenting real-world experiences and user feedback, it offers a clearer picture of the technology's impact. Schools are encouraged to use these insights to make data-driven decisions, ensuring that AI tools serve as effective aids to learning rather than sources of confusion or disengagement. This approach aligns with calls from educators for more systematic assessments of the AI landscape.