Webinar: Responsible AI in assessment

For the 8th installment of our Innovations in Language Assessment webinar series, we welcome DET Principal Assessment Scientist Dr. Jill Burstein! AI is increasingly being integrated into digital assessments, with automated essay scoring and item generation as key examples. While beneficial, the use of AI on assessments also poses risks that need to be identified and carefully managed. Other sectors (such as tech and government) already have responsible AI (RAI) standards. The assessment community must also develop RAI standards. Standards should be driven by human values, and thoughtfully designed to mitigate risks associated with the use of AI. The AERA, APA, & NCME Standards (2014) include automated scoring; however, those standards are insufficient as the use of AI for assessment has expanded beyond scoring. While extensive RAI guidelines exist, these point out potential risks, but not specific solutions. By contrast, standards identify the risks, and provide actionable practices to hold organizations accountable. Additionally, from both practical and scientific perspectives, RAI standards inform responsible application of standards, and serve as an auditing mechanism. This presentation provides an overview of RAI, and its application for assessment. The novel Duolingo English Test RAI Standards are presented as a case study to demonstrate how standards are applied to mitigate AI risks, and are an essential part of professional responsibility. Read the standards here: https://duolingo-papers.s3.amazonaws....