Duration: 1:08:41

1,662 views

29

As AI is becoming part of user-facing applications and is directly impacting society, deploying AI reliably and responsibly has become a priority for Microsoft and several other industry leaders. In recent years, Microsoft has developed a set of AI principles and standards alongside a company-wide ecosystem to guide responsible AI development and deployment. In this webinar, Microsoft researchers Dr. Besmira Nushi and Dr. Ece Kamar will share crucial learnings gained from founding and implementing such principles in practice in a large industry setting, where investments in AI span from automation to enhanced human productivity and augmentation.

The webinar will present examples of how these learnings are shaping our research on developing principles and tools for bringing the AI principle of reliability and safety to reality. In particular, it will showcase an ecosystem of open-source tools that are intended to accelerate the machine learning (ML) development life cycle by identifying and mitigating failures in a faster, systematic, and rigorous way. These efforts to develop tools are guided by the observation that aggregate metrics are not sufficient to evaluate AI reliability; we need deeper insights into detailed model performance. The discussion will conclude by giving a glimpse into our long-term vision for empowering AI developers with integrated responsible AI tools covering the complete AI life cycle.

Together, you will explore:

■ Error Analysis—a tool for identifying and diagnosing failure modes of an individual ML model. Diagnosis is supported either via interactive data explorations or via model explanations based on interpretability techniques provided in InterpretML.
■ BackwardCompatibilityML—a tool for expanding these insights to the scenario of model updates, helping engineers make informed decisions about which model to select for deployment while taking into account regions in which an updated model progresses and regresses.

𝗥𝗲𝘀𝗼𝘂𝗿𝗰𝗲 𝗹𝗶𝘀𝘁:

■ Error Analysis Tool: https://erroranalysis.ai
■ BackwardCompatibilityML tool (GitHub) :  https://github.com/microsoft/backwardcompatibilityML
■ Error Analysis (Azure Blog): https://techcommunity.microsoft.com/t5/azure-ai/responsible-machine-learning-with-error-analysis/ba-p/2141774
■ BackwardCompatibilityML (Microsoft Research Blog): https://www.microsoft.com/en-us/research/blog/creating-better-ai-partners-a-case-for-backward-compatibility
■ Towards Accountable AI: (Paper): https://www.microsoft.com/en-us/research/publication/towards-accountable-ai-hybrid-human-machine-analyses-for-characterizing-system-failure
■ Responsible AI Resources (Project page): https://www.microsoft.com/en-us/ai/responsible-ai-resources
■ Ece Kamar (Researcher Profile): https://www.microsoft.com/en-us/research/people/eckamar/
■ Besmira Nushi (Researcher Profile): https://www.microsoft.com/en-us/research/people/benushi/

*This on-demand webinar features a previously recorded Q&A session and open captioning.

Explore more Microsoft Research webinars: https://aka.ms/msrwebinars