Attention to Context is What xAI Needs: A Framework for Generating Context-Dependent Requirements for Human-Centric Explainable AI (xAI)

Abstract

As the rapid integration and widespread deployment of Artificial Intelligence (AI) systems across vital societal domains have yielded promising results, their opacity - the "black box" problem - poses significant barriers to trust in, accountability of, broad acceptance, and adoption of this transformative technology. While Explainable AI (xAI) seeks to address the black box problem, current initiatives often prioritize computational and algorithmic transparency over the actual explanatory needs of human users. Since explanations are ultimately for the benefit of humans, it is crucial to tailor them to their needs.

This interdisciplinary thesis addresses two major gaps that persist in current xAI development: an overemphasis on computational approaches due to the underutilization of Social Science and Humanities (SSH)-based knowledge on the concept of explanation; and insufficient attention to the role of context in shaping what constitutes an adequate explanation for diverse, non-AI expert stakeholders.

To bridge these gaps, this research project proposes the Framework for Generating Context-Dependent Requirements (FGCR). The FGCR is a multidisciplinary practical guide and toolkit designed to guide xAI developers in identifying contextually suitable xAI solutions for diverse stakeholders by placing the explainee at the center of the xAI development cycle. It provides a structured process and accompanying suite of tools to support this goal across varied contexts. The framework integrates insights from social sciences, humanities, engineering requirements management, and xAI research. Its components include novel taxonomies (an SSH-based taxonomy of explanation (classifying the "what" and "how" of human explanation) and a taxonomy of the xAI landscape) alongside established engineering requirements management practices to identify what constitutes an adequate and contextually appropriate human-centric explanation.

The research was conducted in three phases. Phase 1 comprised a multi-disciplinary literature review synthesizing SSH-based concepts of explanation, the current landscape of xAI approaches (with a focus on artificial neural networks (ANNs)), and engineering requirements management processes. Phase 2 involved the design and construction of the FGCR. Phase 3 undertook proof-of-concept testing of the FGCR using a qualitative case study of autonomous vehicle (AV) crashes, specifically, a real-world series of incidents in which Tesla's Autopilot system disengaged shortly before colliding with stationary first-responder vehicles. The study tested whether the FGCR could effectively categorize varied stakeholder inquiries and generate adequate xAI model requirements.

The FGCR was evaluated through a proof-of-concept qualitative study involving semi-structured interviews with 26 stakeholders representing a diverse cross-section of society including: pedestrians, drivers, legal experts, first responders, and AI developers. The study elicited stakeholders' explanatory needs and applied the FGCR's components to generate specific xAI model requirements. Findings from the proof-of-concept testing show that the FGCR can identify and classify varied explanatory needs and translate them into xAI model requirements and potential xAI solutions that meet those requirements.

Importantly, findings indicate that while the diversity of AI applications and the complexity of context both play a pivotal role in explanations and drive the need for tailored xAI solutions, it is the need of explainees that take primacy in determining an explanation that is both relevant and sufficient. Essentially, while technical accuracy is necessary, the adequacy of an explanation is primarily determined by the explainee(s)' context and intended use of the information.

This research contributes to the field by providing a structured and practical framework for xAI developers to move beyond a "one-size-fits-all" technical approach for development of transparent, and understandable human-centric AI systems. By bridging theoretical SSH and engineering requirements management knowledge, it offers a roadmap for creating explainable AI systems that are genuinely intelligible to humans they affect. The result is a practical framework and toolkit that prioritizes context, stakeholder diversity, and the needs of those affected by AI systems at the center of the xAI development cycle.