diff --git a/sigs/README.md b/sigs/README.md index 14f7a2c..f6e4bc4 100644 --- a/sigs/README.md +++ b/sigs/README.md @@ -22,6 +22,7 @@ SIG的全称是Special Interest Groups,即“特别兴趣小组”。MindSpore | [Parallel](parallel/README.md) | 自动并行技术 | [@baiyouhui](https://gitee.com/bert0108) | | [DataCompliance](datacompliance/README.md) | 数据合规风险分析 | [@gopikrishnanrajbahadur](https://gitee.com/gopikrishnanrajbahadur) [@clement_li](https://gitee.com/clement_li) | | [MindQuantum](mindquantum/README.md) | 量子计算软件与算法 | [@dorothy20212021](https://gitee.com/dorothy20212021) | +| [XAI](xai/README.md) | 可解释AI技术 | [@jasonpolyu]( jason-c.zhang@polyu.edu.hk) | ## 学习资源 diff --git a/sigs/xai/README.md b/sigs/xai/README.md new file mode 100644 index 0000000..99611ad --- /dev/null +++ b/sigs/xai/README.md @@ -0,0 +1,152 @@ +# **MEP – XAI** + +## **Summary** + +Explainable AI (also termed transparent AI) is a form of artificial intelligence whose behavior is easily understood by humans. Unlike a "black box" in machine learning, in which the creators of an AI cannot explain how a specific decision was made, it implies the "explainability" of the algorithm's operation. The MindSpore XAI SIG is an initiative designed to build a collaborative environment for innovative research and industrial applications in XAI. + +## **Motivation** + +Theoretical flaws in machine learning decision-making mechanisms + +Due to data samples' general limitations and biases, this association learning will inevitably learn a spurious relationship. A model based on this as a decision-making basis may perform well on most test data, but in fact, the reasoning and decision-making ability based on correct causality has not been learned, and its performance will be greatly reduced when faced with new data with distribution shift from the training samples. + +Application pitfalls of machine learning + +First, due to the limitations and biases of data sample collection, data-driven AI systems are also biased, tantamount to bias in human society. Entrusting the future and destiny of individuals to such a biased artificial intelligence system damages social justice and causes contradictions among social groups. + +Secondly, the "black box" deep neural network often makes low-level mistakes humans do not make, leading to potential security risks. + +Lastly, and most importantly, from the point of view of the decision-making mechanism, the current analysis of deep learning algorithms is still in an opaque exploratory stage. Especially for super-large-scale pre-trained neural networks with hundreds of millions of parameters, such as BERT[1], GPT3[2], etc., the decision-making process is still not clearly explained academically. Such "black box" deep neural networks cannot be fully understood and trusted by humans for the time being, and the potential risks of large-scale application of such pre-trained models cannot be ignored. + +Traditional AI systems fail to meet regulatory requirements + +In major fields such as finance, medical care, and law, legislation on the prevention and supervision of the application risks of artificial intelligence systems has been gradually strengthened and implemented. + +## **Goals** + +### **The goals of this SIG are as follows:** + +1. To develop novel solutions to basic scientific problems such as poor robustness, poor interpretability, and strong dependence on data of artificial intelligence methods represented by deep learning; + +2. To improve the state-of-the-art XAI solutions, such as perturbation, counterfactual, and explainable GNN; + +3. To explore the basic principles of machine learning, develop explainable and general-purpose next-generation artificial intelligence methods; + +4. To promote the innovative application of explainable artificial intelligence methods in the scientific/industrial fields; + +5. To promote academic activities, including academic workshops, conferences, and contests; + +6. To contribute to open-source software for XAI based on MindSpore + +### **Non-Goals** + +None. + +### **Proposal** + +**User Stories** + +* Explainable AI + +Explainable AI can be organized according to three phases of the life-cycle of XAI: (i) Methodology; (ii) Evaluation; (iii) Application. A new taxonomy, data-driven approach, and knowledge-aware approach are developed based on whether external knowledge is involved in the explanation generation process, and their characteristics are summarized as follows: + +
+ +
Figure 1. A taxonomy of data-driven and knowledge-driven AI [3]
+ +Users need to justify the effectiveness of explanations before trust and confidence between them and machines can be built. We group the evaluation methods into two categories: computational metrics and cognitive metrics. Typically, computational metrics are automated and standardized, task-dependent, explanation-type-specific, and can be automated without human involvement. There are several computational metrics in terms of evaluation targets as follows: + +* Post-explanation Performance +* Faithfulness and Fidelity +* Robustness +* Localization + +Furthermore, computational metrics require knowledge of AI and XAI, so users of computational metrics should be AI experts rather than laypeople. Conversely, the cognitive metrics are usually collected through user studies on cognitive experiments so that they can be applied to all levels of users. + +* Trust-native AI + +Human cognition has two processing systems: + +System 1 is responsible for processing intuitive, rapid, unconscious, non-verbal, inertial precision information, that is, precision-related cognitive processing. + +System 2 is responsible for processing slower, conscious, structured, reasoned complex information, i.e., Trust-Related Cognitive Processing. + +
+ +Here we adopt a computer-scientific perspective and define knowledge as information about how entities relate to one another under certain circumstances. In addition to the usual information source in a machine learning pipeline, training data, and knowledge can also be incorporated. If this knowledge is pre-existent and independent of learning algorithms, it can be called prior knowledge. Furthermore, such prior knowledge can be expressed in formal representations, which exist externally and separately from the learning problem and training data. Informed machine learning is defined as machine learning that explicitly incorporates such knowledge representations. + +
+ +
Figure 2. An overview of knowledge-enhanced machine learning [4]
+ +Based on the three above analysis questions about knowledge source, representation, and integration, this taxonomy serves as a classification framework for informed machine learning. + +
+ +
Figure 3. A taxonomy of knowledge-enhanced machine learning [4]
+ +Here we give a first conceptual overview of the knowledge representation types, and for each knowledge representation, we collect the informed machine learning approaches and present the observed (paths from) knowledge source and the observed (paths to) knowledge integration. + +* Algebraic Equations +* Differential Equations +* Simulation Results +* Spatial Invariances +* Logic Rules +* Knowledge Graphs +* Probabilistic Relations +* Human Feedback + +**Placeholder (e.g., Planned Activities)** + +We plan to organize seminars/workshops/conferences to promote academic development and industrial applications in the field of XAI. + +**Notes/Constraints/Caveats (optional)** + +NA + +**Risks and Mitigations** + +NA + +**Design Details** + +NA + +**Test Plan** + +NA + +**Graduation Criteria** + +NA + +**Upgrade / Downgrade Strategy** + +NA + +**Implementation History** + +NA + +**Drawbacks** + +NA + +**Alternatives** + +NA + +**Infrastructure Needed (optional)** + +NA + +**References** + +[1] DEVLIN J, CHANG M W, LEE K, et al. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding[J]. arXiv e-prints, 2018: arXiv:1810.04805. + +[2] BROWN T, MANN B, RYDER N, et al. Language models are few-shot learners[C/OL]// LAROCHELLE H, RANZATO M, HADSELL R, et al. Advances in Neural Information Processing Systems: Volume 33. Curran Associates, Inc., 2020: 1877-1901. + +[3] Li X H, Cao C C, Shi Y, et al. A survey of data-driven and knowledge-aware explainable ai[J]. IEEE Transactions on Knowledge and Data Engineering, 2020, 34(1): 29-49. + +[4] Von Rueden L, Mayer S, Beckh K, et al. Informed Machine Learning--A Taxonomy and Survey of Integrating Knowledge into Learning Systems[J]. arXiv preprint arXiv:1903.12394, 2019. +