Introducing XAISuite Version 2.0: Comparing Machine Learning Explainers

Scroll to the bottom of the page if you just want to delve right in to the technical documentation, without the introductions.

Welcome to our source page. Our mission is to make machine learning explanation easy and available to all! Whether you are a data scientist, researcher, or just a person curious about how you can gain a better understanding of artificial intelligence, XAISuite is the library for you. Please be sure to contribute and contact us if you have any questions.

Introduction

XAISuite (Explanatory Artificial Intelligence Suite) is a library for comparing explanatory systems. In addition, it streamlines the training and explanation generation process into one line of code so that users can focus on insights instead of the mechanics of how models are trained and explained.

While we would like to highlight the fact that you only need one line of code for training and explaining your models with XAISuite, our main contribution is a novel algorithm that outputs a similarity value between two ordered vectors containing the outputs of machine learning explainers.

But first, what are explainers? Machine learning models are opaque models, so we have no idea what’s going on inside of them. Explainers help us understand machine learning models we have trained and therefore give us aa better idea of why machine learning models fail in particular instances.

XAISuite accomplishes machine learning model training and explanation generation in three steps: (1) data loading, (2) data processing, and (3) model training and explanation generation. Each of these steps are delved into more detail in our documentation, in the demo tutorials, and are overviewed in the graphic below:

A schematic of XAISuite functionalities

A key part of XAISuite is flexibility, and, in our mission to make machine learning available to all, we have made XAISuite available in the following formats:

As a Python Library (with XAISuite and XAISuiteGUI)
On the Command Line (with XAISuiteCLI)
In block-code (with XAISuiteBlock)

A note on formats: XAISuiteCLI and XAISuiteGUI run on v1.0.8. This will not change in the near future.

As far as we know, XAISuite is among the first comprehensive libraries that allow users to both train and explain models, and the first to provide utilities for explanation comparison. XAISuite was created with a focus on users, and our interface reflects that. We also pioneered the ability to interact with machine learning models on the command line.

Installation

You can install the XAI Suite through PyPI:

pip install XAISuite

This will automatically install the latest version and is the reccomended way to download the library. The version on Github may not be stable. If you already have XAISuite and want to upgrade it, do:

pip install XAISuite --upgrade

Follow the instructions in individual folder READMEs for further installation instructions. For example, to install the command-line tool for XAISuite, do

brew install xaisuitecli

XAISuite 2.0.0 is not backward-compatible with XAISuite 1.0.8. If you want to install version 1.0.8, simply type: pip install XAISuite==1.0.8

Getting Started

For comprehensive example code and an introduction to the library, see the Demo Folder. The Demo folder is never fully complete and we will add more and more tutorials as the project progresses.

If you are looking for a model or dataset to use, sklearn has several cool options.

Examples of graphs and tables generated by the XAISuite version 1.0.8 can be found [here].(https://drive.google.com/drive/u/2/folders/10t4_GYDPJl2sM9hDOuezbum-yqKpN4fc).

Follow the instructions in individual folder READMEs for further installation instructions.

Below, we include an example of explaining a Support Vector Classifier Model as a demonstration of what XAISuite can accomplish.

from xaisuite import*
from sklearn.svm import*

z = DataLoader(make_classification, n_samples = 700)
y = DataProcessor(z, processor = "TabularTransform")
x = ModelTrainer(SVC(), y, explainers = ["lime", "shap"])
x.getExplanationsFor([])["lime"].ipython_plot(20)
a = InsightGenerator(x.getExplanationsFor([]))
corr = a.calculateExplainerSimilarity("lime", "shap")

#You can condense this in one line

corr = InsightGenerator(ModelTrainer(SVC(), DataProcessor(DataLoader(make_classification, n_samples = 700) , processor = "TabularTransform"), explainers = ["lime", "shap"]).getExplanationsFor([])).calculateExplainerSimilarity("lime", "shap")

For example involving PyTorch, SciKeras, and custom models, check out the tutorials and example code in the Demo folder.

How to Contribute

We welcome the contribution from the open-source community to improve the library!

To add a new functionality into the library or point out a flaw, please create a new issue on Github. We’ll try to look into your requests as soon as we can. Keep in mind that, as this is an open-source project, you release any copyright protection over code you may contribute to the XAISuite Project.

Technical Report and Citing XAISuite

Use the following BibTex to cite XAISuite:

For XAISuite v1.0.8:

@misc{mitra2023xaisuite,
      title={The XAISuite framework and the implications of explanatory system dissonance},
      author={Shreyan Mitra and Leilani Gilpin},
      year={2023},
      eprint={2304.08499},
      archivePrefix={arXiv},
      primaryClass={cs.LG}
}

For XAISuite v2.0:

@misc{mitra2023xaisuite,
      title={Paper pending},
      author={Shreyan Mitra and Leilani Gilpin},
      year={2023},
      eprint={},
      archivePrefix={arXiv},
      primaryClass={cs.LG}
}

Contact Us

If you have any questions, comments or suggestions, please do not hesitate to contact us at xaisuite@gmail.com