How to explain black-box machine learning algorithms?

Building machine learning models that are unbiased, transparent and explainable is the holy grail of AI.

mltrons xAI module: Building AI Solutions That You Can Trust
Data Science|Data Strategy|Competitive Advantage|High Productivity|Cost Saving|COVID-19

" When decisions are sensitive, we need algorithms that we can trust. "

“To be or not to be” became the mantra of thought and self-reflection in the philosophical arena when Hamlet uttered these words in Shakespeare’s famous tragic play. In today’s business world, driven by decisions made by artificial intelligence, that mantra has changed into “to trust, or not to trust”.

ML Meme - garbage in garbage out

When it comes to sensitive decisions, we have seen AI fail in providing accurate predictions. From inaccurately predicting the quality of air to providing wrong diagnoses of a patient, we understand that our models are inherently biased and opaque. Despite the hype of AI, more than 60% of businesses are yet to get value from AI, and most of it is because of lack of trust in these algorithms - many times, there is a clear lack of business domain knowledge.

The current business landscape is still very sceptical when it comes to implementing and trusting these AI systems. Many companies have initiated the process, but have yet to realize the value. This is mainly due to the understanding gap between data science teams and business stakeholders. We talked to many business stakeholders over the past few months, that are on the receiving ends of these predictions, and found the inability of the data scientist to explain the why and how behind AI systems predictions to be the biggest factor of mistrust and scepticism towards the data science initiative. People in the data science teams are highly technical with a knack for complexity to signal the extent of their skillset. However, business stakeholders are the complete opposite: they don’t care about the technology used, but how the results generated by the model tie-up with their business goals and KPIs.

This is impossible to achieve unless the data scientist can answer these essential questions:


  • Why should I trust the results generated by the model?
  • What was the rationale used by the model to generate the results?
  • What are the upside and the downside of using the model in production?
  • Do the results align with the business logic or not?

Only after answering these questions, the data scientist can bring recommendations to the business user, and expect to make some progress.

To solve this, the data scientist has two choices:

  • Explain the black-box models by building an interpretable model on top of it. This is the logic behind LIME & SHAP. SHAP is more widely used since it guarantees a fair distribution of contribution for each of the variables and has a wide array of graphs. Sadly, this approach requires a lot of iterations, lacks interactivity and is not scalable especially when you’re dealing with sensitive datasets and decisions. More than that, the visualizations are not appealing and interactive. Their static nature creates an even further divide between the data scientist and the business stakeholder. The absence of dynamic and interactive graphs makes it extremely difficult to generate value from SHAP or LIME, thus a better way to use these techniques is required.

  • Use interpretable models: instead of using black-box models such as deep neural networks, a data scientist can try to optimize simpler models like logistic regression or decision trees to make predictions. There will be a trade-off in accuracy and interpretability, but the data scientist will need to decide what is important to generate value and will need to focus on the marginal benefits between the two models. If the marginal increase between accuracies is not significant, it’s more ideal to implement simpler models and tie the predictions directly with business KPIs. Sadly, with the increasing complexity in the data, we’re collecting today, simpler models do not perform well.

So the question arises: Is there a better way of building trust in our machine learning models?

Yes, there is! At mltrons, our vision is to increase the adoption of AI and accelerate towards achieving singularity. To make that happen, we embark on the mission to help data scientists build AI algorithms that are understandable, explainable and unbiased. This will ensure that everyone affected by AI will be able to understand why decisions were made and ensure the AI results were unbiased, accurate and free of any logical inconsistencies.

To fulfil our mission, we’re engineering a plug-n-play xAI system for data scientists that will assist them in understanding, explaining, visualizing and validating the why and the how behind black- machine learning predictions — in a fully immersive and interactive manner. The system will aim to help the data scientist and business stakeholders build trust in the AI system and make fully informed decisions.

mltrons xai module diagram

What differentiates mltrons xAI engine from alternatives currently in the market is our system’s ability to function across multiple datasets and custom-built models. Instead of making scientists switch to a new stand-alone system, we aim to implement our system within the current workflow of data scientists.

This means that data scientists can now bring in their Jupiter notebooks, data sources — Amazon, MySQL, HDFS and custom-built models using XGBoost, CatBoost, PyTorch, Tensorflow, SageMaker to the mltrons engine — mltrons engine will take in their input and will work as an added layer to provide explainability on how these algorithms work, think and output results. The data scientist will be able to then explain the results in simple business-friendly language, ideally understood by anyone, through our interactive visualizations, reports, and shareable dashboards.

To supercharge our mission, we’re connecting with data scientists and working hands-on with them, especially in the field of healthcare and financial services. We will be launching our beta in another month and will keep improving it by working together with our community. So if you believe in increasing AI adoption and building more trust in machine learning algorithms, connect with us!

powered by Typeform

You can sign-up with your information and we'll get back really soon.