Lay Abstract for Applied Stochastic Models in Business and Industry article: Lost in a black-box? Interpretable machine learning for assessing Italian SMEs default

Each week, we publish lay abstracts of new articles from our prestigious portfolio of journals in statistics. The aim is to highlight the latest research to a broader audience in an accessible format.
The article featured today is from Applied Stochastic Models in Business and Industry with the full Open Access article now available to read here.
Crosato, L, Liberati, C, Repetto, M. Lost in a black-box? Interpretable machine learning for assessing Italian SMEs default. Appl Stochastic Models Bus Ind. 2023; 118. doi: 10.1002/asmb.2803

The European Union (EU) economy is deeply grounded in Small and Medium Enterprises (SMEs), that is why default prediction for SMEs is of the uttermost importance for scholars as well as for financial intermediaries and policy makers in their effort to support them and to ease credit constraints to which they are naturally exposed. Whether for private credit-risk assessment or for public funding, independently of the type of data imputed to measure a firm health status, prediction of default should succeed in two aspects: maximize correct classification and clarify the role of the variables involved in the process. Most of the times, the contributions based on Machine Learning (ML) techniques neglect the latter aspect, often with better results with respect to standard parametric techniques that provide, on the contrary, a clear framework for interpretation. In other words, ML techniques rarely deal with interpretability which, according to a recent document released by the European Commission, should be kept “in mind from the start”. Interpretability is central when applying a model in practice, both in terms of managerial decisions and compliance: it is a fundamental requisite to bring a model into production. Interpretable models allow risk managers and decision makers to understand their outcome and to knowingly take courses of actions. Accordingly, ML models — no matter how good in classifying default — should be made readable to avoid that their inherent uninterpretable nature may prevent their spreading in the literature on firms’ default prediction as well as their use in other contexts regulated by transparency norms.

This work tries to fill this gap by applying two different kind of ML models and three parametric models to Italian Manufacturing SMEs’ default prediction, with a special attention to interpretability. Italy represents an ideal testing ground for SMEs default prediction since its economic framework is more extensively configured by firms up to this size than the average of EU countries. Default was assessed on the basis of the firms’ accounting information retrieved from Orbis, a Bureau van Dijk (BvD) dataset. The main original contribution of the paper is to address ML models’ interpretability in the context of default prediction, via an approach based on model agnostic-techniques and in particular adding Accumulated Local Effects (ALEs) to the most widely used Shapley values. This way, the variables can be ranked in terms of their contribution to the classification and their impact on default prediction can be determined. Another contribution of the paper is the benchmarking of the ML models’ outcome with Logistic, Probit and with Binary Generalized Extreme Value Additive (BGEVA) classifications, both according to standard performance metrics and to the role played by the input features. The paper also moves a step forward with respect to the current use of ALEs, fully exploiting the tool and supplying them also for the parametric models, obtaining a few interesting results. First, eXtreme Gradient Boosting (XG-Boost) outperformed the other models mainly for total classification accuracy and default prediction rate. Second, the impact of the variables assessed by XG-Boost is fully consistent with the economic literature, whereas the same cannot be said for its competitors. Thanks to the ALEs framework for interpretability, risky thresholds, non-linear patterns and other additional insights emerge for predictors even in standard models.


More Details