Luigi Scorzato

In recent years, the question of the reliability of Machine Learning (ML) methods has acquired significant importance, and the analysis of the associated uncertainties has motivated a growing amount of research. However, most of these studies have applied standard error analysis to ML models—and in particular Deep Neural Network (DNN) models—which represent a rather significant departure from standard scientific modelling. It is therefore necessary to integrate the standard error analysis with a deeper epistemological analysis of the possible differences between DNN models and standard scientific modelling and the possible implications of these differences in the assessment of reliability. This article offers several contributions. First, it emphasises the ubiquitous role of model assumptions (both in ML and traditional science) against the illusion of theory-free science. Secondly, model assumptions are analysed from the point of view of their (epistemic) complexity, which is shown to be language-independent. It is argued that the high epistemic complexity of DNN models hinders the estimate of their reliability and also their prospect of long term progress. Some potential ways forward are suggested. Thirdly, this article identifies the close relation between a model’s epistemic complexity and its interpretability, as introduced in the context of responsible AI. This clarifies in which sense—and to what extent—the lack of understanding of a model (black-box problem) impacts its interpretability in a way that is independent of individual skills. It also clarifies how interpretability is a precondition for a plausible assessment of the reliability of any model, which cannot be based on statistical analysis alone. This article focuses on the comparison between traditional scientific models and DNN models. However, Random Forest (RF) and Logistic Regression (LR) models are also briefly considered.

It is mostly agreed that Popper's criterion of falsifiability fails to provide a useful demarcation between science and pseudo-science, because ad-hoc assumptions are always able to save any theory that conflicts with the empirical data, and a characterization of ad-hoc assumptions is lacking. Moreover, adding some testable predictions is not very difficult. It should be emphasized that the Duhem-Quine argument does not simply make the demarcation approximate, but it makes it totally useless. Indeed, no philosophical criterion of demarcation is presently able to rule out even some of the most blatant cases of pseudo-science, not even approximatively. This is in sharp contrast with our firm belief that some theories are clearly not scientific. Where does this belief come from? In this paper I argue that it is necessary and possible to recognize the notion of syntactic simplicity that is able to tell the difference between empirically equivalent scientific and non-scientific theories, with a precision that is adequate to many important practical purposes, and it fully agrees with the judgments generally held in the scientific community.

One of the main goals of scientific research is to provide a description of the empirical data which is as accurate and comprehensive as possible, while relying on as few and simple assumptions as possible. In this paper, I propose a definition of the notion of few and simple assumptions that is not affected by known problems. This leads to the introduction of a simple model of scientific progress that is based only on empirical accuracy and conciseness. An essential point in this task is the understanding of the role played by measurability in the formulation of a scientific theory. This is the key to prevent artificially concise formulations. The model is confronted here with many possible objections and with challenging cases of real progress. Although I cannot exclude that the model might have some limitations, it includes all the cases of genuine progress examined here, and no spurious one. In this model, I stress the role of the state of the art, which is the collection of all the theories that are the only legitimate source of scientific predictions. Progress is a global upgrade of the state of the art.

I argue that the key to understand many fundamental issues in philosophy of science lies in understanding the subtle relation between the non-empirical cognitive values used in science and the constraints imposed by measurability. In fact, although we are not able to fix the interpretation of a scientific theory through its formulation, I show that measurability puts constraints that can at least exclude some implausible interpretations. This turns out to be enough to define at least one cognitive value that is able to penalize, without damages, precisely those bad features that deceive purely empirical assessments. This leads to the formulation of a simple model of scientific progress, that is based only on empirical accuracy and conciseness. The model is confronted here with many possible objections and with challenging cases of real progress. Although I cannot exclude that the model might be incomplete, it includes all the cases of genuine progress examined here, and no spurious one. In this model, I stress the role of the state of the art, which is the collection of all the theories that are the only legitimate source of scientific predictions. Progress is a global upgrade of the state of the art.

Simple assumptions represent a decisive reason to prefer one theory to another in everyday scientific praxis. But this praxis has little philosophical justification, since there exist many notions of simplicity, and those that can be defined precisely strongly depend on the language in which the theory is formulated. The language dependence is a natural feature—to some extent—but it is also believed to be a fatal problem, because, according to a common general argument, the simplicity of a theory is always trivial in a suitably chosen language. But, this trivialization argument is typically either applied to toy-models of scientific theories or applied with little regard for the empirical content of the theory. This paper shows that the trivialization argument fails, when one considers realistic theories and requires their empirical content to be preserved . In fact, the concepts that enable a very simple formulation, are not necessarily measurable, in general. Moreover, the inspection of a theory describing a chaotic billiard shows that precisely those concepts that naturally make the theory extremely simple are provably not measurable. This suggests that—whenever a theory possesses sufficiently complex consequences—the constraint of measurability prevents too simple formulations in any language. This explains why the scientists often regard their assessments of simplicity as largely unambiguous. In order to reveal a cultural bias in the scientists’ assessment, one should explicitly identify different characterizations of simplicity of the assumptions that lead to different theory selections. General arguments are not sufficient