Vitalism and Machine Learning

There's a methodological mistake, often made in the early stages of a science. It was pointed out in 1865, relative to biology, by Claude Bernard, in his influential An Introduction to the Study of Experimental Medicine. 

Bernard wrote that we tend to over-simplify explanations. It's natural for us. In some sense, science would be impossible without our ability to guess at simple theories, as Charles Sanders Peirce pointed out, describing his notion of abductive reasoning. This is because science is about understanding and improving theories, and they need to be comprehensible, or else they can hardly be tested and improved. It's a process that makes heavy use of our intuition.

But we go overboard, and rush to judgement, quite often, ignoring Newton's lesson to us: frame no hypothesis. By which he meant that he could not find an explanation for gravity, and so he didn't try to invent one -- he only provided a description, and this is where we are, still, today, with our theories of natural forces.

In an objective sense, science is about constructing comprehensible theories of the world, not comprehending the world itself. But we still want to understand the world itself. This is a desire for the feeling of understanding, not for the much more unnatural form of understanding in science: in the form of a theory that inspires experimentation and helps to uncover enlightening principles.

For Claude Bernard, vitalism was a good example of this. The notion of a vital force was already not a practical concern to researchers by 1865. It had recently been understood that much biological physiology could be explained by analysis and experiment, including the chemical analysis of organic tissues in the hot field of organic chemistry. Not everything could be explained, but some things could be, and much could be described -- much more than might be expected. The idea that there is a force like gravity, specific to living things, wasn't needed. 

In fact, vitalism is too simple to be correct -- in a way, it's disrespectful of nature. It's a trivial explanation based on our woefully limited understanding -- driven by our natural instincts, which are capable of distinguishing life from the inorganic, since we are living organisms ourselves.

I'll paraphrase the case Bernard makes: imagine someone sees a bird, lifting off the ground. Without further evidence, the person could easily explain that birds are suffused with 'anti-gravity substances'. But the bird's ability turns out to be layers of complex interactions between special qualities of birds and little understood physical-chemical laws. It's very complex. There's no simple explanation. That's why it's biology and not physics: physics deals with phenomena that are 'easy' to isolate, and more complex problems are tossed downstream to the chemists and biologists.

That said, there was a vital force, if you stretch the definition of force beyond the physicists' technical terminology: the teleology-like mechanism of genetics. But that guiding hand is not like gravity, which is what the 19th-century vitalists were looking for. 

So it was quite reasonable for Bernard to claim vitalism to be counter-productive. It was better to say "we don't yet understand why living creatures do certain things, like grow, reproduce, form living shapes, etc". Even in 2013, we still can't completely characterize the difference between living things and inorganic matter, let alone explain the phenomena -- but that doesn't mean we should explain away our ignorance, with something that sounds real, like a 'vital force'. Real science is a much more humble affair: we have to admit that we don't have many answers to very serious and important questions. And there's a corollary: Science must be opposed to Scientism, the offhand speculative explanation of things 'scientifically', which is comically prevalent, and exactly as irrational as relying on angels and spirits to explain features of the natural world.

Bernard's book also pointed to a mistake being made by young researchers, a different kind of 'too simple explanation', which was: "it's all chemistry". 

As Elizabeth Gasking pointed out in 1970, in The Rise of Experimental Biology, for many decades after Bernard's book, medical investigators ignored their own ignorance of biological processes, in favor of simplistic chemical explanations. Bernard's book tried to counter this by presenting the influential idea of the 'internal environment' of the organism, what we would now call its physiology. The point was that this internal environment was so poorly understood that the consequences of chemical interventions are very difficult to understand. This advice, now critically important to the biological sciences, is stilled ignored by, for example, the propaganda-driven pharmaceutical industry.

Let's take another common mistaken simplification, this time in our approach to the study of the human mind. 

It's common to assume that the mind is 'plastic', a tabula rasa with no a priori structure, which has a magical statistically-empowered general-learning mechanism which, somehow, always develops into the mind of a human being. But it's very unlikely that any creature, produced in this way, would be able to survive in the world, let alone be identifiable as the same species, capable of communicating with its own kind, etc.

The 'plastic' mistake is sometimes called behaviorism, empiricism, network-learning, or associationism. But really, it's very like vitalism, in that behaviorists are trying to find a simple, not useful, not verifiable, irrational 'force' or mechanism that 'does everything' or 'explains everything'. It didn't work, and it's completely discredited in biology. But it still hangs on, mostly in engineering, because it is easier to program based on this discredited model, than it is to program based on the unknown processes of the actual mind ... there are in fact many ways one can make a machine 'learn' in this way, some of which are useful for making products. But these techniques are not part of the natural sciences, and their employers shouldn't pretend that they are, as many do in modern technology companies.

Computer engineers should heed Bernard's lesson from biology: over-simplification of complex living systems is bound to get you in trouble. Stay aware of important mysteries. We need to keep it clear: techniques for machine learning are very distinct from the unknown operation of the human mind. This should be a basic tenet of computer engineering, like the distinction between formal and natural languages. Machine-learning techniques have nothing to do with human intelligence ... except that, of course, human intelligence invented these ideas, a point that offers a much more important insight into actual human cognition than any machine-learning technique itself.