It is worth considering the consequences of adopting the following cosmological viewpoint: that the universe is completely organized with, or composed of, overlapping fields of more or less coherent forces/structures, which we'll call centers, that have free will or self-determinism. They are actively struggling to resolve the intrinsic, extrinsic and interpenetrating influences within their purview, and they are actively, continually making choices.
Immediately the human idea of 'determinism' then needs to be reconsidered: free will is the cause of all effects, as every center works to resolve its own issues, to the best of its ability. Life and consciousness then seem to be merely those aggregates of these kinds of structures that have come to some kind of agreement. The whole universe is then full of life, and so there's no need to postulate a superphysical vital force. Also, there is then nothing particularly mysterious about coherent action at a distance, since this is simply a distant interaction. And there is little surprising about the wave-like behavior of particles, since these are living systems aware of their surroundings.
Think of how we struggle to make computers 'deterministic', which we call 'reliable', but these efforts at best provide a localized effect, isolated from influences that we subdue, in order to get results that we like. Making a 'deterministic' machine is in this way much like any goal. Our interest in making things deterministic, and seeing things in this way, is innate. Trying to control one's environment is a tool in our survival kit, albeit one with ambiguous effects.
But there's a cognitive extension of our innate concept of 'determinism', which makes no sense at all. It flies in the face of all evidence, and for some reason puzzles philosophers to this day.
It is the notion that the universe is a system of (conveniently dead, like a clock, and hence mentally 'comprehensible') matter (energy, forces, etc.) that simply moves from one state to the next state, and that these physical changes take place from the beginning of time to the end, and between any two points in time -- and these changes somehow exist, even though they are unknowable and don't yet exist. This expansive tortured fantasy, this intellectual construction, is the foundation of 'determinism'.
The philosopher's self-imposed dilemma is, if this definition of determinism is 'true': why do we feel that we have choices, that we have free will?
In medieval European theology, god knew this series of states ahead of time. It was destiny, fate: things were predestined to happen. But this gave medieval philosophers a headache: how then can we make any positive choice in the direction of morality and religious observation, when everything is already determined?
This idea has been a 'problem', for a few people, for thousands of years, and is now known as the incompatibility of determinism and free will. If events in our lives are hypothetically the sum of all forces impinging upon us, knowable or not, why are we able to do things at will?
Although the evidence of free will is everywhere, some thinkers even went so far as to say that free will was an illusion.
But it is the notion of determinism that does not withstand examination. It is a human fantasy. It is easily dispelled if we would simply admit that we are biological organisms. Which everyone does admit, but for some reason they don't like the consequences.
The universe is an impossibly vast sea of energies, small and large, conflicting and conflating, synthesizing and separating. If you tried your best to isolate two pieces of it, as similar as you could make them, you'd still find that the insanely frenetic forces and motions involved will produce two massively different results. Even atoms in a crystal are not 'lined up'. Everything is constantly pulling and pushing and moving, accumulating complex layers upon layers of structure that sometimes reach a kind of "stillness" that we call equilibrium, but which is very much not at rest (leading for example to our term homeostasis) in multifarious ways.
So, what on earth could determinism mean? Why would there be any issue about free will? Every little wave-particle of matter and bend in space-time seems to have free will!
The problem comes from our minds. And the concept causing the trouble, is time.
I sometimes imagine what would happen if I told my cat, "but, imagine you have a petri dish of all the energy in the universe, and then roll time backwards, if you let time roll forward again, and didn't interfere: the results would be the same." Or, the equivalent "The future is determined. So how do we have free will?"
The thing my cat would have trouble wrapping her brain around, is that I'm imagining control of time itself, something that neither I nor any other creature can ever do. And yet I use these fantasies to confuse myself. I'm sure she would tell me to try to get some rest, so tomorrow I'll be able to concentrate on reality.
(Similarly, my cat would be rather disappointed in my powers of observation, if I then asserted that Free Will was something that only human beings posessed.)
The problem of Free Will is only a problem when you say:
If everything is going a certain way, how do we have free will?
But, how can we say that it is "going a certain way"? Or that the consequence of the sea of forces has a "winner" or "loser" in each case? Or that there's a determined aspect to the future? That it is predictable in any way? It isn't. There is no projected future. We are made of those competing forces, in our case hundreds of trillions of forces directed by a central mechanism, and we will determine the future. Where is the issue?
In computing, we are always struggling to make things deterministic: repeatable behavior, using stochastic means, so that we can automate some agreed upon simulations of cognitive tasks. It barely works, but we've managed to impose our free will onto even more matter in this way. But the machine is not the universe. It's not a model of the universe. It's a model of something we want to get done.
The more we understand our place in the universe, the clearer it is that the determinism / free-will problem is simply a product of our imagination. Consciousness still needs to be understood, but the idea of determinism must be buried. There is no evidence for it, and never can be.
The very idea that "things will move forward the way they will" has nothing to do with reality, but with our attempts to grasp reality and cast it into a tangible model. The idea of determinism is an artifact of our minds. Even though we know full well that, in reality, forces are not resolved until they are resolved, at any moment (which I believe is the sense you get when you open your eyes) we still have this idea that "time only unfolds in one way" ... clearly an attempt to grasp the universe with a single principle. Something that we know simply cannot work.
We've cleared our working models of our limits from other similar attempts -- inner voices, omens and so on. We can speculate on the ways things may move forward, and this can be a useful survival skill, which is perhaps why we seem built to believe, and continue to believe, this strange idea that the universe has a destiny.
Essentially, when we say something like "all the forces in the universe, state A at time X, will result in State B at time X + i" we are doing a few things, none of which is supportable, as we jump into the determinist abstraction in our head, not in the real world: 1) we assume that we believe this, even though we know there will never be any evidence for it, 2) the evidence is, on the contrary, that billions of people, and maybe 10 to the 20th cells, as well as molecules and other forces, overlapping and interpenetrating, competing and cooperating, working with purpose and without, are actively working to change the future. The evidence that the totality of forces will simply, blindly resolve from A to B is unobtainable, and against all experience. It is an abstraction or notion without meaning, a model without evidence, not unlike the idea that, say, that there are undetectable demons overseeing the implementation of the laws of physics.
Every bit of energy in the universe has "skin in the game". We each happen to be primary managers of a rather sizable, coherent chunk of that omnipresent moxie. The future doesn't just happen. We all -- that is, we and our fellow clusters of energy wave-packets -- make the future happen. If we didn't, the universe would not exist.
Let's think about this in terms of an old argument, between the rationalists Leibniz and Arnaud. Leibniz postulated a principle of sufficient reason: everything has a cause, even if we don't know what it is. Consequently, we're splashing around in a sea of cause-and-effect that is deterministic: we're just not aware of all the causes, even our own. The Cartesian Arnauld, author of the Port-Royal Logic, said that in such a universe free will was not possible, yet free will is clearly something that we do all the time. This is no illusion. The resolution is that, yes the cause and effect is unknown, and yes, everything has free will, and the outcome is uncertain until it happens.
This view of a universe full of willing forces is very much like the fabric of the universe painted recently by architectural philosopher Christopher Alexander in the fourth book of The Nature of Order series, entitled "The Luminous Ground". It is also very close to the picture that Newton's implicitly created when he crushed Cartesian dualism and destroyed the notion of a Mechanical Universe, by admitting that there seemed to be no matter, only 'spirit' or 'mind' (that is force, or energy) acting without mechanical contact. Of this, Locke famously said "perhaps god has superimposed upon matter the quality of thought". So everything has free will. This also relates to the joke implied by Schrodinger's Cat gedanken experiment ... of course the cat is no less an observer than a human, and the same must be said of everything in the universe. I won't go back to the pre-socratic philosophers, but certainly Aristotle, in the metaphysics, postulates that all things are alive, and themselves, because of their structure, and that base matter itself did not exist. The number of philsophers supporting some form of this idea is endless, and includes Goethe, Kant, Darwin etc. So one wonders, give that this point-of-view fits the evidence better than its alternatives, why this notion has no official name?
To place this view: it's idealist, in the sense that determinism is just an idea, which may be derived from our innate capacities, and it's empiricist, in the sense that there is no evidence for determinism, and it's naturalistic, in the sense that the natural world provides no hint of determinism (which loops around to idealism).
In this view it is neither wrong nor right to believe that there is a vital force outside physics and chemistry that must explain how things emerge: it depends on how you look at physics and chemistry, which have both had implicit vital forces since Newton. Noam Chomsky pointed out that David Hume described Newton as "destroying the notion of a mechanical universe" because gravity and attraction cannot be explained mechanically: they are ghostly actions at a distance. Spirit (or mind, in Locke's speculation) is all there is. These are forces. There is no such thing as "dead matter" and the only reason we believe that this is true, is that "common sense" physics, of mechanics and active force vs. passive matter, that keeps reasserting itself in the minds of non-scientists. So that the press reported quantum physics as "destroying the mechanical model" when Newton had done so ages before.
Feynnmann opens his lectures with a simple, universal but unexplainable fact about nature: that forces tend to pull things together, until at some point they push apart. These are measurable and observable, but not explained. I can show you a simulation where this one principle begins to structure matter.
With thanks to Isaac Newton, for pointing out that the mechanical universe is impossible; to Charles Darwin for pointing out that creatures make their own way in the world; to Albert Einstein for illuminating our misconceptions regarding time; and to Wittgenstein, Chomsky and the Ordinary Language philosophers for pointing our that our deepest problems tend to be lodged in our brains, and in the way in which we express our thoughts.