Habits for any natural science of people

The natural sciences have explored 'people' as a subject since their beginnings. But this has proven to be extremely difficult to do sufficiently objectively for good results. 

'Difficult' is not the same as 'impossible', however.

Of course, complete objectivity is impossible ... but improved objectivity is very possible. Huge philosophical movements have denied this distinction, irrationally. The repercussions are still everywhere today.

I believe an important point needs to be better understood, and more explicitly understood, throughout any kind of human science: complex human features must be studied through the use of informants. 

This has been the core of the new linguistics, for example. If everyone agrees on the grammaticality of something, it's not because grammar has been written down (it hasn't) and they have memorized it (they haven't), but instead because this complex faculty that evaluates grammar grows, with the same, very limited environmental stimulus, to be essentially equivalent in everyone. Otherwise we couldn't understand one another. To study this phenomenon, we simply need to accept that the language faculty is much like the stomach: primarily a genetic endowment.

What the stomach does is complex, but you can begin to know more about the stomach by thinking about it, guessing at its operation, and testing its complex reaction to situations that are carefully constructed to give interesting answers: experiments. All those steps are required for real science.

We do the same thing in studying language. If I remove the "in" from the previous sentence, we can all agree that it is still a grammatical sentence, and its meaning hasn't much changed. But if I remove "same", it is grammatical, but the meaning has changed. If I remove "do" it is no longer grammatical, but the meaning can be guessed at. These experiments tell us something about our language faculty, using this complex instinct as the experimental subject, and ourselves as informants.

To do this, one must respect the informant, that is, the experimental subject. One must, because they are helping you to understand some feature of the complex, almost-completely-mysterious human brain. We know vanishingly little about the brain in any animal, even less about our own, and even less about its very complex features.

That means that to explore and understand more about the mechanism of some very complex human capacity, say our ability to 'program smoothly', one must be very cautious to examine situations that are raised about subjective human impressions in various situations. You must listen to the experimental subject. Their subjective impression is your experimental subject. It will help you to identify and isolate some human capacity, if you listen carefully, get them to explain what they can, and try to make it repeatable, and testable, in various ways.

That's how science works. Physicists don't ignore a lab assistant who says "we have a recurring, completely unexpected result". They don't say "oh, we understand everything" and fire the assistant and smash the laboratory's experimental apparatus. I mean, this could happen, but no one would think it good. When it's very early days in the theoretical development, they especially don't do this.

On the contrary, in computer science, this set of habits has simply not developed at all -- outside of, in a very limited way, user interface design. Leading computer scientists have no time for such subtlety. 

As an example, let me describe something that happened to me recently. I was having lunch with one of the major programming leaders of our time. I said that I had identified a new phenomenon, and I needed to show it to him, so he could experience it. He refused to look at it, feeling that he already understood it, even though, based on his description, I could tell he was talking about something related, but not sufficiently isolated. Now, any psychologist, or even a user experience designer, would have been interested. But because he felt this was territory he'd already been over, he didn't want to see the results. Even though we were talking about extremely complex and poorly understood human capacities.

This kind of indifference is rife throughout the computer world, though not universal. I've met the leaders of many movements in computing, who, one would think, would be very sensitive to human informants. When doing their own work, they often are. But I find most of them to be simply unaware of what we've learned in the last 50 years about the importance of informants for studying complex human phenomena, and the importance of sharing informant results. They believe that the relatively minor improvements in programming are more important, despite the fact that there's tremendous dissatisfaction and disagreement in the engineering world today … a situation which might be improved by a bit of natural science.

All of this might be partly because computing developed to respond immediately to the "drive to build". Especially the building of larger and more complex logical systems. The sensitivity of programmers who feel that something is wrong is ignored, even though a thorough, sensitive examination of such feelings will likely be the basis for moving forward to better scientific understanding of complex human capacities, and better tools for programming. We're in a state right now where a few programming movements feel they've found "the answer", and people are simply exhorted to "get it". While such attitudes are expedient when building a programming team, or even a programmer's movement, they are antithetical to science.