One property of Bayes’s theorem, in fact, is that our beliefs should converge toward one another—and toward the truth—as we are presented with more evidence over time. In figure 8-8, I’ve worked out an example wherein three investors are trying to determine whether they are in a bull market or a bear market. They start out with very different beliefs about this—one of them is optimistic, and believes there’s a 90 percent chance of a bull market from the outset, while another one is bearish and says there’s just a 10 percent chance. Every time the market goes up, the investors become a little more bullish relative to their prior, while every time it goes down the reverse occurs. However, I set the simulation up such that, although the fluctuations are random on a day-to-day basis, the market increases 60 percent of the time over the long run. Although it is a bumpy road, eventually all the investors correctly determine that they are in a bull market with almost (although not exactly, of course) 100 percent certainty.

In theory, science should work this way. The notion of scientific consensus is tricky, but the idea is that the opinion of the scientific community converges toward the truth as ideas are debated and new evidence is uncovered. Just as in the stock market, the steps are not always forward or smooth. The scientific community is often too conservative about adapting its paradigms to new evidence,64 although there have certainly also been times when it was too quick to jump on the bandwagon. Still, provided that everyone is on the Bayesian train,* even incorrect beliefs and quite wrong priors are revised toward the truth in the end.

Some children would have waited until after their first trip to Diagon Alley.

"Bag of element 79," Harry said, and withdrew his hand, empty, from the mokeskin pouch.

Most children would have at least waited to get their wands first.

"Bag of okane," said Harry. The heavy bag of gold popped up into his hand.

Harry withdrew the bag, then plunged it again into the mokeskin pouch. He took out his hand, put it back in, and said, "Bag of tokens of economic exchange." That time his hand came out empty.

"Give me back the bag that I just put in." Out came the bag of gold once more.

Harry James Potter-Evans-Verres had gotten his hands on at least one magical item. Why wait?

"Professor McGonagall," Harry said to the bemused witch strolling beside him, "can you give me two words, one word for gold, and one word for something else that isn't money, in a language that I wouldn't know? But don't tell me which is which."

"Ahava and zahav," said Professor McGonagall. "That's Hebrew, and the other word means love."

"Thank you, Professor. Bag of ahava." Empty.

"Bag of zahav." And it popped up into his hand.

"Zahav is gold?" Harry questioned, and Professor McGonagall nodded.

Harry thought over his collected experimental data. It was only the most crude and preliminary sort of effort, but it was enough to support at least one conclusion:

"Aaaaaaarrrgh this doesn't make any sense! "

The witch beside him lifted a lofty eyebrow. "Problems, Mr. Potter?"

"I just falsified every single hypothesis I had! How can it know that 'bag of 115 Galleons' is okay but not 'bag of 90 plus 25 Galleons'? It can count but it can't add? It can understand nouns, but not some noun phrases that mean the same thing? The person who made this probably didn't speak Japanese and I don't speak any Hebrew, so it's not using theirknowledge, and it's not using my knowledge -" Harry waved a hand helplessly. "The rules seem sorta consistent but they don't mean anything! I'm not even going to ask how a pouchends up with voice recognition and natural language understanding when the best Artificial Intelligence programmers can't get the fastest supercomputers to do it after thirty-five years of hard work," Harry gasped for breath, "but what is going on? "

"Magic," said Professor McGonagall.

"That's just a word! Even after you tell me that, I can't make any new predictions! It's exactly like saying 'phlogiston' or 'elan vital' or 'emergence' or 'complexity'!"

When we think of the scientific method, we tend to think of an experimenter in his laboratory, probably holding a test tube and wearing a white coat, who follows a series of steps that runs something like this: make some observations about a phenomenon; create a hypothesis to explain those observations; design an experiment to test the hypothesis; run the experiment; see if the results match your expectations; rework your hypothesis if you must; lather, rinse, and repeat. Simple seeming enough. But how to go beyond that? Can we train our minds to work like that automatically, all the time?

Holmes recommends we start with the basics. As he says in our first meeting with him, “Before turning to those moral and mental aspects of the matter which present the greatest difficulties, let the enquirer begin by mastering more elementary problems.” The scientific method begins with the most mundane seeming of things: observation. Before you even begin to ask the questions that will define the investigation of a crime, a scientific experiment, or a decision as apparently simple as whether or not to invite a certain friend to dinner, you must first explore the essential groundwork. It’s not for nothing that Holmes calls the foundations of his inquiry “elementary.” For, that is precisely what they are, the very basis of how something works and what makes it what it is.

And that is something that not even every scientist acknowledges outright, so ingrained is it in his way of thinking. When a physicist dreams up a new experiment or a biologist decides to test the properties of a newly isolated compound, he doesn’t always realize that his specific question, his approach, his hypothesis, his very view of what he is doing would be impossible without the elemental knowledge at his disposal, that he has built up over the years. Indeed, he may have a hard time telling you from where exactly he got the idea for a study —and why he first thought it would make sense.

The first man who said 'fire burns' was employing scientific method, at any rate if he had allowed himself to b e burnt several times. This man had already passed through the two stages of observation and generalization. He had not, however, what scientific technique demands—a careful choice of significant facts on the one hand, and, on the other hand, various means of arriving at laws otherwise than my mere generalization.

Measurement has too often been the leitmotif of many investigations rather than the experimental examination of hypotheses. Mounds of data are collected, which are statistically decorous and methodologically unimpeachable, but conclusions are often trivial and rarely useful in decision making. This results from an overly rigorous control of an insignificant variable and a widespread deficiency in the framing of pertinent questions. Investigators seem to have settled for what is measurable instead of measuring what they would really like to know.

The scientific method is a potentiation of common sense, exercised with a specially firm determination not to persist in error if any exertion of hand or mind can deliver us from it. Like other exploratory processes, it can be resolved into a dialogue between fact and fancy, the actual and the possible; between what could be true and what is in fact the case. The purpose of scientific enquiry is not to compile an inventory of factual information, nor to build up a totalitarian world picture of Natural Laws in which every event that is not compulsory is forbidden. We should think of it rather as a logically articulated structure of justifiable beliefs about nature. It begins as a story about a Possible World—a story which we invent and criticise and modify as we go along, so that it ends by being, as nearly as we can make it, a story about real life.

It is often held that scientific hypotheses are constructed, and are to be constructed, only after a detailed weighing of all possible evidence bearing on the matter, and that then and only then may one consider, and still only tentatively, any hypotheses. This traditional view however, is largely incorrect, for not only is it absurdly impossible of application, but it is contradicted by the history of the development of any scientific theory. What happens in practice is that by intuitive insight, or other inexplicable inspiration, the theorist decides that certain features seem to him more important than others and capable of explanation by certain hypotheses. Then basing his study on these hypotheses the attempt is made to deduce their consequences. The successful pioneer of theoretical science is he whose intuitions yield hypotheses on which satisfactory theories can be built, and conversely for the unsuccessful (as judged from a purely scientific standpoint).

We have three approaches at our disposal: the observation of nature, reflection, and experimentation. Observation serves to assemble the data, reflection to synthesise them and experimentation to test the results of the synthesis. The observation of nature must be assiduous, just as reflection must be profound, and experimentation accurate. These three approaches are rarely found together, which explains why creative geniuses are so rare.

Natural science is founded on minute critical views of the general order of events taking place upon our globe, corrected, enlarged, or exalted by experiments, in which the agents concerned are placed under new circumstances, and their diversified properties separately examined. The body of natural science, then, consists of facts; is analogy,—the relation of resemblance of facts by which its different parts are connected, arranged, and employed, either for popular use, or for new speculative improvements.

In reality, scientists are just people like you and me. Most of us don't wear lab coats (I don't) or work with bubbling beakers or sparking van de Graf generators (unless they are chemists or physicists who actually work with that equipment). Most scientists are not geniuses either. It is true that, on average, scientists tend to be better educated than the typical person on the street, but that education is a necessity to learn all the information that allows a scientist to make discoveries. Still, there are geniuses, like Thomas Edison, who had minimal education (he only attended school for a few months) but a natural instinct for invention. So education is not always required if you have talent to compensate. Scientists are not inherently good or evil nor are they trying to create Frankensteins, invent the next superweapon, or tamper with the operations of nature. Most are ordinary people who have an interest and curiosity to solve some problem in nature and rarely do they discover anything that might threaten humanity.

Scientists are not characterized by who they are or what they wear, but what they do and how they do it. As Carl Sagan put it, "Science is a way of thinking much more than it is a body of knowledge." Scientists are defined not by their lab equipment but by the tools and assumptions they use to understand nature—the scientific method. The scientific method is mentioned even in elementary school science classes, yet most of the public still doesn't understand it (possibly because the mad scientist Hollywood stereotype is more powerful than the bland material from school). The scientific method involves making observations about the natural world, then coming up with ideas or insights (hypotheses) to explain them. In that regard, the scientific method is similar to many other human endeavors, such as mythology and folk medicine, which observe something and try to come up with a story for it. But the big difference is that scientists must then test their hypotheses. They must try to find some additional observations or experiments that shoot their idea down {falsify it) or support it {corroborate it). If the observations falsify the hypothesis, then scientists must start over again with a new hypothesis, or recheck their observations and make sure that the falsification is correct. If the observations are consistent with the hypothesis, then it is corroborated, but it is not proven true. Instead, the scientific community must continue to keep looking for more observations to test the hypothesis further.

This is where the public most misunderstands the scientific method. As many philosophers of science (such as Karl Popper) have shown, this cycle of setting up, testing, and falsifying hypotheses is unending. Scientific hypotheses must always be tentative and subject to further testing and can never be regarded as finally true or proven. Science is not about finding final truth, only about testing and refining better and better hypotheses so these hypotheses approach what we think is true about the world. Any time scientists stop testing and trying to falsify their hypotheses, they also stop doing science.