“All models are wrong but some are useful” George Box
The origin : decision making
What do we really do when we face seriously a difficult decision? Look a this note about the French posture on varicella vaccination “Faut-il vacciner les nourrissons contre la varicelle ? De la difficulté de la décision vaccinale”. It gives a very good example will all the good ingredients:
- Several narratives possible : “leave population exposed to the virus” vs. “vaccination for all”.
- High stakes : hundreds of victims per year for each narrative.
- Uncertainties : long term vaccines effects ; large scale acceptation of vaccination.
Concerning health care matters, most information is open to the public. You can read the full story from the bibliography of french vaccination info service. In this case, the discussions are backed by models such as the one of annex 6.2 of the report on varicella vaccine recommendation. A glimpse on the document, and you will probably classify it in the box “scientific result (read only if forced to)”.
Why, facing hard decisions, our communities can ask for a scientific approach? What constraints does it implies? How is it related to though decision making in the end?
Scientific computing constraints
Knowing this context, how can we define Scientific computing? We will introduce two key concepts:Reproducibility and Fasifiability. Both concepts were discussed by the philosopher of science Karl Popper.
Reproducibility
If you aim for a repeatable scientific computation, there is more than the eyes meet :
- for any configuration similar to yours…
- …anyone other than you, can reproduce your results…
- …without your supervision…
- …using any equivalent resource (hardware and software)
For this reason, people in academia are first trained for years at narrowing real-life problems to simple ones. In fluid dynamics for example, plenty of litterature look at the the backward facing step or infinite cylinder wakes. (With a former co-worker J. Favier, we reduced a Cephalocereus senilis (old man cactus) to an infinite cylinder here.)
By reducing the problem, and stating the method used with enough precision, academics makes their work reproducible.
In the model of the vaccination report, the author goes beyond results and shows:
- the references used to create the mathematical model
- the exact mathematical model
- the sensibility of the model, and how it changes the conclusion.
- the calibration of the model, and where did the parameters came from.
Falsifiability
This second trait is less intuitive, and is a cornerstone in Karl Popper ideas. Leaving aside the philosophical aspects, there is a very practical angle : it make the work more credible. Peer-review are built on this. What is a peer-review, if not a pure refutability debate? These can bring bad news, as these examples show:
- You proved the computation with your new model agrees with the experiment. If you disable your model, does it disagree ?
- The precision gains you claimed is smaller than what is observable in the experiments. If so, explain why such precision matters?
- The result of your advanced model is presented with extremely accurate inputs. Show the impact of less accurate inputs.
In a scientific approach, one discusses early how to refute its own computation.
To illustrate this, in the report on varicella vaccine recommendation, the author shows how her results compares with prior independent observations at the national level (Table 1). She also focuses on the missing data that could change results values or certainties.
Usage in a narrative: the last step
A scientific argument alone cannot change minds. The population resilience to accept and react to climate change makes a world-scale evidence. The narrative is backed by an incredibly large scientific consensus. Yet it is unable to move opinions completely.
When a community must take a decision, it select what it trust to be the best narrative. From a brief rhetorical point of view, the modes of persuasion are made of logic (logos), credibility (ethos), and emotions (pathos). Using this decomposition, a scientific result can positively weight on logic, and a bit on credibility. However, the cost of the scientific work, the opacity of scientific jargon, and the nuances can weight negatively on both credibility and emotions.
This is why a scientific work need a last push in the back : it must be correctly inserted into an convincing narrative.
In the report on varicella vaccine recommendation, the author states the narrative in her introduction, answering many context questions like “why making the study now?”, or “what is the initial official vaccine recommendation?”. Then it is inserted in a larger discussion fed by many other actors which are all the contributors to the final report.
Takeaway
Scientific computing is a valuable asset to weight on a decision. It requires:
- A good Reproducibility : does anyone else could reach the same result without you?
- A proper Falsifiability : did you really try to challenge your result?
- A convincing narrative : what is the problem at stake and what did we learn about it? (and cut the scientific crap, please!)