For all practical purposes, it is impossible to ‘prove’ that something is ‘true’. This does not allow you to go out and rob a bank, for example. An inability on the part of the prosecution to establish the precise moment of the robbery, for instance, would not excuse your action in a court of law. We cope in the world by recognizing that there is a substantial amount of inexactitude, or ‘error’, in all that we do and consider. This does not mean we accept a shop‐keeper giving us the wrong change, or worry that a train timetable gives us information only to the nearest minute when the train might arrive before (or more commonly after) the advertised time. There are lots of ‘gross’ measurements that are good enough for us to plan our days without having to worry about the microdetail of all that we see and do.
For example, it is very difficult to describe an individual’s ‘usual’ intake of vitamin C, or to relate that person’s intake of vitamin C to risk of stroke. On the other hand, if we can accumulate sufficient evidence from many observations to show that increasing levels of usual vitamin C intake are associated with reduced risk of stroke (allowing for measurement error in assessing vitamin C intake and the diagnosis of particular types of stroke), it helps us to understand that we can work with imprecise observations and laws which are not immutable. Moreover, it is important (for the sake of the growth of scientific knowledge) that any belief which we hold is formulated in a statement in such a way as to make it possible to test whether or not that statement is true. Statements which convey great certainty about the world but which cannot be tested will do nothing to improve our scientific understanding of the world. The purpose of this book, therefore, is to learn how to design studies which allow beliefs to be tested, and how to cope with the imprecision and variation inherent in all measurements when both collecting and analyzing data.
1.2 LOGIC
In science, we rely on logic to interpret our observations. Our aim is usually to draw a conclusion about the ‘truth’ according to how ‘strong’ we think our evidence is. The type of observations we choose to collect, and how we collect them, is the focus of research design: ‘How are we going to collect the information we need to test our belief?’ The decision about whether evidence is ‘strong’ or ‘weak’ is the province of statistics: ‘Is there good evidence that our ideas are correct?’ As Sherlock Holmes put it, ‘It is a capital mistake to theorise before one has data’.3
There are two types of logic commonly applied to experience.
1.2.1 Inductive Logic
The aim with inductive logic is to infer a general law from particular instances: arguing from the particular to the general. This type of logic is good for generating new ideas about what we think might be true. It is less good for testing ideas about what we think is true.
Examples of Research Designs that Depend on Inductive Logic
Case studies provide a single example of what is believed to be true. The example is so compelling by itself that it is used to infer that the particular instance described may be generally true. For example:
A dietitian treating a severely underweight teenage girl worked with a psychotherapist and the girl’s family to create an understanding of both the physiological and psychological basis and consequences of the disordered eating, resulting in a return to normal weight within six months. The approach contained unique elements not previously combined, and could be expected to have widespread benefit for similar patients.
A case study can be interesting and provide a powerful example. But it provides very limited evidence of the general truth of the observation.
Descriptive studies bring together evidence from a number of related observations that demonstrate repeatability in the evidence. For example:
In four old peoples' homes, improved dining environments using baffles to reduce noise interference and allowing more time for staff to take orders and serve meals resulted in improved nutritional status among residents after one year.
This type of cross‐sectional evidence from numerous homes is better than evidence from a single home or a case study.
The generalizable conclusion, however, depends on a number of factors that might also need to be taken into account: what was the turnover among residents – did new residents have better nutritional status when they arrived, were they younger with better appetites, did they have better hearing so that they could understand more clearly what options were available on the menu for that day, etc.? One of the difficulties with descriptive studies is that we may not always be comparing like with like. We would have to collect information to demonstrate that apart from differences in noise levels and serving times, there were no other differences which could account for the change in nutritional status. We would also want to know if the circumstances in the four selected care homes were generalizable to other care homes with a similar population of residents.
Experimental studies are designed to assess the effect of a particular influence (exposure) on a particular outcome. Other variables which might affect the outcome are assumed to be held constant (or as constant as possible) during the period of evaluation.
Establish if a liquid iron preparation is effective in treating anaemia.
If an influence produces consistent effects in a chosen group of subjects, we are tempted to conclude that the same influences would have similar effects in all subjects with similar characteristics. When we evaluated the results from our observations, we would try to ensure that other factors which might affect the outcome (age, sex, dietary iron intake, dietary inhibitors of iron absorption, etc.) were taken into account.
1.2.2 Deductive Logic
Deductive logic argues from the general to the particular. This type of logic involves a priori reasoning. This means that we think we know the outcome of our observations or experiment even before we start. What is true generally for the population4 will be true for each individual within the population. Here is a simple example:
All animals die.
My dog is an animal.
My dog will die.
This type of logic is very powerful for testing to see if our ideas are ‘true’. The logic is: if ‘a’ is true, then ‘b’ will be the outcome. If the evidence is robust (i.e. as good a measure as we can get, given the limitations of our measuring instruments) and shows a clear relationship, it should stand up to criticism. And as we shall see, it provides the basis for the statistical inferences based on the tests described in later chapters.
There is a problem, however. The example above about my dog is relatively simple and straightforward. We can define and measure what we mean by an ‘animal’, and we can define and measure what we mean by ‘death’. But suppose we want to understand the impact of vitamin A supplementation on risk of morbidity and blindness from measles in children aged 1 to 5 years living in areas where vitamin A deficiency is endemic. Defining and measuring variables in complex biological systems is much harder (particularly in the field of nutrition and dietetics). It becomes harder to argue that what is true generally for the population will necessarily be true for each individual within the population. This is for two reasons. First, we cannot measure all the factors that link ‘a’ (vitamin A deficiency) and ‘b’ (morbidity and blindness from measles) with perfect accuracy. Second, individuals within a population will vary from one to the next in terms of their susceptibility to infection (for a wide range of reasons) and the consequent impact of vitamin A supplementation.
For deductive logic