Lecture
Let us consider the procedure for establishing a fact in empirical research by analogy with the procedure of a jury trial, to which Popper appeals.
Two criteria: the analysis of evidence and the presumed knowledge of an ideal Observer about how things actually were. Behind this criterion lies the problem associated with the correspondence theory of truth. According to this conception, truth can be established precisely by evaluating the degree of correspondence of the observed state of affairs to the factual one. But the factual state of affairs may be unknown.
|
Observer (objective state of affairs) |
Evidence in favor of the crime having been committed |
No evidence (evidence against) |
|
The person is innocent |
+ - Verdict «guilty» |
- - Verdict «not guilty |
|
The person is guilty |
++ Verdict «guilty» |
- + Verdict «not guilty» |
Two errors are possible.
A similar mode of reasoning characterizes, in scientific research, the decision that one or another fact has taken place. The researcher obtains facts under field conditions or in a specially organized methodological procedure, where the empirical data can be considered by analogy with evidence. The data may indicate that the fact has taken place – a positive effect; or that the effect is negative. In both cases an error is possible. An artifact having a different origin may be recognized as a fact. And, conversely, an actual fact may be missed or turn out to be obscured by noise.
Facts are established, i.e. they are procedurally mediated by reflection upon the empirical evidence rather than given to the researcher directly – and this distinguishes research employing any scientific method. Critical thinking is the foundation of any scientific method, including the experiment.
A conclusion about the experimental effect (fact) can be made on the basis of establishing a significant relationship between changes in the IV and the DV (through the use of measures of association), as well as by establishing significant differences in the DV between the experimental and control conditions (measures of difference).
If measures of association are chosen, then decisions about the choice of correlation coefficient are further required. If measures of difference are chosen, a number of decisions are required about their correspondence to the data-collection plan and to the type of DV indicators.
When establishing relationships between variables measured on different scales, decisions about their transformations are required. These and other decisions are made on the basis of substantive transitions from the aims of the study to the search for procedures corresponding to the establishment of the necessary psychological scales and the methods of quantitative evaluation of the obtained effects.
Statistical conclusions about significant differences make it possible to establish only the fact of changes in the DV. This is the first step toward a conclusion about the result of the action of the IV. The second step is the substantiation that the difference in the conditions of the IV was experimentally controlled.
Aspects of decision-making:
- do the managed differences actually function as causally acting conditions?
- a multifaceted evaluation of validity (construct, operational, internal).
Statistical decisions about rejecting null hypotheses are one of the stages in arriving at reliable conclusions about the established relationship. The magnitude of the obtained difference is here evaluated from the standpoint of the assumed minimal effect (at the given significance level and for the given sample size). The experimental effect must exceed this minimal value. The fact of a statistical evaluation does not mean that the study is necessarily quantitative.
The two principal classes of effect-size indicators are the r-class and the d-class of indicators. These indices make it possible to bring to a common metric the results obtained in studies using different methods.
The choice of metric depends on the scheme and hypotheses of the study.
Medium effects: d=0.50; r=0.30
This means that in the study a difference was obtained between the means of two groups amounting to half a standard deviation, or that two variables share about 9% of their variance.
Comments