Lecture
An essential element of planning is the method of forming groups. The most rigorous criterion requires randomization, a random order of forming the groups.
Inter-individual differences, or differences between subjects and inequality between groups, are the main source of threats to internal validity in between-group designs.
Increasing the sample size is sometimes regarded as a way of controlling non-systematic confounds. But this does not solve the problem of the non-equivalence of samples, because:
- even with randomization, people who are «similar» in one property may end up in different groups;
- with large n, the law of large numbers comes into play – the more participants there are in the experiment, the greater the probability that side variables (SV) will manifest in the differences (i.e., the more confounds there are).
Types of between-group designs (comparison schemes):
1. A simple design for an experimental and a control group without pretesting.
Randomization – R – here acts as a strategy for assigning subjects to groups from the population; it allows the groups to be regarded as equivalent and the difference in the dependent variable (DV) to be attributed precisely to the action of the independent variable (IV).
A more complex variant is the randomized block design. It is used when subgroups are formed within each group.
Equating the experimental and control groups with respect to confounding with block side variables («knowledge about the experiment», pupils from different classes, and so on) can be achieved by assigning subjects from each block to the experimental and control groups in random order.
Pretesting partially solves the problem of accounting for and controlling the initial level of the DV, but the non-equivalence of the groups may in this case remain hidden in unmeasured variables.
2. A design with pretesting and posttesting and a control group.
Such designs may have insufficient external validity.
3. The Solomon design.
A comparison of four groups: two experimental and two control, with the introduction of the factor «presence or absence of pretesting».
Strategies for the selection and matching of subjects control for the factor of inter-individual differences.
The aspect of the representativeness of subjects concerns external validity; the equivalence of groups concerns internal validity.
Matching occurs when subjects are not selected from a population but distributed among groups from an already existing pool of subjects.
That is, here the condition of randomized selection is not met.
4. The strategy of stratified selection or matching of subjects (the strategy of random distribution of strata).
1) Forming groups corresponding to the identified characteristics of the strata (sex, age, and so on). There will be as many such groups as there are levels distinguished according to variations of the side variable.
2) Subjects are selected at random from each stratum into the experimental and control groups. There will be as many experimental groups as there are levels representing the IV. As a result, in each of the selected (or matched) groups, every level of the side variable is equally represented.
This strategy is often used to control an additional variable.
5. The strategy of pairwise equating.
It is applied in the case of matching, when the sample of subjects has already been determined and they can be subjected to pretesting.
1) Identify the pair of subjects with the most pronounced values of the side variable (and then those with less pronounced values, then still less pronounced ones, and so on).
2) Distribute the subjects of each pair between the two groups according to a random strategy or an even–odd strategy.
This strategy is applied mainly to small samples.
6. Random selection of groups.
It is applied when approximating the experiment of full correspondence entails implementing the X-treatments under real conditions of life activity. For example, comparing different teaching methods.
Sample representativeness is the main aspect of assessing population validity.
A sample of subjects drawn from a population cannot represent the latter flawlessly, but high representativeness can be achieved with correct strategies for selecting subjects.
Selection strategies in between-group designs include randomization (random selection to create equivalent groups) and random assignment to groups with prior distinction of strata. Representativeness is also assessed in terms of accounting for the motivation of the subjects that determined their participation in the psychological experiment (the motivation of volunteers, and so on).
Comments