25. Sources and control of the unreliability of data. The relationship between reliability and validity.

Lecture



One of the criteria for the objectivity of a study is its reproducibility.

The possibility of conducting repeated studies that yield the same phenomena and regularities. That is, the possibility of comparing data obtained for different participants or populations.

In this context, the concept of the reliability of data is introduced.

Reliable data, when obtained again under the same procedural conditions, produce insignificant deviations from the original values. This requires a probabilistic evaluation of the data on the basis of statistical decisions.

- The problem of the intersubjectivity of data, i.e., the possibility of obtaining the same data by different researchers.

The problem of the unreliability of the results of psychological research is directly related to the planning of the study's samples – their representativeness and the number of individuals (or observed cases) included in them.

1) The reliability of data is a condition for evaluating the validity of a study

The unreliability of data may be determined by the instability of psychological reality, by its being mediated through the subjective world of another person, by the unreliability of the methodological procedure, and so on.

The unreliability of data is their variability across repeated trials as a result of the non-systematic influence of extraneous factors and the variability of variables.

That is, the source of the unreliability of data is non-systematic confounding – when any of the extraneous variables, or their combinations, irregularly intrude into the dependence under study.

The source of an extraneous variable conditioned by the time factor may be either internal causes (changes in the participant's states) or external ones (accidental distraction by noise, a phone call, etc.).

To control such confoundings, one must bring the experiment closer to an infinite one – increasing the number of trials.

This reliability is related to the assessment of internal validity.

2) The reliability of methods – an aspect of operational validity

3) Reliability as the stability and reproducibility of results (an aspect of external validity).

Comments

To leave a comment

If you have any suggestion, idea, thanks or comment, feel free to write. We really value feedback and are glad to hear your opinion.
To reply

Lectures and tutorial on "Experimental psychology"

Terms: Experimental psychology

Psychological experiment
Knowledge by experiment
Deductive approach
Introduction to the psychology of the experiment
Manifest dreams
Natural experiment
Experimental research in psychology
Evaluation of the experimental method in psychology
Behavior of lower organisms
Multi-factor Experimenting Methodology
Correlative psychophysiology
100 Tests on the Topic of Professional Ethics, with Answers
Introduction to Experimental Psychology. Subject Matter and Tasks; History
1. The concept of the experiment as an active method of research. Experimentation in the broad and narrow sense of the word.
2. The psychological experiment and the natural-science experiment.
3. Norms in the structure of the experimental method.
4. Criteria of the objectivity of methods of psychological research.
5. Types of psychological hypotheses and the methods of research corresponding to them.
6. The three basic conditions for drawing a conclusion about a causal relationship. Requirements for the formulation of cause-and-effect hypotheses.
7. Levels of hypotheses tested in a psychological experiment; transitions between them.
8. Experimental control (as a condition for planning and conducting a psychological experiment); its types and means.
9. Types of confounding and forms of their control in a psychological experiment.
10. The conception of the experimental fact as the result of a decision.
11. The experimental method and the observation method in psychology.
12. Inductive laws. The place of induction in the experimental and qualitative methods in psychology.
13. The specificity of different types of psychological experiment.
14. Approaches to understanding the term "experimental planning". Substantive and formal planning.
15. Artefactual conclusions in psychological research.
16. Competing theories and the so-called third competing hypothesis.
17. Deductive inference by the modus tollens syllogism and the asymmetry of conclusions.
18. The thought experiment and mental models of experiments.
19. Classifications of experimental designs. Planning as the selection of designs.
20. Meta-analytic research in psychology.
21. Intra-individual schemes and their application (aims, sources of threats to validity, etc.).
22. The causal type of explanation and generalization. Reductionism in psychological explanations.
23. Requirements that must be met in order to draw reliable/valid conclusions.
24. Systematic confoundings as threats to internal validity
25. Sources and control of the unreliability of data. The relationship between reliability and validity.
26. Schemes of correlational research.
27. The relationship between external and internal validity in intra-individual and between-group schemes.
28. Between-group schemes. The main strategies for selecting and matching participants into groups.
29. Schemes for controlling accompanying confoundings.
30. Additional variables and additional variation in psychological research.
31. Quantitative experiments and quantitative laws in psychology.
32. The observation method in experimental psychology.
33. Types of empirical data in psychological research.
34. The specifics of experiments conducted for scientific and for practical purposes.
35. The «experimenter effect» and the problem of the identity of experimental conditions; Johnson's experiment.
36. The main quasi-experimental designs.
37. Complete and incomplete experimental designs.
38. Planning as a means of increasing the validity of an experiment.
39. The world of theories and psychological reality. The methodological approaches of Popper and Holzkamp.
40. Control of group composition in between-group designs. The problem of the representativeness of the subject and the sample.
41. Operationalization of variables, operational and construct validity
42. Artificial and field experiments in psychology
43. The specifics of the laboratory experiment in experimental psychology
44. Testing statistical hypotheses and its connection with the testing of experimental hypotheses.
45. Control over inference and types of generalization in psychological research.
46. Types of validity as applied to psychological research.
47. The demonstration experiment (study) and the conception of psychological law in Lewin's school
48. Cross-cultural research in psychology
49. The modeling approach in psychology
50. Factorial experiments and conceptions of the interaction of variables in experimental psychology
51. True and pre-experimental designs (according to Campbell) of research in experimental psychology
52. Types of variables and their confounds in a psychological experiment.
53. Types of representativeness; the connection between the notions of representativeness and validity.
54. Classifications of empirical methods in psychology
55. Analytical and graphical representation of the main result of action (MRA) of variables and their interactions
56. Variables and designs in correlational research.
57. Quantitative and qualitative data in the experiment and in other research methods.
58. Statistical decisions and formal planning of an experiment
59. The method of «double stimulation» in the transition to non-classical psychology
60. Special effects clarified by factorial designs. Factorial designs in the representation of non-experimental data.
61. The emergence and control of sequence effects
62. The problem of psychological explanation. The deductive-nomological model.
63. The multilevel experiment and cross-individual designs.
64. The longitudinal method in psychology; the cohort
65. The concept of method and technique in psychological research.
66. Functional control as a condition for planning and conducting a psychological experiment.
67. The relationship between the correlational and quasi-experimental approaches in psychology.
68. Qualitative research in psychology
69. Correlational data and structural modeling in experimental psychology
70. Population hypotheses and sample surveys
Psychological measurement, its types, the measurement scale for a psychological experiment
Experimental psychology and educational practice