Validity

a. Internal Validity

The most common definition of validity is best described by the question: Are we measuring what we think we are measuring?

There are two important forms of validity: internal and external validity.

Internal validity asks the question, "Is the connection between the independent variable and dependent variable clear and unambiguous to make a causal inference? (Leman, 1991). Internal validity is, therefore, concerned with correctly concluding that an independent variable is, in fact, responsible for variation in the dependent variable or "Did the treatment cause the effect?". If the experiment can clearly establish that the treatment cause an effect, then the experiment has internal validity. If something other than the treatment cause the effect, then the study doesn't have internal validity.

Setting up an experiment to obtain internal validity involves:

1. manipulating the independent variable - the definition of the independent variable needs to be clear; the manipulation must follow the logic of the experiment. Manipulation of an independent variable is the key part of an experiment. Since we are trying to establish a relationship between the independent and dependent variables (between two variables) it is important that the manipulation be done properly. To establish internal validity, one must treat experimental and control groups identically except for the manipulation of the independent variable. If the treatment is not identical, that is, the experimental and control groups are treat4ed differently on some dimension other than the independent variable, the experiment is internally invalid or confounded and does not provide a legitimate basis for drawing inferences.

Ex: Suppose a person wished to determine whether taking two aspirin on returning home from the office would relax him/her for the evening. To test this hypothesis, he/she might take aspirin on some evenings, but not on others, and assess the different effects of the treatment (aspirin) and control (no aspirins) situations. If this person washed the aspirin down with a glass of beer and the beverages were discontinued on control days along with the aspirin, then an important competing hypothesis for the conclusion that aspirin was a valuable relaxant would be that the beverage was responsible for the observed effect, thus invalidating the person's conclusion. This example, illustrates the basic problem of internal validity. Experiments are termed internally valid only when the obtained effect can be attributed confidently to the independent variable that was manipulated by the experimenter.

2. identifying physical factors as variables that co-vary - meaning that they act together. If other variables than the independent variable, such as, time of day, differences in instructions or any other identifiable variable is varying with the independent variable then a clear causal inference can't be drawn and the internal validity suffers.

3. identifying and eliminating extraneous variables - In any experiment there are always some variables, other than the independent variable, which could influence the observed effects. It is these potentially confounding variables that have to be dealt with or held constant and in order to deal with them most effectively, we must have some knowledge as to what they are. Some of these extraneous variables that needed to be controlled include physical variables such as sound, light, changes in the calibration of instruments, etc.

b. Internal Validity and Human Subjects

The use of human subjects is vastly different than a chemist using molecules. Measuring and manipulating human behavior, thought, feelings, is much more difficult than measuring and manipulation the behavior of molecules: human subjects may not passively accept manipulation; human subjects bring a wide variety of psychological reactions to a situation and, therefore, may well interpret the researcher's treatment differently than what he/she intended; people know that they are in a study and may act accordingly. For example, if they like the investigator may try to give him/her the results they think he/she wants. In general, it is far more difficult to establish internal validity for behavioral studies with humans than with molecules.

c. External Validity

External validity asks the questions: Can you generalize the results? As with internal validity, external validity can be checked by replication. That is, for internal validity, the replication needed to be checked with the same operations and the same variables; the check on external validity demands more. Since external validity asks about generalizations beyond the specific experiment, the replications should be with other settings, with other subject populations, and with other, but related variables.