Introduction to Validity in Behavior Analysis
When you design a behavior intervention plan or evaluate a research study, you need to be confident that your conclusions are sound. That’s where validity comes in. In applied behavior analysis, validity refers to the degree to which a measurement tool or experimental design measures what it claims to measure and supports the conclusions drawn. Understanding the different kinds of validity is essential for the BCBA exam because questions frequently ask you to identify threats, choose appropriate controls, or interpret study results. This guide covers internal, external, construct, and social validity—each with real ABA examples and exam tips.
Table of Contents
- Introduction to Validity in Behavior Analysis
- Internal Validity: Did the Intervention Cause the Change?
- External Validity: Can the Findings Be Generalized?
- Construct Validity: Are You Measuring What You Think You Are?
- Social Validity: Is the Intervention Acceptable and Important?
- Quick Checklist for the Exam
- Summary and Exam Tips
- References
Internal Validity: Did the Intervention Cause the Change?
Internal validity asks whether the independent variable (the intervention) was responsible for the observed change in the dependent variable (behavior), rather than some extraneous factor. For BCBA candidates, this is the most heavily tested validity type. You will see scenarios where an intervention appears to work, but a threat like history or maturation might be the real cause.
Common threats to internal validity include:
- History: Unplanned events (e.g., a new medication, a family crisis) occur between baseline and intervention that influence behavior.
- Maturation: Natural developmental changes (e.g., learning to walk, language development) produce behavior change, not the intervention.
- Instrumentation: The way behavior is measured changes (e.g., a new observer uses different criteria).
- Attrition: Participants drop out, skewing results.
- Testing effects: Repeated measurement itself alters behavior (e.g., practice effect).
Example: Internal Validity in a DRO Intervention: A BCBA implements a differential reinforcement of other behavior (DRO) procedure to reduce screaming in a 4-year-old child. Baseline shows 10 screaming episodes per hour. After the DRO, screaming drops to 2 per hour. However, during the same week, the child started a new speech therapy program that teaches calm requesting. The drop in screaming could be due to the speech therapy rather than the DRO. This is a history threat. To control for this, the BCBA could use a reversal design (ABAB) to see if screaming returns when DRO is withdrawn. Alternatively, a multiple baseline across settings would show that screaming only decreases when DRO is introduced in each setting.
External Validity: Can the Findings Be Generalized?
External validity concerns whether the results of a study or intervention can be generalized across different settings, people, behaviors, and times. On the exam, you will often see questions about generalization and maintenance. Even if an intervention works in a controlled clinic setting, it may fail in a natural environment like home or school due to differences in stimuli or contingencies.
Factors that influence external validity:
- Setting generality: Does the intervention work in other environments?
- Person generality: Does it work with different clients or staff?
- Behavior generality: Does the intervention affect other, untargeted behaviors?
- Time generality: Do effects persist over time (maintenance)?
Example: Generalizing from Clinic to Home: A token economy is highly effective in reducing disruptive behavior in a clinic classroom. The child earns tokens for compliance and exchanges them for preferred items. However, when the same system is tried at home, disruptive behavior remains high because parents inconsistently deliver tokens, the backup reinforcers are less preferred, and the home environment has more distractions. The intervention lacks setting generality. To improve external validity, the BCBA should program for common stimuli (e.g., use similar token boards), train parents thoroughly, and systematically introduce the token system at home with initial support.
Construct Validity: Are You Measuring What You Think You Are?
Construct validity evaluates whether the operational definitions and measurement procedures actually capture the theoretical construct they intend to measure. In ABA, constructs like “motivation,” “function,” or “social skills” must be clearly defined and measured validly. A common exam trap is confusing construct validity with internal validity. Construct validity focuses on the alignment between the construct and the measurement, not on cause-and-effect.
Key threats to construct validity include:
- Inadequate operational definition: If “on-task behavior” is defined too broadly, it may include irrelevant behaviors (e.g., looking at the teacher while daydreaming).
- Confounding constructs: The measure may reflect more than one construct (e.g., an IQ test may also measure test-taking anxiety).
- Expectancy effects: The experimenter or participant expectations influence outcomes.
Example: Construct Validity in a Functional Analysis: A functional analysis aims to identify the function of self-injurious behavior (SIB). The construct “function” is inferred from patterns of behavior across conditions (e.g., attention, escape, alone). If the attention condition shows high SIB, but the therapist delivers attention in a way that also includes brief escape from demands, the measure confounds attention and escape functions. Thus, the construct validity of the functional analysis is compromised. To improve it, ensure each condition isolates a single contingency.
Social Validity: Is the Intervention Acceptable and Important?
Social validity (Wolf, 1978) refers to the social significance of the goals, the social appropriateness of the procedures, and the social importance of the effects. The BCBA exam often includes questions about assessing social validity, especially in the context of ethical practice and stakeholder satisfaction. It is not enough for an intervention to be effective; it must also be acceptable to the client, family, and community.
Three dimensions of social validity:
- Goals: Are the target behaviors important to the client and society?
- Procedures: Are the interventions acceptable and ethical?
- Effects: Do the outcomes produce meaningful, satisfying changes?
Assessing Social Validity in Practice: A BCBA working with a family uses a parent satisfaction questionnaire after implementing a bedtime routine intervention. Parents rate the procedures as easy to follow, the goals as important (e.g., reducing night wakings), and the effects as positive (e.g., child falls asleep faster). High ratings indicate strong social validity. Conversely, if parents find the intervention too time-consuming or aversive, they may not implement it consistently, leading to poor outcomes even if the intervention is technically effective. Social validity data should be collected regularly and used to modify interventions.
Quick Checklist for the Exam
Use this checklist to test your understanding of each validity type:
- Internal validity: Can I identify the specific threat (e.g., history, maturation, instrumentation) from a scenario? Do I know which experimental design controls for it?
- External validity: Can I describe how to program for generality across settings, people, and behaviors? Do I recognize when a lack of generalization is due to poor external validity?
- Construct validity: Is the operational definition adequate? Could the measurement be capturing more than one construct?
- Social validity: Have I assessed goals, procedures, and effects from the stakeholder’s perspective? Are the interventions socially acceptable?
Summary and Exam Tips
On the BCBA exam, you will encounter scenarios that ask you to identify the validity type at risk or to suggest a remedy. A common trap is confusing internal and external validity. Remember: internal validity is about whether the intervention caused the change (causality), while external validity is about whether the findings apply to other situations (generalizability). Another trap is forgetting that social validity is a separate, critical dimension—even if a study has high internal and external validity, if the stakeholders reject the goals or procedures, the intervention is unlikely to be sustained.
For additional practice, try designing a simple study and listing possible threats to each validity type. Also, review free BCBA mock exam practice questions to test your skills. For a deeper dive into controlled experimental designs, see the description, prediction, and control goals of behavior analysis.
Finally, remember that validity is not an all-or-nothing property. Researchers and practitioners must consider multiple kinds of validity when evaluating evidence. By mastering these concepts, you will be well-prepared for exam questions and for real-world decision-making.
