What Are Threats to Internal Validity? A BCBA Candidate’s Guide
Internal validity refers to the extent to which an experiment demonstrates a functional relation between the independent variable (IV) and the dependent variable (DV). When threats to internal validity are present, you cannot confidently conclude that changes in the DV were caused by the IV. For BCBA candidates, understanding these threats is essential for designing rigorous single-subject research and for answering applied questions on the exam.
Table of Contents
- What Are Threats to Internal Validity? A BCBA Candidate’s Guide
- The Most Tested Threats to Internal Validity on the BCBA Exam
- ABA Worked Examples: Identifying Threats in Scenario-Based Questions
- Exam Relevance: Common Traps and How to Avoid Them
- Quick Checklist for Evaluating Internal Validity in ABA Studies
- Summary: Strengthen Your BCBA Exam Prep on Internal Validity
- References
Internal Validity vs. External Validity in ABA
Internal validity asks: Did the intervention cause the behavior change? External validity asks: Will these results generalize to other people, settings, or behaviors? In applied behavior analysis (ABA), single-subject designs prioritize internal validity through repeated measurement and experimental control. You can learn more about single-subject experimental designs in our dedicated guide.
Why the BCBA Exam Emphasizes Internal Validity
The BACB’s 6th Edition Task List includes items like D-4 (design a single-subject experiment) and D-5 (identify and address threats to internal validity). You must be able to recognize these threats in vignettes and propose design adjustments to strengthen experimental control. Practicing with our BCBA practice exam materials can help you apply these concepts.
The Most Tested Threats to Internal Validity on the BCBA Exam
Each threat can undermine your ability to attribute behavior change to your intervention. Below are the most common ones, paired with ABA examples.
History: Real-World Events That Affect Data
A history threat occurs when an extraneous event outside the experiment—such as a change in medication, a family move, or a new teacher—affects the dependent variable. For example, during a functional analysis of self-injurious behavior, a parent reports that the child started a new sleep routine. The resulting decrease in behavior might be due to improved rest, not the contingency being tested.
Maturation: Time-Related Changes in Participants
Maturation threats involve biological or developmental changes that occur naturally over time. A skill acquisition program for a toddler may show gains in fine motor skills that are simply due to typical development, not the discrete-trial training. Always consider whether the change would have happened without intervention.
Instrumentation: When Measurement Tools Are Inconsistent
Instrumentation threats refer to changes in the measurement system itself. Examples include observer drift (i.e., data collectors gradually interpret behaviors differently), equipment malfunction (e.g., a timer fails), or changing from one data collector to another mid-study. During a DRO schedule, if a new therapist starts recording data, the artifact may appear as a sudden improvement or worsening of behavior.
Testing Effects: Impact of Repeated Measures
Testing effects (also called practice effects) occur when repeated exposure to an assessment changes performance. In a pre-post test design measuring social skills, a child may improve simply because they have practiced the test items, not because of the social skills group. In single-subject designs, repeated observation can sometimes lead to behavioral improvements unrelated to the IV.
ABA Worked Examples: Identifying Threats in Scenario-Based Questions
Applying your knowledge to realistic vignettes is essential for exam success. Here are three scenarios with analysis.
Example 1: History Threat in a Functional Analysis
Scenario: A functional analysis of aggression (escape condition) shows a sharp decrease in behavior during the second session. Upon further investigation, the analyst learns the child received a new sensory toy before the session that served as a competing reinforcer.
- Antecedent: Demands presented (escape condition)
- Behavior: Aggression
- Consequence: Toy available (extraneous variable)
Threat: History. The toy is an outside event that confounded the analysis. The hypothesized function of escape might appear false if the history threat is not controlled.
Example 2: Maturation Threat in a Skill Acquisition Program
Scenario: A BCBA implements a fine motor tracing program for a 4-year-old. Data show steady improvement over six weeks. However, the child also started attending a weekly occupational therapy session during that period.
- Antecedent: Worksheet presented
- Behavior: Tracing accuracy
- Consequence: Praise and token
Threat: Maturation (and possibly history). The natural motor development plus outside therapy could account for the gains, not the intervention alone. To rule out maturation, use a multiple baseline design across skills or participants.
Example 3: Instrumentation Threat in a DRO Schedule
Scenario: A data collector initially records 10 instances of screaming per session, but after a training workshop, the same collector records only 2 instances, even though no intervention changed. The therapist’s criteria for ‘scream’ shifted.
- Antecedent: Classroom setting
- Behavior: Screaming (definition drifted)
- Consequence: No immediate consequence; DRO delivery changed
Threat: Instrumentation. Observer drift changed the measurement. Treatment integrity was compromised. The remediation includes regular interobserver agreement (IOA) checks and retraining.
Exam Relevance: Common Traps and How to Avoid Them
Even well-prepared candidates fall into these traps. Recognize them early.
Confusing Threats with Confounding Variables
A confounding variable is any uncontrolled factor that varies systematically with the IV. A threat to internal validity is a specific type of confounding variable (e.g., history, maturation). On the exam, a question may ask: “What threat is most likely present?” and the scenario describes an event like a field trip (history). Don’t choose “confounding variable” as an answer unless it is the only option; look for the specific threat named.
Overgeneralizing Threats Across Designs
Some threats appear differently depending on the design. For example, history threats are more problematic in reversal designs because the time frames are longer. In a multiple baseline across behaviors, the staggered introduction helps rule out history. A maturation threat may be less of a concern in a very short study (e.g., 3 sessions) but very likely in a 6-month study.
Additional Exam Trap: Ignoring Baseline Trends
If the baseline shows a decreasing trend, then a subsequent intervention that continues the decrease may simply be a maturation or regression threat—not an effect of the IV. Always examine baseline patterns before concluding a functional relation.
Quick Checklist for Evaluating Internal Validity in ABA Studies
Use this checklist when designing a study or analyzing a research article for the exam.
- Identify potential confounds: List any events or changes outside the IV that could affect the DV.
- Check measurement consistency: Calculate IOA across all phases; retrain observers if IOA falls below 80%.
- Examine temporal patterns: Look for gradual trends in baseline that might indicate maturation.
- Evaluate design strength: Does the design (reversal, multiple baseline, alternating treatments) control for the likely threats?
- Rule out testing effects: If repeated assessments are used, consider whether practice effects are plausible.
- Document all extraneous variables: Maintain a log of environmental events that could serve as history threats.
- Use replication: Within-subject replication (e.g., returning to baseline, or across participants) strengthens internal validity.
Summary: Strengthen Your BCBA Exam Prep on Internal Validity
Mastering threats to internal validity is a cornerstone of experimental design in ABA. On the BCBA exam, you will be tested on your ability to identify these threats in vignettes and choose design strategies to mitigate them. Key takeaways include:
- Internal validity is about causal confidence—ensuring changes are due to the intervention.
- History, maturation, instrumentation, and testing are the most frequently tested threats.
- Use multiple baseline designs to control for history and maturation when reversal is not possible.
- Regular IOA checks and clear operational definitions prevent instrumentation threats.
- Practice with scenario questions to sharpen your identification skills.
For additional resources, explore our BCBA exam prep guide and check out the BACB’s official task list for more detailed competency requirements. Additionally, refer to peer-reviewed resources like the Journal of Applied Behavior Analysis for examples of how researchers address internal validity.
