Chapter 13: Introduction to Psychometrics
Why Psychometrics Matters for This Research
The common concern: “Can you trust data that comes from a survey?” Most people’s exposure to surveys is push polls, quick surveys, or surveys written by untrained researchers. This chapter explains why the research methodology is sound.
What is a Construct?
A construct is a latent variable that cannot be directly observed or measured — we measure it indirectly through observable indicators.
Examples:
- “Intelligence” — can’t directly measure it, measure through test scores
- “Organizational culture” — can’t directly see it, measure through survey responses about behaviors
Latent vs. Reflective Constructs
Reflective constructs: The construct causes the indicators. Changing the underlying thing changes all indicators in the same direction.
- Example: Changing someone’s “intelligence” would change all their test scores
This book uses reflective constructs for culture, job satisfaction, identity, etc.
Three Statistical Tests for Valid Constructs
Before any analysis between constructs, researchers must verify the measures themselves:
1. Discriminant Validity
Making sure items that are not supposed to be related are actually unrelated.
- E.g., items measuring “deployment frequency” should not be related to items measuring “organizational culture”
- Ensures constructs are measuring distinct things
2. Convergent Validity
Making sure items that are supposed to be related are actually related.
- E.g., all items measuring “organizational culture” should correlate with each other
- Confirms the items are all capturing the same underlying construct
3. Reliability (Internal Consistency)
Making sure items are read and interpreted similarly by those taking the survey.
- All people answering “organizational culture” questions should be understanding them similarly
- Usually measured with Cronbach’s alpha or composite reliability
These tests come before any analysis of relationships. Validity and reliability must be established first, then correlation/prediction analysis can begin.
The Software Delivery Performance Construct Issue
When building a construct for software delivery performance:
- Lead time, release frequency, and MTTR → pass all three statistical tests → form a valid, reliable construct
- Change fail rate → does NOT pass all tests as part of the construct
Solution: software delivery performance (as a construct) = lead time + release frequency + MTTR. Change fail rate is measured separately but is strongly correlated with the construct.
This is an important nuance: whenever the book says “software delivery performance predicts X,” it means the three-metric construct.
How to Use These Questions in Your Own Organization
Once a construct is validated, you can use the same questions in your own surveys. To calculate a score:
- Assign numerical values (1–7) to each Likert response
- Calculate the mean across all questions for each respondent
- Perform statistical analysis on the responses
The Westrum culture construct, the identity construct, and others in this book are validated — you can use them directly.
Psychometric Validation for Delivery Metrics
Unlike the other constructs (culture, satisfaction), delivery metrics (lead time, deploy frequency, MTTR, change fail rate) are not typical Likert constructs — they’re actual performance measures. But they’re asked as survey questions because:
- Getting exact data from every organization’s systems is impractical
- Self-reported data is validated against known high/low performers
- Cluster analysis results are consistent year over year
Key Takeaway for Practitioners
The research methodology here is rigorous because:
- Hypotheses are formed before data analysis (inferential predictive, not exploratory fishing)
- Constructs are validated before relationships are tested
- Same measures replicated across four years → consistent findings increase confidence
- Sample size (23,000+ responses) provides statistical power
- Published in peer-reviewed academic journals (not just a report)
This is why the findings are not just anecdote — they represent statistically meaningful patterns across thousands of organizations.