Evaluation Questions & Philosophy of Science

POSC 315 — Lecture 13.2

Foundational Evaluation Questions

1. Need: Does a social gap justify intervention?
2. Evaluability: Are goals clear and data collectible?
3. Process: How is the program actually operating?
4. Impact: What difference does it make?
5. Efficiency & Equity: At what cost and for whom?

Going Deeper

1. Was this really the problem?
2. Design–problem alignment?
3. Fidelity: Was implementation true to design?
4. Missing data or blind spots?
5. Alternative models worth testing?
6. Concrete next steps for improvement?

Enter Science: How Do We Know?

Policy analysis borrows its rigor from the broader scientific enterprise.

Science → systematic, evidence‑seeking process aimed at explanation and prediction.

The Scientific Method (In a Policy Key)

1. Observe a social condition (e.g., opioid overdoses rising).
2. Theory: Hypothesize mechanisms (pill mills → addiction).
3. Deduce testable propositions (closing mills ↓ overdoses).
4. Collect data (county‑level death certificates).
5. Analyze (difference‑in‑diff).
6. Conclude & Revise.

Philosophy of Science — Why Bother?

Underpins the credibility of our evidence claims.

• What counts as reality? (Metaphysics)
• How can we know it? (Epistemology)
• Why should we value certain outcomes? (Axiology)

Metaphysics (Reality Check)

• Is poverty a structural inevitability or policy choice?
• Ontological stance shapes causal stories.

Epistemology (Knowledge Claims)

• Rationalism: Reason can unveil policy truths (e.g., welfare calculus).
• Empiricism: Evidence rules (randomized evaluation of housing vouchers).
• Skepticism: Stay humble — data are theory‑laden.

Axiology (Values in the Room)

• Efficiency vs Equity trade‑offs.
• Transparency, accountability, social justice.
• Your implicit values steer research questions.

Bridge to Next Deck

Methodological worldviews (positivism, constructivism, etc.) and why they matter for crafting policy evidence.