Making Sense of Research : What's Good, What's Not, and How to Tell the Difference

Foreword xi Henry M. Levin Preface xiii The Goals of This Book xiv Who This Book Is For xv What This Book Is Not xv Overview of the Contents xvi A Few Words of Explanation xvii Acknowledgments xviii About the Authors xxi Asking the Right Questions 1(17) What Is Our Approach to Making Sense of Research? 2(1) Five Questions About Research 3(9) The Causal Question: Does It Work? 5(3) The Process Question: How Does It Work? 8(1) The Cost Question: Is It Worthwhile? 9(1) The Usability Question: Will It Working for Me? 10(1) The Evaluation Question: Is It Working for Me? 11(1) Four Case Studies of Education Research 12(5) Class Size Reduction: Does Size Matter? 13(1) Phonics: Can It Teach Them All to Read? 14(1) Vouchers: Are Private Schools in the Public Good? 15(1) Whole-School Reform: Greater Than the Sum of Its Parts? 16(1) Further Reading 17(1) Behind the Scenes in the World of Education Research 18(14) The World of Education Research 18(4) The Worldviews of Researchers 19(1) Research Methodologies 20(1) The Purposes of Research 21(1) Who Does Education Research? 22(1) Who Pays for It? 23(1) Where Is It Published? 24(1) Can You Trust Education Research? 25(2) What Does the Future Hold for Education Research? 27(2) Does Education Research Really Matter? 29(1) Further Reading 30(2) The Causal Question: Does It Work? (Part I) 32(22) The Causality Conundrum 33(1) Simple (Minded) Methods of Establishing Causality 34(4) The Before-and-After Approach 35(1) The Comparison Group Approach 36(1) Internal Validity 37(1) The Elements of an Experiment 38(7) Why Random Assignment? 38(2) Units of Randomization 40(1) Defining the Treatment, Outcomes, and Sample 41(1) What Is the Treatment? 41(1) What Are the Outcomes? 42(1) What Is the Sample? 43(1) Analyzing Experimental Results 44(1) What Can Go Wrong With Experiments? 45(1) Why Are Experiments So Unpopular With Education Researchers? 46(6) Does Class Size Reduction Work? 48(1) Does Phonics Instruction Work? 49(1) Do Private-School Vouchers Work? 50(1) Does Whole-School Reform Work? 51(1) Further Reading 52(2) The Causal Question: Does It Work? (Part II) 54(21) Quasi-Experiments and Non-Experiments 55(2) What They Are Not 55(1) What They Are 56(1) The Researcher's Bag of Tricks 57(4) Design 57(2) Statistics 59(2) Quasi-Experimental Approaches 61(3) Matching 61(2) Interrupted Time Series 63(1) Bad Quasi-Experiments 63(1) Non-Experimental Approaches 64(9) Correlational Studies 64(2) ``Natural'' Experiments 66(1) Bad Non-Experiments 67(2) Does Class Size Reduction Work? 69(1) Does Phonics Instruction Work? 70(1) Do Private-School Vouchers Work? 71(1) Does Whole-School Reform Work? 72(1) Further Reading 73(2) The Process Question: How Does It Work? 75(16) What Is Qualitative Research? 76(1) The Qualities of Qualitative Research 77(2) Naturalistic 78(1) Descriptive 78(1) Focused on Meaning and Explanation 78(1) The Power of Qualitative Research 79(1) Qualitative Research Methods 80(3) Interviewing 80(1) Observing 81(1) Analyzing Documents 82(1) From Whence Comes the Quality in Qualitative Research? 83(7) How Does Class Size Reduction Work? 86(1) How Does Phonics Instruction Work? 87(1) How Do Private-School Vouchers Work? 88(1) How Does Whole-School Reform Work? 89(1) Further Reading 90(1) The Cost Question: Is It Worthwhile? 91(19) The Concept of Costs 92(2) The Ingredients Method of Cost Analysis 94(6) Identifying Ingredients 94(2) Valuing Ingredients 96(1) Analyzing the Distribution of Costs 97(1) Aren't Budgets Good Enough? 98(1) A Cost Analysis Caveat 99(1) Two Important Cost Questions 100(9) Is It Cost-Feasible? 100(1) Is It Cost-Effective? 101(2) Why Does This Matter? 103(2) Is Class Size Reduction Worthwhile? 105(1) Is Phonics Instruction Worthwhile? 106(1) Are Private-School Vouchers Worthwhile? 107(1) Is Whole-School Reform Worthwhile? 108(1) Further Reading 109(1) The Usability Question: Will It Work for Me? 110(19) Will it Work in My Setting? 111(4) Students 112(1) Settings 112(1) Treatments 113(1) Outcomes 114(1) Some Rules of Thumb for Making Generalizations 115(4) Surface Similarity 116(1) Ruling Out Irrelevancies 116(1) Making Discriminations 117(1) Interpolation and Extrapolation 117(1) Causal Explanation 118(1) Research Reviews and Meta-Analysis 119(1) Special Considerations for Qualitative and Cost Studies 120(2) Qualitative Studies 120(1) Cost Studies 121(1) Making Trade-Offs 122(6) Will Class Size Reduction Work for Me? 124(1) Will Phonics Instruction Work for Me? 125(1) Will Private-School Vouchers Work for Me? 126(1) Will Whole-School Reform Work for Me? 127(1) Further Reading 128(1) The Evaluation Question: Is It Working for Me? 129(22) What Is User-Driven Research? 130(4) The Causal Question: Is It Working for Me? 132(1) The Process Question: How Is It Working for Me? 132(1) The Cost Question: Is It Worthwhile for Me? 133(1) User-Driven Research at Lincoln Middle School 134(3) Middle School: Panacea or Pandora's Box? 134(1) Adopting the User-Driven Research Mind-Set 135(2) Asking the Right Questions 137(6) Question Number One: Is the Middle School Model Working for Us? 137(1) What Does the Research Say? 137(1) Analyzing Achievement Test Data 138(1) Question Number Two: How Is the Middle School Model Working (Or Not Working) for Us? 139(1) Surveying the Faculty 139(2) Interviewing the Faculty 141(1) Interviewing Students and Parents 142(1) Question Number Three: Is the Middle School Model Worthwhile for Us? 143(1) Counting the Costs 143(1) Cost Effective? 143(1) Refocusing on Outcomes 143(1) Defining the Problems 144(1) Exemplars of User-Driven Research 145(5) Inquiry for School Improvement: The Accelerated-School Model 145(2) Closing the Literacy Gap: The Reading Apprenticeship 147(1) Teachers as Staff Developers: East Side Community High School 148(1) Sustaining School Improvement: The Professional Learning Community 149(1) Further Reading 150(1) Resource: Bibliographies for Case Studies 151(6) Class Size Reduction 151(1) Phonics Instruction 152(2) Private-School Vouchers 154(1) Whole-School Reform 155(2) References 157(13) Index 170