to more fully explain the story behind the numbers.
The book concludes with a clarion call for giant leaps, not baby steps. This is not the time for the meek and tentative.
In the pages that follow, you will find all the tools you need to be bold and to forge ahead with confidence because the students you serve deserve nothing less than your courage, resilience, and perseverance.
A Word About Sources
The extensive reference section acknowledges the work of many scholars in this field, and I hope that I have done them justice. I have found a great deal of commonality among their writings and also a strong sense of consistency with my own research. I have also included observations from the field and, where possible, named the people, schools, and districts involved. In other cases, I have used a synthesis of observations and my conclusions from my work in fifty states and more than thirty countries. These observations and conclusions represent my best thinking on the matter in late 2019, but I acknowledge that much research remains to be done. Where there is a relevant citation of the work of others, I have included it to the best of my ability. The conclusions without citations represent my observations from extensive research and field study over more than twenty-five years.
| PART I |
Discovering When to Trust Educational Research
Our journey begins as every educational decision must—with evidence. Since publication of the original research on equity and excellence schools (Reeves, 2004), there have been two competing narratives about the influence of poverty on student achievement. The first and dominant narrative is that demography is destiny. This is an echo of American sociologist, theorist, and empirical researcher James S. Coleman’s (Coleman et al., 1966) report, Equality of Educational Opportunity, which contends that the strongest predictor of student academic success is the mother’s level of education. From the 1960s through the early years of the 21st century, the assertion that schools with high percentages of low-income students who are members of ethnic minorities and do not speak English at home would inevitably have low achievement seemed incontrovertible. Whenever there were outliers—that is, schools with students from low-income families who performed well academically—these schools stood out because they were so unusual. This narrative remains dominant in North American education. When I present data to the contrary—that not only individual schools but also entire districts are defying the odds—some educators and administrators may challenge the accuracy and credibility of the data. Even when I present data from their own districts and schools, there remains hard-core disbelief in the notion that poor children and children of color can succeed in the American educational system.
In my keynotes, seminars, and personal conversations across the United States and around the world, I often ask, “Why do teachers and administrators distrust educational research?” The answers include the following.
• The research doesn’t apply to us. We are different because we are urban (or rural or suburban), our union contract is different, our budget is different, and our parents are different.
• The research has a tiny sample size that cannot be generalized to the broader student population.
• The research is from schools with heroic teachers and administrators, but they will burn out because their efforts are unsustainable.
• The research is commercially tainted because publishers conduct it while attempting to sell their textbooks, online systems, or instructional programs.
• The criteria for success are too low, so what the research calls meeting standards is not equivalent to what our district regards as a successful outcome.
• The per-pupil funding in the research is higher than in our schools.
• The successful schools cherry-pick students.
The objections are consistent and pervasive, so many school leaders and teachers subsequently avoid implementing the positive practices of these exemplary schools. Therefore, part I makes the important case for when readers should trust the research. Chapter 1 presents a typology of evidence, proceeding from level 1 research (personal beliefs) through level 5 research (preponderance of the evidence—the gold standard of educational research to which these chapters aspire). Chapter 2 then considers seven common challenges to educational research and offers a respectful reply to each challenge.
CHAPTER 1
Understand the Five Levels of Educational Research
Educators and leaders are weary of the vague claims that “research shows …” In conversations around the globe, they tell me that ambiguous claims are not enough to change their practices. They want to see credible evidence of the impact of improved leadership and teaching practices, and will not settle for claims without credible evidence. This chapter will help you to be a more critical consumer of educational research. Just as we ask students to evaluate claims based on the evidence, we must model that same level of critical thinking every time we listen to a presentation or read an article or book. As this chapter argues, there is an enormous difference between the credibility of a “journey story” about one person’s experience—a sample size of one—and the highest level of research, the preponderance of evidence.
In this chapter, we explore five levels of educational research. These include (1) personal beliefs, (2) personal experiences, (3) collective experience, (4) systematic comparisons, and (5) preponderance of the evidence. When legislators and other policymakers, leaders, or teachers state, “Research shows that …,” they might be referring to any one of these types of research. However, as citizens hoping to implement effective change in our schools, we can only truly rely on the higher levels of research—systematic comparison and preponderance of the evidence—to produce results that could feasibly be applicable and transferrable to our own schools. It is, therefore, essential that when we hear claims about research, we identify which type of research the people making the claims are using as the basis for their conclusions.
Level 1: Personal Beliefs
People are entitled to their own beliefs. That’s a guarantee in the United States (thanks to the First Amendment to the Constitution), as well as in many other nations around the world. The United States is hardly unique in protecting freedom of beliefs. Indeed, the founding documents of the United States drew heavily from thought leaders in England, France, and many other countries (Lepore, 2018) where laws prohibit governments from interfering with the free exercise of anyone’s religious beliefs, right to free speech, and freedom of assembly. As long as our beliefs do not interfere with the rights of others, we are free to believe whatever we wish.
Hand-in-hand with this freedom of personal beliefs, however, is the ability to believe something that is categorically wrong—and to persist in that belief due to personal preference, even in the face of evidence to the contrary. When presented with scientific evidence that defies personal beliefs, it is not unusual for someone to respond, proudly, “Your research may say that out-of-school suspension, punishment for missing homework, corporal punishment for behavioral infractions, and forcing students to stand in the corner with a dunce cap doesn’t work, but it sure worked for me!” These claims are contrary to research and common sense (Reeves, 2011a), but in a free society, we tolerate them—at least as long as the actions associated with these claims do not harm others.
Whether the forum is a meeting of teachers, a public comment portion of a school board meeting, a legislative hearing, or a speech at a political rally, listeners must ask, “What type of research is this?” When the answer is a personal belief, unburdened by factual evidence, then we can accept it as a by-product of the constitutional guarantee of free speech and exclude it from the realm of evidence-based claims, no matter how sincerely held the belief may be. We cannot debate beliefs any more than we can debate sincerely held religious views. Rather, when encountering strongly held personal