Tuesday, August 14, 2007

Statstics and Spiritual Transformation

Statistics are, when all is said and done, nothing but statistics; numerical read-outs compiled from surveys which often employ arbitrary criteria. As such, statistics are not always the most reliable indicator of the overall health of the church. "American evangelicals," according to Notre Dame sociology professor Christian Smith, "are among the worst abusers of simple statistics," often exploiting data in the interest of "waking up" the church to a dire emergency which threatens its very existence. Smith offers a particularly egregious example.

In a recent issue of American evangelicalism's flagship magazine, I found a glossy, four-page, centerfold advertisement for a national leadership summit, announcing this:

Wake Up Call
"Christianity in America Won't Survive Another Decade."
Unless We Do Something Now


Anyone who gives this claim a moment's thought should realize immediately that it is preposterous. Unless we act now, U.S. Christianity will be dead in ten years? Please. I hope somebody will let God know. Still, the quotation suggests that some authority has determined this, so let us explore further. Inside the ad we are greeted by "A Call to Arms," and are informed that, "This generation of teens is the largest in history—and current trends show that only 4 percent will be evangelical believers by the time they become adults. Compare this with 34 percent of adults today who are evangelicals. We are on the verge of a catastrophe." This is really incredible. But still, specific statistics are staring up from the page: 4 percent compared to 34 percent. Plus, the ad shows a lineup of evangelical heavyweights endorsing this summit, including Chuck Colson, Ted Haggard (before his fall), Jerry Falwell, Pat Robertson, Jack Hayford, and many more. Not only was the president of the National Association of Evangelical(Haggard) backing this, but a recent official nae resolution on youth ministry is also reprinted in the ad, suggesting a general NAE endorsement of this program.

It turns out not to be possible from the ad to determine who is quoted saying that "Christianity in America Won't Survive," but the statistic that only 4 percent of today's teens will grow up to be evangelicals does have a reference. The fine print shows that it comes from a 1997 youth ministry book about "the bridger generation" written by a professor at a significant evangelical seminary. Having read nearly every book on youth ministry published in the last 15 years, I realize that I own the very book. I take it off my shelf and find the source of the 4 percent statistic. It turns out, as reported on pages 165-66, that this professor had done an "informal survey" of 211 young people in three states over seven months in the mid-1990s. No information about the methodology in sampling these youth is given—most likely it was a convenience sample. Based on this evidence, the author writes, "The vast majority of bridgers today have not accepted Christ. In a recent and informal survey of 211 bridgers, only 4 percent responded that they were born-again Christians who had trusted in Christ alone for salvation." Older Americans are then said, based on another similar "informal survey," to have accepted Christ at higher rates. The conclusion: "According to present trends, we are about to lose eternally the second largest generation in America's history."

Let us ponder what is wrong with this picture. There is nothing wrong per se with an "informal survey" of a convenience sample of a few hundred people—we can often learn a lot of value from such exercises. However, it is impossible to make accurate inferences about a huge generation of youth at a national level from only 211 of even the best of samples, much less from an "informal," non-probability, most likely convenience sample.

Furthermore, even if the numbers were somehow nationally representative, they still tell us absolutely nothing about "present trends." Trends have to do with changes observed over time. The numbers presented in the book have to do with differences in generations measured at one point in time. Without knowing whether the observed cross-sectional differences reflect a cohort effect (what the author assumes) or a life course effect (a more plausible alternative), we are no more able to project the claim that we are on the verge of losing a generation eternally than we can predict that all babies today are going to grow up to wear diapers as adults because we see that they are wearing diapers now.

And even if the differences represent a cohort (not life course) effect that will remain stable as this young generation ages, it still makes no sense to think that "we are about to lose" an entire generation. About to? Like, real soon? In fact, it would take many decades for the projection to play itself out, even if all of the assumptions behind it were correct, which they are not. The claim to imminent demise is silly on elementary logical grounds. Thus, the book's author is guilty of making some unwarranted inferences about national representation and future events based on limited data and faulty logic. This is not a crime. But it has problematic consequences.

Namely, based on such erroneous conclusions, a national movement is now being organized to re-educate 20,000 youth pastors in 44 cities around the nation. Tickets for the three-hour event are $39 each, $99 for a church's entire pastoral staff. Pastors who attend this "high level briefing" in which "top voices" will "present the hard facts," the ad states, will be doing their part to avert the catastrophe of dwindling church attendance, increase in peer pressure, a growth in porn and violence on TV, and the decay of "our Christian nation." By implication, those who don't attend are enabling the catastrophe. How do we know that this national catastrophe is imminent, that this "crisis" "demands our response," and that "this generation is depending on our attendance"? Again, because a seminary ministry professor conducted an "informal survey" nine years ago with 211 young people in three states, selected by methods about which we know nothing, asked questions we know almost nothing about, and then made the logical error of drawing an unwarranted inference from a small and non-representative sample about trends and future faith conditions of entire generations.

Contrary to popular belief, then, statistics do lie and when using them in the interest of advancing the truth, much caution needs to be taken. I was reminded of Smith's article today when I read the results of another recent survey published in USA Today showing young adults dropping out of church at an alarming rate. It is a survey based on a more reliable sample than the one cited by Smith, but it still elicited an unwarranted reaction from Princeton sociology professor Robert Wuthnow, who said, "Unless religious leaders take younger adults more seriously, the future of American religion is in doubt." We don't need a national survey to know that the church in America is not exactly experiencing a golden age, but such statements reflect an undue reliance on mere data divorced from the mystery of God, who moves according to his Spirit to rekindle the fires of spiritual fervor in times and seasons which cannot be predicted by statistics.

The church is better served by the counsel of Ed Stetzer, who suggests a change in the way the church does ministry, from spiritual entertainment to spiritual transformation. "People are looking for a faith that can change them and to be a part of changing the world," says Stetzer. That is substantive advice, not over the top rhetoric.

No comments:

Post a Comment