An Unpublished Rebuttal to an Article That Enraged Me

In 2002, in the American Birding Association magazine Birding, I read an article on the increasing participation in birding written by two outdoor recreation researchers (at least one of whom is very prominent), a letter to the editor from a reader (who for the record is not just a totally random birder, but who appears to be the author of at least one book, a guide to birding in his home state) who objected to the researchers' claims, and a surprisingly disrespectful and shitty-toned response from the researchers. The exchange made me quite angry because the response to the reader was arrogant and dismissive to the point of reflecting poorly on the profession, in my opinion. I also thought that the article itself was shoddy, based on what I could glean from the methodology and results published online of the survey that underlies it.

I was sufficiently riled and offended that I wrote my own letter to the editor, but realized that I could not send it in. The world of outdoor recreation researchers is small, and though I am not a well-known person on the scene as was the author, a letter with my name and location would immediately be recognized by co-workers who read the publication and potentially remembered by others I met later on. I could not send in this letter as a "citizen" but only as a professional researcher associated with a particular organization and that didn't seem very smart for a lot of reasons.

When I first started this blog, I thought that when I was no longer working in the outdoor recreation field, it would perhaps be relatively safe to put up the letter here. I don't know, maybe it's still folly to do it (given the way things get googled), but I don't have to worry about being seen as representing any organization at this point and it's such an old issue, who really cares at this point. (Except perhaps the author I am criticizing.)

I was not able to find the letter to the editor and the researchers' response online, but the original article can be found here.

Here is my response, with some identifying information removed to minimize (I hope) the chance of this coming up on someone's web search:

Although we have heard already from the authors of the birding participation article [citation removed] again in the August issue, I believe this topic is important enough to the future of birding to merit further comments. I did not agree with all of FH’s observations, but I think some of his objections have been unfairly dismissed. Much of C’s rebuttal boiled down to an argument from authority, a rhetorical play that lacks logical force and does a disservice to those who are trying to understand the state of birding.

C & H seem to be making three main points in their article:
(1) Fact: There are 70.4 million birders in the United States.
(2) Fact: The number of birders is growing rapidly.
(3) Implication: Birders constitute a major political and economic force based on these high numbers.

Though I would not characterize these as completely untrue statements, they do not appear to have the evidence to back them up that the authors claim. I suggest that there are questions and potential problems with these claims that the authors do not address.

Measuring the number of birders in the US is a harder problem than C & H let on. I allow that a rather large percentage of respondents surveyed in the NSRE (extrapolating to 70.4 million people nationwide) said “Yes” to whether they have “during the last 12 months viewed, identified, or photographed birds.” The researchers conducting the survey are, as C strenuously reminds us, professionals who can be trusted to perform these calculations accurately. The authors freely admit that this is a loose definition, anticipating and to some degree negating FH’s objections. But despite the authors’ assurance that the survey only addresses “birding that takes place when the participant purposely goes outside or takes a trip away from home for birding and other recreational purposes” (seeing scarlet ibis at the zoo doesn’t count), I believe this definition is inadequate and allows the results of the survey to be misconstrued.

Looking at the question from the point of view of the average person, and not the outdoor recreation researcher, is instructive – technically, anyone who found herself “viewing” a bird even once in the last year would have to answer yes to this question. Not only intentional birding, be it backyard feeder-watching or visiting a NWR [national wildlife refuge], would count, but incidental and secondary birding as well. Does it make sense to call someone who went out into the yard to catch a breeze and saw a house sparrow a birder? How about someone who saw a heron fly by while she was fishing? Surely we would want to limit birding to include only that activity which occurs on an intentional trip outdoors (or viewing of backyard feeders) to view/identify/photograph birds as the primary activity.

Bird-viewing occurring as secondary to some other recreation activity would skew our understanding of birding. I suggest that this secondary birding is responsible, at least in part, for the authors’ finding that “birders” participate in more outdoor recreation activities than non-birders; people who walk outdoors, visit nature centers, picnic, etc., are more likely to engage in spurious bird-viewing than those who do not put themselves in birdy environments. The inclusion of this spurious bird-viewing activity artificially inflates the number of birders. Even those who do not find 70.4 million a prima facie implausibility have reason to question the figure based on the non-rigorous way the question was asked in the survey. (Note: the NSRE website listed a more satisfying Q139 – asking about birdwatching trips taken away from home – but does not list any module in which it was asked, nor do the authors report the results in their article. While this question is too limiting to stand alone, as it does not address backyard birding, it would present a valuable comparison.) Knowing the number of people who viewed birds in the last year because they wanted to, rather than who just came across birds as they went about their daily lives, is imperative to making any statements about the “popularity” of birding or how many people the activity of birding “attracted.”

As nearly everyone feels that birding is becoming a much more popular activity, C & H’s findings about increased participation in birding make intuitive sense. Examine their data a little more closely, however, and the findings are not quite as clear as they seem at first. The authors make much of the “dramatic growth” in participation between the 1983 NSRE and the 2001 NSRE without commenting at all on the changes in survey methodology that could undermine the trendability of the data. Survey research is very sensitive to changes in question text, methodology (including survey modality), and context, all three of which seem to have occurred in the NSRE since 1983 (based on the information I could find on the website). For instance, the 1983 NSRE was a face-to-face survey (not a telephone survey) that asked whether respondents did during the past 12 months “go birdwatching.” This wording includes an element of intent in the activity that merely “viewing birds” does not. We would expect some increase in self-reported “birding” in the latest NSRE as an artifact of this wording change alone. Although I do not think this accounts for all of the difference between the two surveys, the fact that C & H do not bother to cite any reason we should believe that the two surveys are measuring the same thing makes their findings somewhat dubious.

To some degree, all this confusion is understandable. Birding presents challenges to the researcher not posed by outdoor activities like hunting and fishing. For example, it’s hard to imagine anyone just happening to shoot a deer or catch a bass during their normal day or while taking a walk with the family in the park. Birding, by its nature, can be undertaken opportunistically, at a moment’s notice, and really, without even trying. I think this quality makes it all the more enjoyable to those of us who love it (who among us hasn’t broken the otherwise unbearable monotony of a long drive or long afternoon at work by watching whatever birds go by?) but does mean that extra vigilance is required when we attempt to quantify birdwatching behavior.

It is interesting to realize that another national survey, the US Fish and Wildlife Service’s 2001 National Survey of Hunting, Fishing, and Wildlife-Associated Recreation (HFWAR), found markedly different birding participation results from the NSRE. The HFWAR estimates the number of birders in the US to be 46 million (based on their preliminary report at [web address deleted]). Since the HFWAR is as statistically valid and has as long a legacy as the NSRE, the difference in findings underscores the importance of being careful about what we ask, how we ask it, and especially how we report it. 46 million birders, though still a good solid number, is not nearly as exciting as 70.4 million.

Certainly in the field of outdoor recreation participation, where funding for research is scarce, the NSRE is a valuable source of information. However, we can’t let our need for information and our desire to see our enthusiasm and hard work reflected in the lifestyle of the average American make us read more into the data than is truly there. Yes, we should applaud the very real gains in increasing interest in and access to birding, but it is easy to get a number like 70.4 million in our heads and become complacent or over-confident. Birders do not have the political clout or the economic impact that a number like that implies. I agree with FH that we do not do the birds, or ourselves, any favors by using inflated numbers.

Sincerely,

Sally [last name removed]
[location removed]
[email removed]