On Doing Science

Last updated: 14 August 2017

Science is the search for truth regardless of how good the story is, whereas ‘marketing or advertising’ is the search for a good story regardless of the truth.”  –Donald Kroodsma to Jeffrey Podos, UMass Biology, 4 October 2004

Birdsong Performance Studies: A Contrary View

Here is the Backstory

When Jeff Podos used the University Police to threaten me with criminal harassment charges for asking questions about his research, I realized that I had better document all that transpired. Here is the website that has grown since that December 2014 threat.

MARCH 2017–Finally, after nearly three years, my Forum article is being published in Animal Behaviour, together with three replies. Next will come my (submitted, and revised) response to the replies. Here is where you can read what will go public:

Kroodsma, D., Birdsong performance studies: a contrary view, Animal Behaviour (2016), (available here)

Podos, J., Birdsong performance studies: reports of their death have been greatly exaggerated, Animal Behaviour (2016),  (available here)

On Andrew Gelman’s blog, Podos has expressed considerable displeasure at my lack of response to his counter-replies. So here is the detailed response that he has asked for but that never could have been published, for a variety of reasons, but not because of the substance of my remarks.

Vehrencamp, S. L., de Kort, S. R., and Illes, A. E. Response to Kroodsma’s critique of banded wren song performance research, Animal Behaviour (2016),  (available here)

A somewhat detailed, unpublished response to Vehrencamp et al. can be found here.

Cardoso, G. C., Advancing the inference of performance in birdsong, Animal Behaviour (2016),  (available here)

Kroodsma, D. E. (submitted, May 2017). Birdsong ‘Performance” Studies: A Sad Commentary

Above is my attempt at a response to Podos, Vehrencamp et al., and Cardoso. For those interested in how it was received by the editor and a referee, and for my responses to them, you can read all of that here.

Kroodsma, D. E. (revised and accepted for publication, July 2017). Birdsong ‘Performance’ Studies: A Sad Commentary (next step I believe is to give Podos, Vehrencamp et al., and Cardoso a chance to reply, which might take some time)

Here is a “web extra” that shows how wide filter bandwidths in software programs introduce significant errors in measurements of frequency bandwidth.

Here is the more detailed response directed to Podos, as referenced above.

Here is the more detailed response to Vehrencamp et al., as referenced above.

 

INTRODUCTION

During May 2014, Goodwin and Podos (2014) presented a paper at the ornithological meetings in Rhode Island that I immediately recognized as impossible to be true, i.e., false. Here is the correspondence for my attempts to address the falsehoods in that paper and others by Podos and his students (and other collaborators). I have grouped the correspondence into five sections, for which I provide a brief introduction here. (Necessary 6th section added July 2017.)

Goodwin, S. E., and J. Podos. 2014. Team of rivals: alliance formation in territorial songbirds is predicted by vocal signal structure. Biology Letters. 10:20131083.

Section 1) ATTEMPTS TO COMMUNICATE DIRECTLY WITH JEFF PODOS AND HIS STUDENTS
From July through December 2014, Podos (and students) refuse to respond to inquiries about Goodwin and Podos (2014), an NSF-sponsored research publication, contrary to all NSF guidelines about openness in use of tax-payer money. Podos silences me by using the UMass Amherst police to threaten me with criminal harassment charges if I attempt to communicate one more time with “UMass Biology” (in which I am emeritus). The Police further dictate that I am to tell ~50 international correspondents on this topic that none of them are allowed to contact Podos either.

Useful information regarding scientific research funded by the National Science Foundation is here: http://www.nsf.gov/bfa/dias/policy/dmp.jsp. In part, it states the following:

Dissemination and Sharing of Research Results . . . Investigators are expected to share with other researchers, at no more than incremental cost and within a reasonable time, the primary data, samples, physical collections and other supporting materials created or gathered in the course of work under NSF grants. Grantees are expected to encourage and facilitate such sharing.

Here is the “Ethics in Publishing” statement from the Animal Behavior Society, where Podos is the President of the Society (July 2017): https://www.elsevier.com/journals/animal-behaviour/0003-3472/guide-for-authors#5001 It states, in part, the following:

Professional integrity in the conduct and reporting of research is an absolute requirement of publication . . . , as is a willingness to share information with other members of the scientific community.

All correspondence for Section 1 can be found HERE.

Section 2) BIOLOGY LETTERS—attempts to address the issues publicly
From February to October 2015, I have a running dialogue with Biology Letters about publishing my objections to Goodwin and Podos (2014). Initially I am promised a public hearing on these issues, but all communication ceases after Podos submits to Biology Letters a confidential letter that has purportedly been written by a secret review panel at UMass. “Per University rules,” Biology letters cannot disclose to me the contents of this letter. The only name Biology Letters will give me is Dean John McCarthy (Vice Provost for Graduate Education and Dean of Graduate School; Distinguished Professor of Linguistics), whose name was apparently on the document.

But Dean McCarthy claims to know nothing about this letter. Dean McCarthy has “no idea” who wrote the document, has “no idea” who submitted the document, and had no role (“none”) in preparing or writing the document, nor is he interested in knowing who submitted this secret document in his name. Nevertheless, he does know that this document, the contents of which he apparently knows nothing about, is protected as a “student educational record” under the “Family Educational Rights and Privacy Act (20 U.S.C § 1232g) (b) (1),” which of course has nothing to do with protecting published authors from answering questions about their research.

So . . . Who wrote the secret communication to Biology Letters? And why must it be secret? And why must such great measures be taken to avoid a public exchange about published research? And where do university administrators receive their advanced training in stonewalling and obfuscation?

All correspondence for Section 2 can be found HERE.

Section 3) ANIMAL BEHAVIOR, the Journal; attempted submission of Forum Article
From September 2014 to January 2015, after seeking feedback from those whom I critique in my review article (as required by the journal), my initial attempt to submit a Forum Article is thwarted by an editor who is angry that I sought that very feedback as required by the journal. I am accused of many misdeeds. My proposed article is publicly rejected before it is even submitted, announced in emails sent to target authors Podos, Nowicki, Searcy, Vehrencamp, Goodwin, and Moseley (all of whom apparently wrote the editor and lobbied against the article).

During November 2015, with new editor Susan Foster, I resumed attempts to publish my review article, entitled “Birdsong Performance Studies: A Contrary View” (submitted manuscript available HERE).

That manuscript is my wide-ranging critique of the birdsong performance studies championed by Podos and colleagues, to be revised as a proposed Forum article at Animal Behavior. The first nine pages are devoted to refuting Goodwin and Podos (2014) by exposing its four fatal flaws. The first 21 pages are devoted to flaws in the studies of chipping sparrows and swamp sparrows by Podos and associates. In some candor, I show how pseudoscientific methods have been used to confirm Podos’ hypothesis involving vocal deviation as a measure of birdsong performance.

Referee comments on above submission, and Kroodsma response

                Uncorrected page proofs!

Self-explanatory. The Forum article was accepted mid-2016 and will be published early 2017.

All correspondence for Section 3 can be found HERE.

Section 4) ASSOCIATION OF FIELD ORNITHOLOGISTS (AFO)—I suggest that a best student paper should be retracted
Goodwin received a best-student paper award for her paper at the 2014 ornithological meetings in Rhode Island. Given the mistruths permeating the paper, I suggested to the AFO that it would be in their best interest to retract this award. It would send a strong message to all graduate students and their advisers that science matters. A second Podos student received the other best student paper award at the meetings, leading to a boast of a “clean sweep” on Podos’ website. After a long-time colleague of Podos (office mates in graduate school) was consulted, my request for review was dismissed.

All correspondence for Section 4 can be found HERE.

Section 5) THE UNIVERSITY OF MASSACHUSETTS AMHERST—attempts to address scientific and ethical conduct with Deans, Vice Chancellors, Provost, Departmental Chairs, etc.
From January 2015 to September 2016, with vigorous hand-waving and tenacious institutional loyalty, but never once directly and independently addressing with me the scientific or ethical issues involved, my allegations of misconduct are dismissed by Departmental Chairs (Karlstrom, Connor), College of Natural Sciences Dean and Associate Dean (Goodwin, Powers), Vice Chancellor for Research and Engagement (Malone), Vice-Provost for Graduate Education and Dean of the Graduate School (McCarthy), and Provost and Sr. Vice Chancellor for Academic Affairs (Newman).

All correspondence for Section 5 can be found HERE.

Section 6) My correspondence with officers of the Animal Behavior Society, where Jeff Podos is now President. During July 2017, I needed to add another section to this saga. In protest of the “leadership” of the ABS, and focusing on ethical issues, I resigned as a Fellow of the society (resignation letter here). A few days later, the secretary of the ABS kindly informed me that I had been scrubbed from the Fellows roster (read here). Period.

In a follow-up to the Officers of the Animal Behavior Society (here), I suggest that maybe they should think twice about what kind of “scientific society” they want. Given the scientific and ethical issues involved, quietly accepting the leadership of the ABS as is no doubt sends the wrong message to anyone who cherishes science as a means to learning truths about the natural world. No young scientist entering the field of Animal Behavior should learn that this is how research is done and how one gets ahead.

 

 

TIMELINE OF IMPORTANT EVENTS

DATE
EVENT
2004: Jan 1
Kroodsma leaves academics, to “get busy living”
2004 to present
“performance literature” flourishes
2004: Oct 8
Kroodsma to Podos, in requested in-house review: “In my view, science is the search for truth regardless of how good the story is, whereas ‘marketing or advertising’ is the search for a good story regardless of the truth, or regardless of how good the data are.” (Little did I know where this was heading over the following decade.) Podos would never communicate with me again, to this day, 13 years later, even though I am emeritus in his very department. (Entire 2004 review here.)
2014: May
Ten years later, at my first professional conference since retiring, I hear Goodwin and Podos (2014) presented orally. I recognize immediately it as entirely false; good marketing wins it a best student paper award at the conference (a second Podos student takes the other best student paper award).
2014: July 9 to Dec 1
Eight attempts to communicate directly with Podos and his students about their research. I receive no response, in violation of both the “Ethics in Publishing” demands of the Animal Behavior Society (of which Podos is president-elect) and the “NSF Data Sharing Policy” of the National Science Foundation (which funded the paper in question; Details in Section 1 correspondence.)
2014: Dec 1
As required by the journal Animal Behavior, I submit my proposed Forum article to all of the authors whom I critique (or applaud), seeking feedback before a formal submission to the journal.
2014: Dec 8
Animal Behavior Editor Michelle Scott angrily rejects my proposed Forum article before it is even submitted, because of my “e-mail distribution to so many colleagues” (though I was required to do so by the journal); this angry rejection letter is sent directly to Podos, Goodwin, Moseley, Searcy, Nowicki, and Vehrencamp, some of the primary targets of my Forum article, no doubt after hearing from each of them. (Details in Section 3 correspondence.)
2014: Dec 18
The University of Massachusetts Police call me, threatening me with criminal harassment charges if I try to contact Podos or his students about their research. (Details in Section 1 correspondence.)
2015: Jan 1
University Police inform me that I must tell all ~50 correspondents, worldwide, that none of them are allowed to contact Podos or his students either. (Details in Section 1 correspondence.)
2015: Aug 15
With an advance copy of my Forum article in hand, Podos et al. submit a new manuscript, “discovering” the same key idea that I had made in my Forum article (“performance” is a characteristic of song types, not males), with no mention of my Forum article. Also, instead of excoriating Cardoso et al., as Podos did in 2012, Cardoso et al. are now repeatedly credited with the key idea, as I had pointed out in my Forum article.
2015: Feb to Oct
Correspondence with the journal Biology Letters to have Goodwin and Podos (2014) retracted is terminated abruptly by the journal after Podos submits a University of Massachusetts panel review to the journal; “per University rules,” the review is secret, given “the Dean’s reasons for not sending on this information” to me. Trouble is, the Dean mentioned is Dean McCarthy of the UMass Graduate School, and he declares “I did not send anything to any journals.” He refuses to clarify this situation after further inquiry. The authors of this secret communication to Biology Letters, and its contents, remain unknown. (Details in Section 2 correspondence.)
2015: Jan to Oct
Correspondence with the Association of Field Ornithologists, suggesting that they might consider retracting a best-student paper award to Goodwin and Podos, terminated after hearing from graduate school office mate of Podos (Bernie Lohr). (Details in Section 4 correspondence.)
2015, Jan to 2016, Sept
Attempts to have University of Massachusetts oversight personnel address matters of scientific and ethical misconduct fail, as there is nothing out of the ordinary in all these matters as seen by Biology Department chairs (Connor, Karlstrom), Deans (Powers, Goodwin), Vice Chancellor for Research and Engagement (Malone), Vice Provost for Academic Affairs (Dumont), and Provost (Newman). As Provost Newman declares, “From the University’s perspective, this matter is closed.” (Details in Section 5 correspondence.)
2016: Feb 5
My Forum article is resubmitted to Animal Behavior, with new editor Susan Foster. (Extensive referee comments here.)
2016: June 30
Forum article accepted. (Uncorrected page proofs here.)
2016: Sept 10
Responses by Target Authors expected sometime in the future.
2017: January 15
Corrected page proofs of my Forum and three replies are available online, at Animal Behaviour’s home page
 2017: 13 August
I’m tiring of all this. As I am writing my final thoughts to the officers of the Animal Behavior Society, I receive an email from Andrew Gelman, suggesting I look at his blog post for the morning, entitled “Bird Fight.”
Go ahead. Take a look: http://andrewgelman.com/2017/08/13/bird-fight/. Here’s your chance to weigh in and offer an opinion about whether I’m an old crank who can’t accept new innovative methods or whether I’m blowing the whistle on shoddy research.
What’s next?
I would love to extract myself from all of this nonsense. I’m going to do my best to head out to pasture now. After all, I did retire from academia 13 1/2 years ago, to “get busy living.” I shouldn’t have to continue these fights about what constitutes science. The essence of it is all so embarrassingly elementary.
I refer readers to the following post by Andrew Gelman, What has happened down here is the winds have changed. You can read that version and try to find for yourself all of the parallels with my efforts, or you can read this Word version in which I have hi-lited all of the phrases and sections that describe the problems with the birdsong performance literature and its authors.
Some of my favorite Gelman quotes are isolated here, including the following:
We have now reached the “emperor has no clothes” phase
the research incumbency rule . . .  once an article is published in some approved venue, it should be taken as truth.
find-statistical-significance-any-way-you-can-and-declare-victory paradigm
For . . . [researchers] . . . to step back and think that maybe almost everything they’ve been doing for years is all a mistake . . . that’s a big jump to take. Indeed, they’ll probably never take it. All the incentives fall in the other direction.
 To put it another way, [Podos and his] . . . friends and students followed a certain path which has given them fame, fortune, and acclaim. Question the path, and you question the legitimacy of all that came from it.
We learn from our mistakes, but only if we recognize that they are mistakes. Debugging is a collaborative process. If you approve some code and I find a bug in it, I’m not an adversary, I’m a collaborator.

SUPPLEMENTARY MATERIAL

Goodwin & Podos_2014_Biology Letters
It was hearing this oral paper delivered at ornithological meetings in Rhode Island during 2014, and my subsequent reading of this paper in Biology Letters, that was for me the “last straw,” the culmination of so many papers with a veneer of science but no substance in this sexual selection literature.

Akcay & Beecher 2015 Critique of Goodwin & Podos 2014
After the June 2014 ornithological meetings, I communicated with Beecher at the University of Washington regarding the problems with Goodwin and Podos (2014). He and his student submitted a critique to Biology Letters; Akcay and Beecher deal with secondary problems and fail to address the really serious issues, but their critique eventually preempted mine at Biology Letters.

Goodwin & Podos_2015–strong defense of 2014
Goodwin and Podos mount a strong defense of their 2014 article, and appear on the surface to deal effectively with the minor wrist-slapping of Akcay and Beecher, as if these were just minor and insignificant differences of opinion on methodologies and interpretation.

Podos et al. (2012)
Here is a public prescription on how to do good science. Authors who had come very close to dismantling Podos’ performance hypothesis are lectured on all manner of topics: 1) proper measurements and methodology, 2) interpretation of data, 3) validity of results, 4) experimental rigor, 5) alternative explanations and hypotheses for data, 6) the ability to reject hypotheses, 7) appropriate use of skepticism, 8) problems in published papers that “undermine the validity of the results reported and the conclusions reached,” and 9) “basic principles” of science. Ironically, Podos and his coauthors are concerned, more broadly, with 10) how papers failing on these measures will “have a profound adverse effect on the way the research field is viewed by the rest of the scientific community.”

Podos et al. (2016)

Here is the paper submitted and published after the advance copy of my Forum article was sent to Podos and several of his coauthors. One of the main points in Podos et al. (2016) is that “performance” is a characteristic of song types, not males; this is an entirely new admission by Podos, and matches my lengthy deliberations and conclusions in my Forum article. Another major difference from all previous publications by Podos, polar opposite to that by Zollinger, Podos, et al. (2012), again matching my Forum article, is that Cardoso et al. should be given major credit for pointing out this idea for songbirds.

Two of the main points from my  Forum, which would not appear until the following year (2017), have now been appropriated by Podos et al., who would later, in Podos (2017),  declare that my points are “. . . not surprising . . . Indeed, similar results have been reported elsewhere (. . . Podos et al., 2016).”