“Science is the search for truth regardless of how good the story is,
whereas ‘marketing or advertising’ is the search for a good story regardless of the truth.”
–Donald Kroodsma to Jeffrey Podos, UMass Biology, 4 October 2004
Summary: On Scientific Integrity.
Here is the one issue that summarizes everything that follows: The literature that I critique in these performance studies thrives on an absence of “Scientific Integrity,” as defined by Richard Feynman:
But there is one feature I notice that is generally missing in Cargo Cult Science. That is the idea that we all hope you have learned in studying science in school—we never explicitly say what this is, but just hope that you catch on by all the examples of scientific investigation. It is interesting, therefore, to bring it out now and speak of it explicitly. It’s a kind of scientific integrity, a principle of scientific thought that corresponds to a kind of utter honesty—a kind of leaning over backwards. For example, if you’re doing an experiment, you should report everything that you think might make it invalid—not only what you think is right about it: other causes that could possibly explain your results; and things you thought of that you’ve eliminated by some other experiment, and how they worked—to make sure the other fellow can tell they have been eliminated. (emphases mine; excerpted from http://calteches.library.caltech.edu/51/2/CargoCult.htm)
The one common thread in all of these performance studies is a “leaning forward,” presenting only information that is consistent with the chosen performance message and concealing information that is contrary. As a result, research results are distorted, and the literature becomes a quagmire of good stories with no basis in reality.
Blatant examples, spanning three academic generations, are Podos, Peters, and Nowicki (2004) and Goodwin and Podos (2014). And in spite of all the contrary evidence I present, it continues, as in Liu, Soha, and Nowicki (2018).
There’s one more message that permeates this entire discourse: Almost no one cares (or dares to care).
Here is the Complete Story
When Jeff Podos used the University Police to threaten me with criminal harassment charges for asking questions about his research (yes, you read that correctly; listen (here) to the recorded conversation or read the transcript in section 1 below), I realized that I had better document all that transpired. Here is the website that has grown since that December 2014 threat.
DECEMBER 2017–Finally, mid-2017, after three years, my Forum article (1, below) was published in Animal Behaviour, together with three replies (2, 4, 6) by the “target authors.” My final published response (7) then appeared later in the year (more detailed unpublished responses are 3 and 5). Here are the documents:
1. Kroodsma, D., Birdsong performance studies: a contrary view, (available here)
2. Podos, J., Birdsong performance studies: reports of their death have been greatly exaggerated (available here)
3. On Andrew Gelman’s blog, Podos has expressed his displeasure at my lack of detailed response to his counter-replies. So, at his request, here is the detailed response (entitled “BULLSHIT is SCIENTIFIC FRAUD”) that he has asked for.
4. Vehrencamp, S. L., de Kort, S. R., and Illes, A. E. Response to Kroodsma’s critique of banded wren song performance research (available here)
5. A brief response to Vehrencamp et al. can be found here.
6. Cardoso, G. C., Advancing the inference of performance in birdsong (available here)
7. Kroodsma, D. E. Birdsong ‘Performance” Studies: A Sad Commentary
Above is the link to my brief published response to Podos, Vehrencamp et al., and Cardoso. For those interested in how it was received by the editor and a referee, and for my responses to them, you can read all of that here.
Here is a “web extra” that shows how wide filter bandwidths in software programs introduce significant errors in measurements of frequency bandwidth.
(For more detailed responses to Podos and to Vehrencamp et al., see items 3 and 5 above.)
INTRODUCTION
During May 2014, Goodwin and Podos (2014) presented a paper at the ornithological meetings in Rhode Island that I immediately recognized as impossible to be true, i.e., false. Here is the correspondence for my attempts to address the falsehoods in that paper and others by Podos and his students (and other collaborators). I have grouped the correspondence into six sections, for which I provide a brief introduction here. During January 2018 I add a seventh section, on how this critique of the performance literature plays out in the literature.
Goodwin, S. E., and J. Podos. 2014. Team of rivals: alliance formation in territorial songbirds is predicted by vocal signal structure. Biology Letters. 10:20131083.
Section 1) ATTEMPTS TO COMMUNICATE DIRECTLY WITH JEFF PODOS AND HIS STUDENTS
From July through December 2014, Podos (and students) refuse to respond to inquiries about Goodwin and Podos (2014), an NSF-sponsored research publication, contrary to all NSF guidelines about openness in use of tax-payer money. Podos silences me by using the UMass Amherst police to threaten me with criminal harassment charges if I attempt to communicate one more time with “UMass Biology” (in which I am emeritus). The Police further dictate that I am to tell ~50 international correspondents on this topic that none of them are allowed to contact Podos either.
Useful information regarding scientific research funded by the National Science Foundation is here: http://www.nsf.gov/bfa/dias/policy/dmp.jsp. In part, it states the following:
Dissemination and Sharing of Research Results . . . Investigators are expected to share with other researchers, at no more than incremental cost and within a reasonable time, the primary data, samples, physical collections and other supporting materials created or gathered in the course of work under NSF grants. Grantees are expected to encourage and facilitate such sharing.
Here is the “Ethics in Publishing” statement from the Animal Behavior Society, where Podos is the President of the Society (July 2017): https://www.elsevier.com/journals/animal-behaviour/0003-3472/guide-for-authors#5001 It states, in part, the following:
Professional integrity in the conduct and reporting of research is an absolute requirement of publication . . . , as is a willingness to share information with other members of the scientific community.
All correspondence for Section 1 can be found HERE.
Section 2) BIOLOGY LETTERS—attempts to address the issues publicly
From February to October 2015, I have a running dialogue with Biology Letters about publishing my objections to Goodwin and Podos (2014). Initially I am promised a public hearing on these issues, but all communication ceases after Podos submits to Biology Letters a confidential letter that has purportedly been written by a secret review panel at UMass. “Per University rules,” Biology letters cannot disclose to me the contents of this letter. The only name Biology Letters will give me is Dean John McCarthy (Vice Provost for Graduate Education and Dean of Graduate School; Distinguished Professor of Linguistics), whose name was apparently on the document.
But Dean McCarthy claims to know nothing about this letter. Dean McCarthy has “no idea” who wrote the document, has “no idea” who submitted the document, and had no role (“none”) in preparing or writing the document, nor is he interested in knowing who submitted this secret document in his name. Nevertheless, he does know that this document, the contents of which he apparently knows nothing about, is protected as a “student educational record” under the “Family Educational Rights and Privacy Act (20 U.S.C § 1232g) (b) (1),” which of course has nothing to do with protecting published authors from answering questions about their research.
So . . . Who wrote the secret communication to Biology Letters? And why must it be secret? And why must such great measures be taken to avoid a public exchange about published research? And where do university administrators receive their advanced training in stonewalling and obfuscation?
All correspondence for Section 2 can be found HERE.
Section 3) ANIMAL BEHAVIOR, the Journal; attempted submission of Forum Article
From September 2014 to January 2015, after seeking feedback from those whom I critique in my review article (as required by the journal), my initial attempt to submit a Forum Article is thwarted by an editor who is angry that I sought that very feedback as required by the journal. I am accused of many misdeeds. My proposed article is publicly (and angrily) rejected before it is even submitted, announced in emails sent to target authors Podos, Nowicki, Searcy, Vehrencamp, Goodwin, and Moseley (all of whom apparently wrote the editor and lobbied against the article).
During November 2015, with new editor Susan Foster, I resumed attempts to publish my review article, entitled “Birdsong Performance Studies: A Contrary View” (submitted manuscript available HERE).
That manuscript is my wide-ranging critique of the birdsong performance studies championed by Podos and colleagues, to be revised as a proposed Forum article at Animal Behavior. The first nine pages are devoted to refuting Goodwin and Podos (2014) by exposing its four fatal flaws. The first 21 pages are devoted to flaws in the studies of chipping sparrows and swamp sparrows by Podos and associates. In some candor (jovial compared to my final response to Podos, entitled “BULLSHIT is SCIENTIFIC FRAUD,” found here), I show how pseudoscientific methods have been used to confirm Podos’ hypothesis involving vocal deviation as a measure of birdsong performance.
Referee comments on above submission, and Kroodsma response
Self-explanatory. The Forum article was accepted mid-2016 and will be published early 2017.
All correspondence for Section 3 can be found HERE.
Section 4) ASSOCIATION OF FIELD ORNITHOLOGISTS (AFO)—I suggest that a best student paper should be retracted
Goodwin received a best-student paper award for her paper at the 2014 ornithological meetings in Rhode Island. Given the mistruths permeating the paper, I suggested to the AFO that it would be in their best interest to retract this award. It would send a strong message to all graduate students and their advisers that science matters. A second Podos student received the other best student paper award at the meetings, leading to a boast of a “clean sweep” on Podos’ website. After a long-time colleague of Podos (office mates in graduate school) was consulted by the AFO, my request for review was dismissed.
All correspondence for Section 4 can be found HERE.
Section 5) THE UNIVERSITY OF MASSACHUSETTS AMHERST—attempts to address scientific and ethical conduct with Deans, Vice Chancellors, Provost, Departmental Chairs, etc.
From January 2015 to September 2016, with vigorous hand-waving and tenacious institutional loyalty, but never once directly and independently addressing with me the scientific or ethical issues involved, my allegations of misconduct are dismissed by Departmental Chairs (Karlstrom, Connor), College of Natural Sciences Dean and Associate Dean (Goodwin, Powers), Vice Chancellor for Research and Engagement (Malone), Vice-Provost for Graduate Education and Dean of the Graduate School (McCarthy), and Provost and Sr. Vice Chancellor for Academic Affairs (Newman).
All correspondence for Section 5 can be found HERE.
Section 6) My correspondence with officers of the Animal Behavior Society, where Jeff Podos is now President. ALL CORRESPONDENCE IS HERE. During July 2017, I needed to add another section to this saga. In protest of the “leadership” of the ABS, and focusing on ethical issues, I resigned as a Fellow of the society. A few days later, the secretary of the ABS kindly informed me that I had been scrubbed from the Fellows roster. Period. Beyond that, silence.
In a follow-up to the Officers of the Animal Behavior Society, I suggest that maybe they should think twice about what kind of “scientific society” they want. Given the scientific and ethical issues involved, quietly accepting the leadership of the ABS as is no doubt sends the wrong message to anyone who cherishes science as a means to learning truths about the natural world. No young scientist entering the field of Animal Behavior should learn that this is how research is done and how one gets ahead. Silence again from the ABS leadership.
Third time, 27 January 2018. “Silence by ABS officers is complicity,”I try to tell them. There are serious consequences for complicity in the kinds of misbehaviors that I have documented. I ask for a professional response from them by 7 February 2018.
31 January post to the ABS officers: ABS OFFICER TUTORIAL: SCIENTIFIC FRAUD, COVER-UPS, AND THE CONSEQUENCES OF FAKE SCIENCE.
1 February 2018. ABS Officers to Kroodsma. We see no evil. None of this is our responsibility
As of 2 February, the final entry is this: SEVERE CONFLICT OF INTEREST RENDERS ABS OFFICERS IRRELEVANT ON ISSUES OF SCIENTIFIC AND ETHICAL MISCONDUCT.
July 2018. Oh, there’s more, of course. See “ALL CORRESPONDENCE IS HERE” (immediately below) for updates.
[9 February–taking a break for a few weeks. Giving officers of ABS some time to cogitate.]
Section 7) YOU WILL KNOW AUTHORS BY HOW THEY CITE THIS PERFORMANCE LITERATURE
As I write in my “A Sad Commentary,” “Authors choose the framework for their arguments and discussion but lose credibility when they rely on flawed research to reinforce their own work.” Let’s see how this whole topic plays out in the literature and where it goes.
You can find the updates here, in the file entitled “You will know authors by how they cite this performance literature.”
Section 8) More on scientific integrity. Podos and Cohn-Haft (Current Biology, 2019)
If interested in more of the same, read the original article by Podos and Cohn-Haft.
I provide a mini-review, revealing how three omissions from the literature enable the fantastic story about sexual selection. Yes, it is the lack of scientific integrity that once again enables this paper.
In this exchange with Current Biology, the four original reviewers defend their positive reviews of Podos and Cohn-Haft, and then a fifth reviewer (of my choosing, not the journal’s) discusses their conflict of interest, how Podos and Cohn-Haft is a perfect example of “story time” (see Gelman blog), and the deception involved with half truths, which are half lies.
Section 9) Jeff’s Big Splash in Darwin’s finches “couldn’t be replicated.”
Where could one make a bigger splash than explaining the evolution of Darwin’s Finches? Indeed, Jeff’s finch papers have been cited over 2000 times! Given all I have learned about Jeff since then, I thought I’d ask Peter Grant (the Darwin’s Finch guru) a simple question: “What has been the contribution of Jeff Podos to understanding the biology of Darwin’s Finches?” His answer came back quickly, even though he was traveling overseas:
From Peter Grant, 16 June 2023: “I was somewhat skeptical of the [Jeff’s] positive result [in his Nature paper] because it paid insufficient attention to the massive work of Robert Bowman (1983) on Darwin’s finch song. . . we have attempted to test the [Jeff’s] hypothesis of a beak-song relationship in our studies on Daphne in the same way that Jeff did, and all tests have yielded negative results . . . There is a more fundamental problem . . . I can add one more piece of contrary evidence . . . In summary the [Jeff’s] Nature paper was a stimulus to the field of bird song, but the stimulus has faded as we have failed to replicate the main result.”
Read the full correspondence between Peter Grant and me.
SUPPLEMENTARY MATERIAL
Goodwin & Podos_2014_Biology Letters
It was hearing this oral paper delivered at ornithological meetings in Rhode Island during 2014, and my subsequent reading of this paper in Biology Letters, that was for me the “last straw,” the culmination of so many papers with a veneer of science but no substance in this sexual selection literature.
Akcay & Beecher 2015 Critique of Goodwin & Podos 2014
After the June 2014 ornithological meetings, I communicated with Beecher at the University of Washington regarding the problems with Goodwin and Podos (2014). He and his student submitted a critique to Biology Letters; Akcay and Beecher deal with secondary problems and fail to address the really serious issues, but their critique eventually preempted mine at Biology Letters.
Goodwin & Podos_2015–strong defense of 2014
Goodwin and Podos mount a strong defense of their 2014 article, and appear on the surface to deal effectively with the minor wrist-slapping of Akcay and Beecher, as if these were just minor and insignificant differences of opinion on methodologies and interpretation.
Podos et al. (2012)
Here is a public prescription on how to do good science. Authors who had come very close to dismantling Podos’ performance hypothesis are lectured on all manner of topics: 1) proper measurements and methodology, 2) interpretation of data, 3) validity of results, 4) experimental rigor, 5) alternative explanations and hypotheses for data, 6) the ability to reject hypotheses, 7) appropriate use of skepticism, 8) problems in published papers that “undermine the validity of the results reported and the conclusions reached,” and 9) “basic principles” of science. Ironically, Podos and his coauthors are concerned, more broadly, with 10) how papers failing on these measures will “have a profound adverse effect on the way the research field is viewed by the rest of the scientific community.”
Here is the paper submitted and published after the advance copy of my Forum article was sent to Podos and several of his coauthors. One of the main points in Podos et al. (2016) is that “performance” is a characteristic of song types, not males; this is an entirely new admission by Podos, and matches my lengthy deliberations and conclusions in my Forum article. Another major difference from all previous publications by Podos, polar opposite to that by Zollinger, Podos, et al. (2012), again matching my Forum article, is that Cardoso et al. should be given major credit for pointing out this idea for songbirds.
Two of the main points from my Forum, which would not appear until the following year (2017), have now been appropriated by Podos et al., who would later, in Podos (2017), declare that my points are “. . . not surprising . . . Indeed, similar results have been reported elsewhere (. . . Podos et al., 2016).”