The Essay
Introduction
Hoover’s War on Gays was, among my three books, my largest, most laborious, yet, without a doubt, the most enjoyable and satisfying research project of my academic career. It was not only a subject deserving comprehensive treatment, it was also a subject of great personal interest.
I started researching it in 2003, a few months after completing my dissertation. It was a slow process, as I was concurrently busy converting my doctoral thesis into my first book.
The easiest and most obvious starting point was collecting the extant FBI files I knew existed — such as relevant parts of Hoover’s secret office files, and those on Sumner Welles, David Walsh, and other singular targets — and compiling everything I knew was written on the subject.
In the process of reading everything in any way related to the topic, I began to identify subjects for Freedom of Information Act requests. Researching the FBI is necessarily based on FOIA requests, a time consuming, often expensive, and sometimes frustrating process.
Obscene File
Besides the obvious FOIA requests on gay rights activists and groups, I decided the FBI’s Obscene File must contain gay-related information. Scholars knew very little about this File.
While on a research trip to Milwaukee, I talked about researching the Obscene File with Athan Theoharis, the most prominent of FBI historians. He thought that asking for this File might not be fruitful because the Obscene File was voluminous and locating information on gays through a FOIA request was probably unrealistic.
File on the Obscene File
I decided to submit a FOIA request for the Obscene File to see what I would get. To my astonishment, about a year later I received a response telling me the Obscene File itself had been destroyed in the 1990s. But an administrative file of the Obscene File existed (and was of reasonable size), and I got it. This file about maintaining the Obscene File indeed contained information about gays, but it also convinced me it warranted its own monograph. This file revealed significant new information about the FBI’s Obscene File itself, so I decided a study about it needed to be completed first. This would help me better understand the FBI’s efforts against what they considered smut, and the bureau’s broader, related interest in gays. Thus my second book, The FBI’s Obscene File (2012), briefly delayed my research on the FBI and gays.
FBI and Gays
During my first book’s editing, I starting writing an article on my early FBI and gays research. I had obtained the FBI’s file on the Mattachine Society and ONE, Inc. It was only during this writing process that I began to make nuanced connections and truly began to comprehend my subject.
Redactions
A major problem with FBI research in general, and Mattachine and ONE in particular, was dealing with all the redacted names in the FBI files. While frustrating, the process was also currently exciting because foiling government redactors is a thrilling experience. It forces one to dig deep and employ the best of historical deductive-reasoning skills (a satisfying feeling I have come to refer to as “Sherlocking”).
To get around blacked-out names, one needs to know as much as possible about one’s research subject. In this case that meant compiling biographical information about as many Mattachine and ONE members as possible. While a name might be redacted in an FBI file, the contextual information appearing around the redacted name (and the number of type-written letters in the redaction) can often but not always reveal it.
With the help of a brilliant undergraduate research assistant, we compiled dossiers full of detailed information about gay rights activists and we uncovered many names which revealed the history of FBI surveillance. (See images 1 & 2.)
As preliminary results of my research and my entrée into LGBT history, I then published an article on my Mattachine/ONE research (which became, in part, the basis of chapter 5), and then another article on the FBI and Harry Hay (also a part of chapter 5). These two articles forced me to think more deeply about the broader topic of FBI surveillance of gays.
I then began to put together the history of some subjects from the Cold War era about whom other historians had already written. I also acquired the FBI files on the Gay Liberation Front and Gay Activists Alliance, two groups about which the scholarly literature was thin and the literature about the FBI’s interest non-existent.
At this point I was not sure how to proceed with my research. I vividly remember thinking to myself, “Good Lord, where do I start?” I decided to turn to the origin of the FBI interest in gays. Already discovered FBI documents told us that the Bureau started collecting data on “Sex Offenders” (including gays) in 1937. Why 1937, I asked?
It didn’t take long to learn there was a mid-1930s sex-crime panic during the Great Depression. Yet I still had no idea what, exactly, sparked the FBI’s interest. Knowing 1937 was the key year for the FBI, I tried to ascertain the start of the panic.
“Sex Offenders”
Historian Estelle Freedman reported that the New York Times created a sex crimes index in 1937. I searched this index, and located the name Charles Mattson, a child who had been kidnapped just after Christmas 1936, and murdered in early January 1937. The date of the case suggested it might be the start of the FBI’s interest in “Sex Offenders.” The existing scholarly literature confirmed that “Sex Offenders,” given contemporary biases, primarily meant gays.
I discovered that news articles about the Mattson case pointedly mentioned the FBI. They even mentioned Hoover having mistakenly claimed that child kidnappings had ended two years before the Mattson case. So I made a FOIA request on the Mattson case, hoping to find some sort of document referencing the FBI’s start in collecting information on gays.
The FBI’s response was stunning. I expected a letter detailing the size of the file and what it would cost me, at 10 cents per page after the first free 100 pages. I didn’t receive a letter, though. While on campus, teaching a class, I received an unexpected phone call from the FBI. The Mattson file constituted a whopping 220,000 pages! (I was anticipating, at best, a few thousand pages.) Practically speaking, it would be impossible for the FBI to process a file this large in a timely fashion, and it would have cost me tens of thousands of dollars.
I decided to narrow my FOIA request on the Mattson case to the start of the file, and the year 1937. But the gargantuan size of the Mattson file revealed that I had hit the mother lode: there was no FBI interest in “Sex Offenders” prior to Mattson, and then boom!
I next researched the FBI’s surveillance of the East Coast homophile movement, including Frank Kameny’s Mattachine Society of Washington (MSW), the New York Mattachine, Donald Webster Cory, and several others. I was successful in obtaining, via FOIA request, the FBI’s combined file on those and other Mattachine groups.
D. B Huggins
Thanks to John D’Emilio’s work, and especially David Johnson's, I knew there was an early Mattachine group in Washington D.C. led by “D.B. Huggins,” about whom we knew little. I didn’t seem to have any FBI files referencing him or his group. I was wrong. FBI redactions made it appear there were no references, but in fact there were.
I had recently acquired from Princeton’s archives a letter Huggins had written to the ACLU. This letter detailed Huggins’ experience and even expressed his belief the FBI had targeted him. Upon reviewing (for the umteenth time) my Mattachine-related FBI files I began to recognize some of the details Huggins had described in his ACLU letter. I experienced one of those ah-ha moments, and remember exclaiming “This is Huggins!” Fairly quickly, then, his story came together, helped by a FOIA request on Huggins. The FBI released to me only Huggins’ correspondence with the Bureau, a couple of internal FBI documents listing his various Washington jobs in the 1950s, and some previously unknown facts about him in bureau notes attached to his correspondence (see image 7). It turned out these new facts about Huggins, especially his employment history, helped me to foil further redactions in the broader Mattachine file (see image 8). I had succeeded in uncovering a seemingly lost episode in LGBT history, and previously unknown information about Huggins.
Image 8
Donald Webster Cory/Edward Sagarin
The same was true with Donald Webster Cory, the pseudonym of early gay rights activist Edward Sagarin. Again, because an FBI redactor failed just once to black out the surname of an informant, I was able to recover a significant example of how the Bureau collected information on gays from therapists and, moreover, information about a previously unknown, early gay rights group Sagarin had organized called Homosexuals Anonymous. (See images 9 & 10.)
Kameny’s Mattachine
I next turned to Kameny’s Mattachine Society of Washington, DC. There were clearly documents in the broader Mattachine FBI file about the MSW and Kameny (whose name was redacted throughout, except for his first name appearing unredacted in one document [see image 11]).
I had earlier made a FOIA request on Kameny, but the FBI claimed to have nothing. This surprising response suggested that the FBI had destroyed Kameny’s individual file, and possibly the broader Mattachine file that I had already obtained. My only recourse was to research the recently opened Kameny papers at the Library of Congress, and I traveled there in May of 2014.
The Kameny papers contain a wealth of information. They included some FBI files he had managed to gain access to via the FOIA, some of which were copies of files I already had. Some were documents not released to me, but that clearly should have been released to me (exemplifying problems dealing with the FBI & FOIA). These new files allowed to me get around some redactions in the FBI files I already had, and they revealed new information about the FBI’s targeting of Kameny and Mattachine.
An Informer in Kameny’s Mattachine
One of the major discoveries I made was the identity of an FBI informant in the Mattachine Society of Washington. In the FBI files released to me, a redactor just once neglected to black out the surname of an informant -- an individual named “Scarberry.” I didn’t know his given name, or anything else about him, and my research efforts drew a blank. I hoped the Kameny papers might reveal something: if this Scarberry was, indeed, an MSW member surely I’d run across his name. I wasn’t too confident of this, however, knowing most MSW member used pseudonyms. I was lucky, and I still vividly remember the moment. As I was turning page after page of manuscript material I finally turned one page and there, right in front of me, was a letter from MSW to a Mr. Warren Scarberry of Suitland, Maryland. I gasped! My heart raced. I had identified an FBI informant, and later learned far more about him than I ever expected. (I was able to identify other FBI informants, including some significant ones, in similar ways.) (See images 12 to 15.)
The Sex Deviates Program and File
My Library of Congress research trip was just about up, and I was ready to drive home the next day. But I still had to figure out how to describe the FBI’s Sex Deviates Program and file.
Athan Theoharis possessed the FBI’s requisition to the National Archives seeking permission to incinerate the Sex Deviates file, and the file was destroyed in 1977. We had only a general description of the file from those National Archives documents, plus indirect references in FBI files. (See image 3, 4, & 5.) I feared my treatment of this important file would be wholly unsatisfying
These were my thoughts in my Washington hotel room right before checking my email. Unexpectedly, there was a message from a New York Times reporter, Matt Apuzzo, who wanted to interview me about the FBI and gays. Apuzzo was stationed in the capital, and I told him I happened to be there as well. So we met for tea.
Apuzzo told me that, in DC, Charles Francis was working with a law firm to gain access to FBI files on gays. He then showed me a document that looked familiar, but upon closer inspection I realized I had never seen it before. It discussed the FBI’s Sex Deviates program. It was the original Sex Deviates Program policy document!
For a second time in one research trip I had ah-ha moment. Considering how little was known then about the FBI and gays, I suspected that Francis and Apuzzo couldn’t fully appreciate the import of this document. It was one that I and Athan Theoharis originally thought was incinerated along with the rest of the Sex Deviates file. Instead, here it was! (See image 16.)
We had previously thought that the Sex Deviates Program dated to June 1951 because that date was referenced in the indirect document (image 3 above). But my research uncovered the earlier version of the program, dating to April 1950. That was a simpler fingerprint and arrest record program (see image 6), markedly different from the June 1951 Sex Deviates Program reconfiguration.
The document detailed the policy and procedures of the Sex Deviates Program, including what to index and how. I couldn’t believe it. This, I knew, would be the core of my new book. This document was clearly the single most important FBI document about gays ever released. Quite unexpectedly, I would be able to describe the Sex Deviates Program in detail. Despite the destruction of the actual file, I would be able to reconstruct a reasonable understanding of the FBI’s Sex Deviates file and program.
The finished book, Hoover’s War on Gays: Exposing the FBI’s “Sex Deviates” Program, explores what extant evidence tells us about the FBI’s obsession with homosexuality from 1937 to 1993.
After a discussion about whether FBI Director J. Edgar Hoover was gay and whether that matters, the book explores the evolution of the FBI’s relationship with gays and lesbians. The FBI's interest in gays began amid a Depression-era sex crime panic. It evolved to perceive gays in a limited way as security risks during the Second World War, to seeing them as major security risks during the Cold War. That led to the creation of the formal Sex Deviates Program and file, and the targeting of gay rights groups, first on the west coast, then the east coast.
The book examines the political issue of homosexuality during both the Johnson and Nixon administrations (that includes the Walter Jenkins episode and Nixon seeking FBI information on gays in the press corp). It surveys the FBI’s targeting of the gay liberation movement, and ends with the destruction of the Sex Deviates file in 1977-78. It discusses the FBI’s unintentional lead in ending federal gay employment discrimination in the 1990s. It’s a vast, complex, and varied history, with multiple causal factors, spanning more than 50 years.
The Author
Douglas M. Charles is Associate Professor of History, Pennsylvania State University, Greater Allegheny Campus. His two earlier books are: The FBI's Obscene File: J. Edgar Hoover and the Bureau's Crusade Against Smut [1910-2011] (Lawrence: University Press of Kansas, 2012), and J. Edgar Hoover and the Anti-interventionists: FBI Political Surveillance and the Rise of the Domestic Security State, 1939-45 (Columbus: The Ohio State University Press, 2007).