His first visit to the island was in 1841. Douglass, who escaped slavery in 1839, attended anti-slavery meetings at New Bedford’s Zion Chapel. Where he described his life as an enslaved human being. William C. Coffin, a white man, was in the audience on at least one occasion. Nantucket-born Coffin was a bookkeeper in a local bank and an abolitionist. He approached Douglass to visit Nantucket to tell his story to a convention of the Massachusetts Anti-Slavery Society.
The Massachusetts Anti-Slavery Society traveled around the Commonwealth with an entourage of speakers. The three-day convention on Nantucket drew nationally recognized abolitionists including William Lloyd Garrison, Theodore Parker, Samuel J. May, and Wendell Phillips. They were drawn to the island because of the controversy over the island’s refusal to admit Eunice Ross to Nantucket High School on account of her color. The event was organized by Anna Gardner, the secretary of the local Anti-Slavery Society. Gardner had taught in the African School on the island for several years and was the teacher who encouraged Eunice Ross to apply to the high school. Gardner had resigned her job in protest when the school committee refused to integrate the schools.
The 23-year old Frederick Douglass rose to speak about his life. His account was so compelling that the Massachusetts Anti-Slavery Society hired him on the spot to join their staff as a speaker. It was the first time that Douglass had spoken to a predominantly white audience and his speech launched his career as an orator and spokesperson for black Americans. Samuel J. May wrote that Douglass “must have spent the night” in the New Guinea neighborhood of Nantucket, now known as Five Corners.
Douglass returned to Nantucket in July the next year to lecture about one of his frequent themes – the meaning of the Fourth of July to black Americans. “What, to the American slave, is your Fourth of July? I answer: a day that reveals to him, more than all other days in the year, the gross injustice and cruelty to which he is constant victim.”
Douglass returned in 1843 to attend another Anti-Slavery Convention, again held for three days at the Atheneum. The meeting was chaired by educator Cyrus Peirce who went on to direct the first public teacher-training school in the United State, the Lexington Normal School.
Douglass was either elected to, or appointed to, the Business Committee which proposed three resolutions. The first was that chattel slavery had to be abolished. The second resolution claimed that American churches were the “bulwarks of slavery” because of their affiliation with slave-holding congregations in the South. The third resolved that slavery in the South could not be maintained with the support of voters in the “nominally Free States.” All three passed.
Douglass next visited the island in October 1850 to denounce the proposed Compromise of 1850. California’s request to be admitted to the Union as a free state would upset the balance between slave and free states in the Senate. A complicated bargain was struck. To get Southern support, the Fugitive Slave Law was strengthened putting all black people in the North, including harsh penalties for anyone assisting a fugitive slave. According to the Inquirer, the Atheneum was filled to “its utmost capacity” to hear him speak. Douglass described the panic the bill was creating in black neighborhoods. Within a few days, Nantucketers created an integrated Vigilance Committee to protect Nantucket’s black community.
Douglass’s last visit to the island was in 1885 when he was 67. He stayed with his second wife, Helen Pitts Douglass, at Sherburne House at 30 Orange Street. He contrasted the visit with his first one when he was a struggling young fugitive. This time he was the guest of a reception at the impressive home of Matthew Starbuck, one of the “Three Bricks” on Main Street. His old friend, Anna Gardner, hosted another reception for him with 150 invited guests. While on the island, Douglass delivered two lectures, one about the Dutch revolutionary William the Silent, the second about the important role that John Brown had played in the prelude to the Civil War. “John Brown began the war that ended American slavery and made this a free republic.”
Douglass maintained a friendship with many abolitionists that he met when he first visited the island. Among them were Charlotte and David Joy and Anna Gardner. Gardner visited the Douglass home, Cedar Hill, in Anacostia, Virginia on two occasions. And, after Frederick Douglass’s death, Helen Pitts Douglass visited Nantucket and stayed with Anna Gardner at her home on Orange Street.