Barbara Walters Once Threatened to Quit 'The View' If Rosie O'Donnell Was Re-Signed

The long-time journalist and the comedian only appeared together on the daytime talk show for one season

Barbara Walters launched The View in 1997 following an already decades-long, revered career as a journalist.

She hand-picked Joy Behar, Star Jones, Debbie Matenopoulos and Meredith Vieira to join her as the initial panel, and added Rosie O’Donnell in 2006 after Jones got the boot.

But by the next year, O’Donnell — who had years earlier completed six seasons of hosting her own daytime talk show, The Rosie O’Donnell Show — fell out of favor with Walters. The comedian caused The View to gain some negative attention from high-profile viewers like Kelly Ripa and Donald Trump, and O’Donnell did not get along well with then-executive producer Bill Geddie, according to Ramin Setoodeh’s forthcoming book, Ladies Who Punch: The Explosive Inside Story of The View.

Lou Rocco/ABC via Getty Images; Heidi Gutman/ABC via Getty Images

So to save the show — and her reputation as a respected, legitimate news anchor — Walters wanted O’Donnell out.

“If you re-sign Rosie to this show, Bill and I are going to quit,” Walters, 89, told ABC Daytime President Brian Frons in 2007 over dinner at Spago in Beverly Hills, according to Ladies Who Punch.

RELATED VIDEO: No Feud Here: The View’s Meghan McCain & Joy Behar Are ‘Totally Professional,’ Source Says

O’Donnell, 57, did not return for the following season, and only made a brief comeback in 2014 after Walters left.

The Long Island, New York, native went on to write a memoir, titled Celebrity Detox (The Fame Game), in which she called out Walters and recommended that the then 77-year-old should retire.

Unhappy with her part in the book, Walters had ABC leak her mention in Celebrity Detox to the New York Post, asking the outlet to include a psychologist evaluation of O’Donnell’s mental health when covering her memoir, according to Ladies Who Punch.

• For more on The View, pick up the latest issue of PEOPLE, on stands Friday

The article came out in September 2007 with a psychiatrist calling O’Donnell “full of rage” and suggesting she “dishes out anger mostly to women because of deep-seated abandonment issues over her mother’s death.” The piece also included a statement from Walters, in which she called O’Donnell’s memoir a “sad book, but I prefer to focus on the happier times we had and the happier times we hope to have in the future.”

Ladies Who Punch: The Explosive Inside Story of The View is out April 2.