Thomas Ravenel will not appear in Southern Charm season 6 as police continue their investigation of sexual assault claims levied against the former politician, announcing his decision to quit Tuesday.
“I’m not doing the show anymore,” he tweeted. “In the contract they have the right to fictionalize your story. They took advantage of me. I decided I’ve got too much to lose and informed them I’m not coming back.”
His Twitter page is currently protected.
PEOPLE confirmed in May that a victim reported an alleged incident with Ravenel that occurred in January 2015 to Charleston police. The incidents listed on the investigation report are “sex offense” and “forcible rape.”
And “Nanny Dawn” — the ex-caretaker to his daughter Kensie, 4, and son Saint, 2½ — claimed to PEOPLE that she is the one who came forward. She said she took action after hearing the story of real estate agent Ashley Perkins, who publicly accused Ravenel of assaulting her mom Debbie Holloway Perkins after they met on Tinder in October 2015.
Ravenel, 55, denied all the allegations through his attorney.
“My client enjoys a certain degree of fame, which has unfortunately made him a vulnerable target for such claims,” his lawyer Richard P. Terbrusch told PEOPLE in a statement. “The person this woman describes is simply not the man that I know. My client is a loving father, successful businessman, and upright member of the community. He is appalled and hurt by these allegations — and is committed to defending his reputation in the appropriate legal forum.”
RELATED VIDEO: Southern Charm’s Thomas Ravenel Accused of Sexual Assault, Allegedly Paid Accuser $200,000
“I have seen a new Thomas,” she said. “I mean, it’s humbling, of course, and now he’s [like] let’s stay in, let’s watch movies, let’s cook dinner, let’s just stay under the radar right now. And I’ve just seen a much softer, more sensitive, kinder, more supportive boyfriend. You know, everyone wants to be with you when you’re on top. Who’s with you when you’re down there? And I’ve told him, I’ve just said, I’m not leaving.”
As for whether she expects her boyfriend to be cleared of any wrongdoing, she replied, “Of course, yes. And it’s all hearsay. I mean, he’s not been charged with anything. These are just allegations. It used to be, you know, you’re innocent until proven guilty. Now you’re guilty until proven innocent.”
His castmates avoided directly addressing the allegations.
“I’m not going to comment until it’s resolved,” said Kathryn Dennis, the mother of his kids. “What I am going to focus on is just the children and staying out of it.”
“I think it’s the right decision for him not to be here,” added Cameran Eubanks.