Credit: Courtesy Lloyd Klein

Even though Jocelyn Wildenstein is accused of attacking him, stabbing him with scissors, throwing hot wax on him and clawing him with her nails, designer Lloyd Klein says of their 13-year relationship, “It was a beautiful romance and a beautiful story. I wish it could have continued in the same way without all of the stress issues.”

Since the alleged attack last Wednesday at her Manhattan Trump World Tower apartment, Wildenstein, 71, is facing both second- and third-degree assault charges. In an exclusive PEOPLE interview, Klein says, “I’m devastated, emotionally devastated.”

Sitting down in a Manhattan hotel, Klein, 49, tells the story of their early romance: how he did not see Wildenstein, sometimes called “Catwoman” by the tabloids, the way others saw her. He also reveals what led up to their fight and her arrest — and then his arrest early Sunday morning when he returned to the apartment.

Both Klein and Wildenstein now have restraining orders against each other, according to a district attorney’s office spokesperson.

Mohammad Molaei Debut Showcase At SIR Studios In Hollywood

The couple first met in 2003 during New York Fashion Week and he invited her to see his designs. “She said she was enchanted by the collection,” he recalls. A few dinners and social events followed. They conversed in French. (Klein was born in Canada and raised in Paris. Wildenstein was born in Switzerland and also speaks French.) “We spoke the same language,” he says. “She understood high fashion. When attending fashion week in Paris, we knew the same people and we had a great connection. She had impeccable taste.”

“We became lovers,” he says. “We tried to meet as much as possible. We had a lot of fun and everything was going extremely well.”

When it came to her extreme plastic surgery, Klein says she always had “feline” features. “Of course, we see that she had some surgery,” he notes. “It’s obvious she had ‘lifting’ and she had some work done. She had some little repairs here and there, but it’s not that she tried to look like a cat, she had those features at 18, these same eyes, the nose, the cheekbones.”

He claims not to know what she had done during their years together. “If she traveled and did some alterations, I was not present,” he says. Still, he says he doesn’t view her the way others do. “I’m not shocked,” he says. “Beauty is subjective.”

Their relationship became more “stressed” in 2008 when she sold her Manhattan townhouse and moved to the Trump Apartments. “There were times when she went into rages. but I mean it’s human,” he says. “Each was okay on its own. It was a little shocking — but I thought she would get better.”

When they argued, he says, “she could take a cup or glass and throw it to the other side of the room.”

The pressure increased and on the night of Dec. 7, he says, “She was under stress, financial duress.” On top of that, they began to trade pointed barbs about their respective family members. “She went into a rage,” he says. “She took a candle that was burning and threw it on my face. My eyes were burning.”

Then he claims, “She scratched me with nails in the face. She has sharp nails. Then she said ‘Don’t get any closer to me or I am going to kill you.’ She stabbed me with a pair of scissors. Small ones. In the chest.”

“She is fragile, so it did not go far,” he notes.

After he went to wash his face in the bathroom, she walked into her closet and got locked inside. (Klein says he doesn’t know how she got locked in.) “I was packing my stuff,” he says. “She asked me to open the door and I said I don’t want to open the door because you are going to come at me again, let me finish packing and then I will open it.”

She called 911 and the NYPD arrived. “I did not want to press charges,” he says. “My plan was to clear the air and hope things go better and that life goes on and we both learn our lessons. I didn’t want her to be hurt in terms of being arrested or charged.”

He returned to the apartment late Saturday evening to pack up his belongings. He says she came running down to the lobby when she saw him leaving. “She was not injured when we saw her coming out of the elevators, she was okay,” he says. “She said she was locked out of her apartment. I was about to leave. And she requested my keys.”

After Klein got in a taxi to go downtown, he got a call from NYPD asking him to return to the apartment – after Wildenstein had claimed he’d attacked her and taken her keys. “That is not the case,” he says. “I didn’t take any of her belongings. I took my belongings — that is it.”

He was placed under arrest and charged with robbery, assault with intent to cause physical injury, grand larceny for taking a credit card and mischief with intent to damage property. “It was horrific,” Klein says of his night behind bars. “For a few hours I was without shoes, my belt and it was freezing and I was put in a cell with criminals.”

“I was completely lost,” he says. “I was shocked and I also thought about Jocelyn at the same time. I couldn’t believe she had [also] gone through this—that they put you in handcuffs and you go with the chain.”

Despite all the headlines and the drama, Klein now says, “I feel sorry for her and feel sad. I wish her the best. It’s a very unfortunate situation. I hope she will reunite with her children [Alec Jr. and Diane] because that is the main part of her sadness. I can’t speak for her but the lack of contact with them is not acceptable to my values.”

As for the future, he wants to return to his work and hopefully, calm. “Am I going back to her? I don’t think so,” he says “but it was a beautiful love story.”

Wildenstein’s attorney, Alan Kachalsky, told PEOPLE that Klein’s arrest should remind people that there are “two sides to each story.”

“The new arrest certainly complicates the situation,” Kachalsky said. “It should remind people that there are two sides to each story. People shouldn’t draw any inferences that she is guilty from only hearing Lloyd Klein’s version of the story. Now he’s going to see what it’s like to be the one arrested and spend 18 hours locked up.”

• With reporting by CAITLIN KEATING