Luann D'Agostino Doesn't Feel Foolish About Quick Marriage: Source

Luann D’Agostino may have only been married to her husband Tom D’Agostino, Jr. for seven months, but the Real Housewives of New York City star doesn’t regret their love affair.

A source close to Luann tells PEOPLE that despite their relationship not working out in the end, Luann still loves Tom — and the feeling is mutual.

“They don’t want to drag each other under the mud,” the source said, adding that Luann and Tom had last spoken on Wednesday. “They both love each other. They care about each other. They just could not be married.”

Luann announced via Twitter on Thursday that she and Tom are calling it quits. “It’s with great sadness that Tom & I agreed to divorce,” the former countess wrote. “We care for each other very much, hope you respect our privacy during this sad time!”

Hours after sending the tweet, the 53-year-old reality star was spotted out in Sag Harbor, New York without her wedding ring — wearing a yellow sundress, matching sandals, black shades and carrying a brown purse.

The Luann insider says she’s spending time with her kids in the Hamptons before jetting off to Europe on vacation.

EXCLUSIVE: Countess Luann de Lesseps drops off her car for repairs the day after she filed for divorce in Sag Harbor — Christopher Peterson/Splash News Online

Luann and Tom’s divorce announcement came nearly a week after Tom was reportedly spotted flirting with a blonde woman in Sag Harbor while Luann was at Jill Zarin’s luncheon in the Hamptons.

Although the couple denied the allegations to Page Six, which first reported the news — with Tom’s rep called the sighting “total rubbish” — the source tells PEOPLE that Tom had a history of flirtations encounters and wasn’t necessarily comfortable being married.

“I don’t think Tom had any actual affairs. There were women he kissed when he was out and Luann couldn’t turn a blind eye to it any more,” the source said, adding that Luann didn’t know any of the women personally. “‘He just wanted to be a bachelor and maintain his bachelor lifestyle. He wanted to eat and drink with his friends and go out and have a good time.”

Luann-Tom-DAgostino — Robin Marchant/Getty

RHONY viewers certainly got that impression from what was depicted of their whirlwind relationship on the show.

The first sign of trouble came from costar Bethenny Frankel, who presented photographic proof to Luann in season 8 that Tom had been unfaithful the day before their Palm Beach engagement party — making out for more than an hour with a Playboy model at New York’s Regency Hotel.

Though Luann opted to forgive Tom for his indiscretion, her costars couldn’t seem to let it go. Leading up to Luann’s New Years Eve wedding in season 9, they continued to confront Luann about Tom’s rumored indiscretions. Luann’s friend Barbara Kavovit was even caught on camera in one episode asserting that the reality star was running from the infidelity rumors about her husband-to-be. Said Kavovit, “I think she has something to prove. She’d rather go through with it and get divorced than call it off.”

Still, Luann and Tom went through with the wedding. And weeks after the nuptials, Tom called himself a “dog with a collar” while chatting with an ex at a party seen in a season 9 episode.

RELATED VIDEO: A Look Back at Some of the Highlights of Luann de Lesseps & Tom D’Agostino Jr.’s Wedding

So why did Luann say “I do” in the end when all the signs where there? According to a source, “She went ahead with the wedding because she was very much in love and wanted to make the marriage work. Even through what she heard or what was printed, she was trying to make it work.”

She also “doesn’t feel foolish” about how short her marriage to Tom lasted, according to the source.

“They had a real quick love affair and she wants to be married,” the source said. “She was married for a very long time and had two children. She comes from a large family in Connecticut and all her brothers and sisters are married. She’s a person who likes being with one person and married – she doesn’t want to be single.”

The insider explained the couple had a prenup, and that the divorce will be quick.

The Real Housewives of New York City airs Wednesdays (9 p.m. ET) on Bravo.

Luann D'Agostino Doesn't Feel 'Foolish' About Her Quick Marriage to Tom: Source

A source tells PEOPLE Luann D'Agostino still loves Tom D'Agostino despite divorce announcement, and "they don't want to drag each other under the mud"