Jessica Simpson was 'Very' Pissed at Nick Lachey for His Past Comment About Her Dad Joe Simpson

"That was low," said Jessica Simpson of her ex-husband Nick Lachey's past remark about the relief he felt in no longer having her dad as a father-in-law

Jessica Simpson could have done without one particular comment from her ex-husband Nick Lachey concerning her father.

Speaking with Andy Cohen on his SiriusXM show Radio Andy on Wednesday, the singer, 39, shared her reaction to a past remark that Lachey, 46, made on a years-old episode of Cohen's talk show, Watch What Happens Live with Andy Cohen.

In the old clip, Lachey said the best thing about not having Joe Simpson as a father-in-law after the couple's split in 2005 was that he no longer had to "play grab-ass under the table on Easter Sunday."

"That was low," Simpson said in response. "And that's, like, not for him to talk about by any means, because he's not connected with us at all anymore."

The mom of three said she was "very" pissed by Lachey's comment. "He was trying to — he was just bitter, you know? He was just trying to take a dig," she said.

Among the various bombshell revelations Simpson uncovers in her new memoir Open Book — which Lachey said he hasn't read yet — is her complicated and close relationship with her dad, who was also her manager.

As she writes, on her wedding day in 2002, her father asked if she was certain she wanted to go through with it, just before he walked her down the aisle. "Dad, I'm walking down this aisle and getting married," Simpson writes, describing how she spoke the words "through gritted teeth."

Simpson and her dad also went through a rocky patch after her parents divorced (which Joe told his daughter about just before she gave birth to her first child) but have since reconciled and the two remain very close.

Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human interest stories.

Kevin Mazur/Getty; Charles Sykes/Bravo/NBCU Photo Bank via Getty

The star, who is now married to former football star Eric Johnson, opened up to PEOPLE about her marriage to Lachey, saying that looking back, she respects him "very much."

"We were young and pioneering our way through reality television, always mic'd and always on," she said. "We worked and we were great at it, but when it came time to being alone, we weren't great at it anymore."