Mary-Kate Olsen and Olivier Sarkozy Clashed Over Having Kids: Source

Mary-Kate Olsen, 33, and her husband, Olivier Sarkozy's, 50, inability to see eye to eye on several issues ultimately led to their divorce after four and a half years of marriage.

According to a source, one of the main issues that caused friction in their relationship was whether or not to have children. Sarkozy, who has two grown children with ex-wife Charlotte Bernard, didn’t want to have any more kids, while Olsen did.

“A few years ago, having a baby was not a priority for her. This has changed,” says the source.

On April 17, Olsen privately signed papers seeking a divorce from Sarkozy, but because of the coronavirus crisis, New York City courts haven’t been accepting divorce filings except in cases of emergency. On May 13 Olsen requested an emergency order to file for divorce, citing fears of being forced out of their apartment. Her request was later denied by a judge.

"Olivier has two grown kids and doesn’t want more," a fashion source tells PEOPLE. "Mary-Kate thought he would budge on that and he hasn’t.”

For more on Mary-Kate Olsen, pick up this week's issue of PEOPLE on newsstands Friday.

But it wasn't only children that drove the fashion mogul and banker apart.

“Olivier is a party boy and a big spender,” says a banking source. “He loves the high life.”

Meanwhile, Olsen who runs two clothing brands, The Row and Elizabeth and James with her twin sister Ashley, is more reclusive.

“Mary-Kate prefers to be quiet and alone and doing the things she likes,” says a fashion source. “She’s not that interested in socializing—she likes small gatherings with select people. They ended up being apart quite a bit.” 

• Reporting by Pernilla Cedenheim and Linda Marx