"She wanted to make it work," a source tells PEOPLE of Vanessa Trump's marriage to Don Jr. "But it can't be easy being a member of that family"

Of all the stresses that reportedly pulled at the 12-year marriage of Vanessa and Donald Trump Jr. — his racy tweets to a former model, alleged affair with a Celebrity Apprentice contestant, and cheapskate approach to Vanessa’s spending among them —friends tell PEOPLE in this week’s new issue that the strain between husband and wife was elemental and nothing new.

The couple had been living “separate lives” for a while, says a source who socializes with them. “Even before all this Trump presidency stuff started, they weren’t good and their marriage was strained.”

And for Vanessa, it was made worse by the glare attending her father-in-law’s campaign and then presidency. Months ago, Vanessa confided to a Palm Beach, Florida, friend that security fears “were stifling.” Her five children with Don Jr., ranging in age from 10 to 3, needed armed Secret Service escorts even in the elevator of their New York City apartment building, she complained.

Credit: Larry Busacca/Getty

Meantime, Don Jr. was running the embattled Trump Organization with his brother Eric, battling on Twitter with their dad’s detractors and facing uncertain legal jeopardy in Special Counsel Robert Mueller’s investigation. “Don Jr.’s become isolated, distant and just plain angry,” says a source at President Trump‘s Palm Beach Mar-a-Lago resort.

For more on the Trump divorce, pick up the new issue of PEOPLE on newsstands Friday.

RELATED VIDEO: Vanessa Trump Was Sister-in-Law Ivanka’s ‘Wonder Woman’ Before Calling it Quits with Donald Jr.

Adds the source, who knows Don Jr. and the Trump family well: “He has not been easy to be around.”

Vanessa, a devoted mom by all accounts (sister-in-law Ivanka Trump raved to PEOPLE in 2016 that Vanessa is a “Wonder Woman” with children), has always prioritized her kids, says her former agent at Elite Model Management, Christine Schott.

“She told me on her wedding night that she hoped to have five children,” Schott says. “She wanted to make it work. But it can’t be easy being a member of that family.”