The Full House stars, 30, recently told Net-a-Porter’s The Edit that their focus these days is split between work (they head up lifestyle brand Elizabeth and James and high-end fashion line The Row) and personal commitments.
The Olsens’ former longtime publicist, Michael Pagnotta, tells PEOPLE he has always admired the sisters’ work ethic and remembers when they decided to leave a life in front of the camera and enter the fashion world.
“They came to New York and wanted to go to college,” says Pagnotta. “They’d been kind of hamster-wheeling their lives, and through the first year of college, when they got to NYU, they actually met people who had a different take on the world. They maybe got a different sense of what their future might be. And maybe it wouldn’t necessarily be in acting.”
He says at this time, the Olsens also realized working in fashion was what they most enjoyed.
“They understood it and seemed to have a knack for it,” says Pagnotta.
For more on Mary-Kate and Ashley Olsen, check out this week’s issue of PEOPLE, on newsstands Friday
He says another possible reason they retreated from the spotlight was that everything got “a little too big,” citing their fame from Full House and the task of running their own brand of products at Walmart.
“I think that as it got bigger and bigger, when the Walmart thing happened, you could see it getting maybe a little too big and overwhelming even, and the ability to remain normal, in the light of that fame and wealth and responsibility.”
But he says through it all, the girls have always had a good head on their shoulders, and he credits that to their strong upbringing.
“They were always part of a very loving family that was aware they were kids and needed to have a real life,” says Pagnotta “The most important thing people should know about them was they were allowed to be kids. They had the normal kind of concerns, the normal kinds of fears, and the same kind of fun.”
“They went through a period of “rebellion” in their teens, he says, but “despite their fame and wealth, they are normal girls for their age.”
These days, Mary-Kate — who married French banker Olivier Sarkozy, 47, in 2015 — is focusing on her new family when she is not working.
“I have a husband, two stepkids and a life. I have to go home and cook dinner,” she told The Edit.
In her free time, “I ride horses on the weekends. You find the thing that helps you relax, and if you don’t have it … you get burned out.”
From Coinage: See Where 6 Stars Were Before They Were Famous
Meanwhile, Ashley spends off-hours with her boyfriend, financier and art collector Richard Sachs, 58.
Pagnotta tells PEOPLE that from a very early age the Olsens were able to find that important balance between work and a personal life.
“They were ambitious and hardworking and talented, driven, but they also knew that ‘no’ was a full sentence,” says Pagnotta. “They were comfortable enough to tell grown men and women around them when it was enough.”
Pagnotta adds, “I would say that they are normal girls for their age, despite their fame and wealth, and they are also just really good-hearted.”