"It's tough, but people say that they pass through it and they come back to you," Mark Wahlberg tells PEOPLE of daughter Ella, 14

Mark Wahlberg‘s got his work cut out for him.

The 47-year-old actor dishes on his life as a father of four in PEOPLE’s latest issue, on newsstands Friday, and admits parenting 14-year-old daughter Ella Rae is “a challenge” in a lot of ways.

“It’s tough, but people say that they pass through it and they come back to you,” he says. “She wants to go be off and doing her thing.”

Adds Wahlberg, “[There’s] a lot of attitude, a lot of aggression, a lot of, ‘You’re the worst, you ruin everything!’ ”

Want all the latest pregnancy and birth announcements, plus celebrity mom blogs? Click here to get those and more in the PEOPLE Babies newsletter.

For the most part though, Wahlberg — whose children with wife Rhea Durham include Ella, Grace Margaret, 8½, Brendan Joseph, 10 next month, and Michael, 12 — is enjoying watching his kids grow up.

Screen Shot 2018-08-03 at 1.01.55 PM_preview
Credit: Mark Wahlberg/Instagram

“We’re going to take a vacation back East before they go back to school, I’m excited about that,” says the Boston-born-and-bred star. “Football season’s about to start for my son.”

RELATED VIDEO: Mark Wahlberg Talks Being a Dad to Four Kids

While Wahlberg’s oldest child may be testing her dad’s limits, her younger sister is still preoccupied with the innocence of childhood.

“My younger daughter is heavy into horses and all that stuff — lots of stuff that keeps her focused and away from boys hopefully until she’s 30, which is great,” he jokes of Grace.

For more about Mark Wahlberg’s life as a father of four, pick up the latest issue of PEOPLE, on newsstands now.