October 10, 2016 02:45 PM

Tom Brady may have a busy season ahead of him, but his most important job is still as dad to his three kids — and that includes being tough when he needs to be.

“I need more patience with my kids,” the New England Patriots quarterback, 39, tells PEOPLE. “I have a quick fuse sometimes, so when they’re older I imagine them going, ‘Dad, you were so hard on me.’ “

Adds Brady, who is dad to 3½-year-old daughter Vivian Lake and 6½-year-old son Benjamin with supermodel wife Gisele Bündchen, as well as son Jack, 9, from a previous relationship, “I think discipline’s important, and typically that’s more the man’s role in the house.”

Want all the latest pregnancy and birth announcements, plus celebrity mom blogs? Click here to get those and more in the PEOPLE Babies newsletter.

When asked if he considers himself a strict parent, the dad of three clarifies that he think there’s a difference between being too tough and being consistent with rule enforcing.

“I wouldn’t say strict, but I’d say disciplined, because there are definitely times when they get over on their dad,” he says, admitting he’s not at all tough on Vivian.

“No way. It’s too hard — she’s too cute, and she just says, ‘Dad, please?’ and I say ‘Okay,’ ” he confesses. “My wife always says, ‘You gotta be tougher on her!’ and I say, ‘I can’t.’

“[Vivian]’s, like, half my wife, so you know … that’s just the way it goes.”

Brady says he learned loyalty, compassion and empathy from his own parents, which he hopes to pass on to his kids.

And according to the NFL star, his own apples haven’t fallen too far from the tree.

“There’s definitely times when they stop and I go, ‘Whoa, yeah, that’s my kid,’ ” says Brady with a laugh. “When they pout and when they don’t get what they want and they stammer around the house — my daughter will cross her arms.

“It’s just the best … I’m a little bit like that,” he admits.

Jen Juneau

You May Like

EDIT POST