"She's happy with three boys," swimmer Michael Phelps tells PEOPLE of his wife Nicole's opinion of living as the only woman in the house
The retired Olympic swimmer has three sons with Nicole: Maverick Nicolas, whom the couple welcomed in September, Beckett Richard, 20 months, and Boomer Robert, 3. And while it’s a home comprised of boys, Phelps says his wife has voiced her contentment with being the lone woman among her family unit — which prompted an adorable reply from their oldest child.
“I think Nicole said something along the lines of, like, ‘I’m so lucky to have a house full of boys and no girls,’ and Boomer looked at her and said, ‘Mommy, you’re the girl in our house,’ ” says Phelps, 34. “So we’ve been saying that and it’s true, she’s the queen.”
“She’s the queen of the household and she’s the only lady in there, so she likes it, I think. She said she enjoys it,” he adds of his wife.
Never miss a story — sign up for PEOPLE’s free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human interest stories.
After making a conscious effort to have children close in age, the proud father says he’s delighted to see his older two boys get along so well and develop a tight bond.
“I think the biggest thing for me is we wanted them as close as they possibly could be so they grow up together,” he says. “Being able to see how much love Boomer has for Beckett and Beckett has for Boomer, but also seeing how much they want to be around the baby all the time — for me, it’s just an amazing thing to see as a dad.”
With the birth of each child, Phelps says he’s understood love in new ways, and that fatherhood continues to change him as a person. “I’m learning more about myself going through this process than I ever thought I would,” he says.
Finding time to connect with his wife, even with their hectic schedules in the way, isn’t out of reach, according to Phelps, who says he and his “perfect person” Nicole go on hikes or venture on other outings whenever possible.
“We’re always making sure that we’re taking that time for a date night or whatever it might be,” he says. “That’s something that we always have to have, that connection, that one-on-one.”
RELATED VIDEO: Michael Phelps’ Pregnant Wife Nicole Says “There Is Always Guilt” as a Mom “No Matter What”
As for the talkative Boomer — whom his dad lovingly refers to as “Booms” — Phelps says his oldest son is quick with chatting back at him. “The one-liners that come out of his mouth are priceless,” he says, laughing.
Raving about his kids, Phelps says the good in parenting outweighs any downsides, and that his family brings him joy beyond any measure.
“Sometimes [things the boys do] will piss me off in a way,” he says. “Booms really presses buttons on me better than any human being out there, but he’s just my mini-me, so for me, [it’s] just the best feeling.”
The athlete adds, “It’s hard, I guess, to put into words.”