When Jimmy Kimmel welcomed his first two children, Kevin and Katie, he was not the star he is today - nor did he have the same income
Credit: Bobby Doherty/New York Magazine

Parenting has certainly changed for Jimmy Kimmel.

When the talk-show host welcomed his first two children, Kevin and Katie, he was not the star he is today — nor did he have the same income.

Posing for the cover of New York Magazine‘s latest issue, Kimmel told Vulture that he had his daughter when he was 24, and the family struggled to make ends meet. He lost his job doing morning radio in Tampa, Florida, so he had to move the family to Palm Springs, California.

“We moved across the country, and I would work from 4 in the morning till around 11 in the morning, and at the time my ex-wife worked too,” he explains. “So I’d pick Katie up from day care as quickly as I could, because we couldn’t afford the $5 an hour to keep her there longer.”

Adds Kimmel, 49, “I’d have her all day till 6 or 6:30, when my ex-wife got home. That was a hard schedule. I was exhausted all the time.”

Want all the latest pregnancy and birth announcements, plus celebrity mom blogs? Click here to get those and more in the PEOPLE Babies newsletter.

Credit: Bobby Doherty/New York Magazine

RELATED VIDEO: Jimmy Kimmel Gives Health Update on His Son and Says He Needs Two More Surgeries

The actor and comedian says things with his wife Molly McNearney and their two young children — 6-month-old son William “Billy” John and daughter Jane, 3 — are very different.

“It’s much easier now, even though I’m busier, because we can afford to have a nanny. We both work full time, so we need that — but we can also afford it,” he shares.

“Most of the pressure I felt with Katie came from me being a 25-year-old guy who had to keep the family afloat,” Kimmel explains. “It was not easy.”

8th Annual L.A. Loves Alex's Lemonade
Credit: Vivien Killilea/Getty

Even though his family is living comfortably now, Kimmel makes it a priority not to spoil his children.

“Raising kids with money is a tricky thing,” he says. “You don’t want to just hand them everything. I don’t know the best way to go about it. I’ve concluded that pretending you don’t have money is not the best way, because kids aren’t stupid.”

“I can’t say what my parents said, which is, ‘We can’t afford the $60 to send you on a band trip.’ That won’t fly,” Kimmel notes.

Credit: Jimmy Kimmel/Twitter

FROM PEN: Girls Trip Star Jada Pinkett Smith on Being There for Son Jaden During The Pursuit of Happyness

RELATED: Mila Kunis Refuses to Raise Entitled Children: “Mommy and Daddy May Have a Dollar, But You’re Poor”

He continues, “The most money I ever made when my first two kids were little was $75,000 or something. I try to discuss this with people who have kids that are a little bit older than mine, and it’s always the small advice that’s the best advice: Make sure they do chores and get a reasonable allowance.”

“You know, there’s that urban legend about the kids who have a private jet and the first time they fly commercial they ask their father, ‘Why are all these people on our plane?’ ” Kimmel says. “I don’t ever want anything like that to happen.”