"I'm happy for her new life, but of course we were sad over here and we miss her," he says

Zoila Chavez has spent 18 years working for house-flipper extraordinaire and design guru Jeff Lewis — putting up with his hot temper and goofy antics on Bravo’s Flipping Out.

But after nearly two decades as Lewis’ housekeeper, the 68-year-old reality star has finally retired.

“It’s been kind of rough,” Lewis told PEOPLE in a phone interview on Thursday. “I’m happy for her new life, but of course we were sad over here and we miss her.”

Jeff Lewis - Home
Credit: Kelsey McNeal/Bravo/Getty

If Chavez’s absence is felt even more right now, it’s because there’s a lot going on in the Lewis household. Not only are the reality star’s house-flipping and design businesses thriving, but he and his partner of eight years Gage Edward are also busy raising their 9-month old daughter, Monroe Christine.

Lewis explains: “Zoila retired about three months ago. She stayed the first six months we had the baby, which is amazing. You know, she’s wanted to retire for two years now, and she’s been with me for 18 years. I just begged her, begged her, begged her — ‘Please stay. I need help with the baby. I need help with this transition.’ And she stayed as long as she could.”

“Physically, after doing that kind of work for all of these years, it really takes its toll,” Lewis continued. “She’s 68 years old. She looks 10 years younger, because I gave her all that plastic surgery. But she’s tired and she’s ready to go.”

He and Chavez have certainly had their ups and downs over Flipping Out‘s 10 seasons. Not afraid to push back from time to time, Chavez’s sassy and stubborn attitude often landed her on the receiving end of Lewis’ famously short temper. In a season 9 episode, she was almost fired after walking out and refusing to return right as Lewis was preparing to leave for an important business trip.

“You better think real carefully about what you’re doing right now because there’ll be no going back,” Lewis warned her in a tense voicemail. “And I promise you—there’ll be no going back.”

Of course, the duo made up and Chavez is back in the Lewis household for season 10, working part-time two days a week. Viewers will get to see her road to retirement throughout the milestone season.

So what exactly has Chavez been doing since leaving? “Most of her family is in Nicaragua, so the first thing she did when she retired is she went to Nicaragua for five, six weeks,” Lewis explained. “Now she’s back and we’re texting — she has a daughter who lives in Koreatown, so she’s staying with her.”

RELATED VIDEO: Jeff Lewis Introduces His New Intern: His Daughter Monroe!

And in case there was any doubt, Lewis was quick to stress that Chavez is not out of his life — though Flipping Out viewers might see a whole new life for Lewis this season.

“When the baby came, everything changed. Everything changed. Our home life, our business life — everything has been shaken up in a good way,” he said. “It forces you to kind of grow and assess what’s working in your life and what’s not working in your life. Also, it’s not about my life anymore, it’s about the baby’s life. All these decisions we make are about what’s good for the baby. And I’m used to making decisions about what’s good for Jeff. So it’s kind of a new life.”

Flipping Out‘s 10th season premieres Aug. 17 (9 p.m. ET) on Bravo.