"They think they're thrilled, but in a more traditional high-end gated community, there would be another 15 kids from their school," he tells PEOPLE of his plans to move


Penn Jillette built his dream home in the middle of the Las Vegas desert 20 years ago, long before he had children .

“I built the ultimate bachelor pad,” the magician, 60, exclusively tells PEOPLE of his 12,000-sq.-ft. home, complete with a Scrabble bathroom, hidden rooms, fire poles, and an indoor theater.

Jillette later decided to have children with his wife Emily, 49, and soon realized that his colorful, quirky house might not be the best place to raise kids.

“In 2005, I started a family, so we began to change the house,” explains the talkative half of Penn & Teller, who will making a limited return to Broadway this summer.

“When we talked to people about making the house child-friendly, it was impossible, so we built another house and added onto it.”

Penn Jillette wife kids las vegas home

The addition includes a kitchen and living space, which he deems the children’s area, where the kids — Zolten Penn, 9 next month, and Moxie CrimeFighter, 10 in June — are free to play and where the family usually eats dinner.

However, there are certain things about his house — which he nicknamed “The Slammer” — that he can’t change, including how isolated it is.

“Now the very pluses of The Slammer become minuses — they’re not close to other children, they’re not close to their families, they’re a long way from schools — all the stuff I really loved about it,” says Jillette. “We may be actually getting rid of this home and move into a place much more traditional.”

The Wizard Wars judge hopes to move into a gated community after returning from Broadway this summer.

“The children don’t know they’re unhappy here,” he says. “They think they’re thrilled, but in a more traditional high-end gated community, there would be another 15 kids from their school that they could walk to. Those planned, gated communities are for families, and they work. This is for a crazy person living alone in the desert.”

For more of our interview with Jillette, and to get a peek inside his Las Vegas house, pick up the latest issue of PEOPLE, on newsstands now.

— Gabrielle Olya