Why Penn and Teller Did The Masked Singer: 'Judges Aren't Assholes'

This post contains spoilers from Wednesday's episode of The Masked Singer.

Just like magic, Masked Singer guest judge Nicole Byer guessed Penn Jillette and Teller as Wednesday's eliminated contestant Hydra.

"I thought, 'Well, about f---ing time.' We went much longer than we thought we were going to," Jillette, 67, tells PEOPLE.

The magician says he and his partner suspected they'd be the first ones eliminated from the Fox competition show, but they actually sang three songs across two episodes since Hydra faced off against Armadillo in Wednesday's duel.

"I wanted to pick songs that didn't give away who we were, so we stayed away from Vegas and we stayed away from our youth and we stayed away from contemporary," Jillette says of performing Train's "Hey, Soul Sister" and ZZ Top's "Sharp Dressed Man."

He adds, "And then we didn't get to do it, but we worked up 'Karma Chameleon.' "

Teller, 74, doesn't typically talk during his magic act with Jillette — which they do together at the Penn & Teller Theater in Las Vegas and on their CW show Fool Us — but he spoke and sang as Hydra on The Masked Singer. The duo even had the judges thinking there might be three singers under that three-headed dragon outfit.

"We knew that if we were two people, we would be busted right away, and we also knew we couldn't get into one person costume," Jillette explains. "So three was a good number."

Jillette recalls sharing the small space with his longtime partner, his reaction to being guessed as various Muppets and how doing the show inspired new magic tricks.

What made you want to go on The Masked Singer as a duo?
PENN JILLETTE: We do most everything as a duo. It's all I really know. We are the longest-running headliners in Vegas history. We have to put a quarter of a million people into the Penn & Teller Theater every year to keep doing that, we do an awful lot of television. If we are asked, we say, "yes." That being said, we were so thrilled to do The Masked Singer. We have so many shows on TV that are built on snottiness and cruelty, and I don't think that our culture needs that right now. The Masked Singer, the judges aren't assholes. They're sweet and they're kind, and everything is done in a pleasant way.

Why did you and Teller pick the Hydra costume?
The costume designers came up with this wonderful Hydra idea: the three-headed dog, guarding the gates of hell, which seemed like a really good choice for us. Also, being magicians, we have spent a lot of our lives forced into little tiny places together. So if you're going to put anybody in a horribly cramped costume, we're the ones to choose. We were jammed in there together and we've been jammed together for 47 years. So, we did okay.

Did you consciously decide to try to make it sound like three people were under the costume?
My voice is distinctive. I worked at carnivals for a long time. I have a carney trash voice. So Teller had to really do the other two characters. He did a wonderful job. One of the characters we called Rex because he was trying to do Rex Harrison, the old, old, old singer. And the other character was just Teller doing his own voice. Teller was the one doing the real vocal showing off. He had to do those two characters and switch off, sometimes in mid-line. And I thought he did a beautiful job.

How did being on stage as Hydra compare to doing your magic act?
There's really very little comparison. When we are doing our magic show, it's entirely our show. It's just the two of us. And we are not in a little box. At least for most of the show, we're out there where people can see us. On top of that, we don't sing. That's one of the things that made this show so fun, is that Teller and I don't get to sing, and Teller and I both love music.

RELATED VIDEO: Christie Brinkley Shares Why Doing The Masked Singer 'Was Life-Changing for the Whole Family'

What role does music play in your life?
I'm an upright bass player. I play bebop jazz and every night before the show I played for an hour as the audience is coming in and I performed as a bass player, CBGB with punk bands, with jazz bands. I do an awful lot of music, so it was wonderful to be able to be on a show that celebrates music.

Were you offended by getting guessed as puppet pairs Bert and Ernie and Statler and Waldorf?
No. What could be more flattering than that? I don't think there's anything more flattering than that.

Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human interest stories.

You and Teller have been performing together for almost 50 years. Any plans of slowing down?
No plans yet, no. We have our 10th season of Penn & Teller: Fool Us, and we're very excited about that. And we are always running at the Penn & Teller Theater in Las Vegas.

The Masked Singer airs Wednesdays (8 p.m. ET) on Fox.

Penn Jillette Says He and Teller Chose to Do 'The Masked Singer' Because 'the Judges Aren't Assholes'