The Dancing with the Stars contestant also says he's never worked so hard physically before

By Tim Nudd and Michele Stueven
Updated September 26, 2011 10:25 AM
Credit: Craig Sjodin/ABC

Chaz Bono has heard it all from the naysayers. But it’s the fans, he says, who have the most impact on him.

“My critics don’t make me work harder. The support from the fans in response to the critics has really made me work harder,” the Dancing with the Stars contestant, 42, tells PEOPLE.

And work hard, he has indeed – in fact, he’s exhausted.

“I’ve never worked this hard physically before,” he says. His dance partner, Lacey Schwimmer, tells him: “I remember when you first came in, you were like, ‘Oh, I work out. I go on the elliptical. I lift weights.’ And then you’re sitting here with a towel over your head, like panting.”

Bono says “everybody’s a threat” on the show at this point, and he just wants to continue showing what he can do.

“It’s not a matter of proving anything,” he says. “I just want to stick around, cause I’m having such a good time.”