"I'm just not a pretty little boy, am I?" says the Gossip Girl costar

Credit: Albert Ferreira/Startraks

Whatever you do, don’t call Ed Westwick pretty.

Asked about Chace Crawford‘s comment in this week‘s PEOPLE about his “Velcro patch of chest-hair,” Westwick, 21, didn’t bat an eye, though he did deliver a snarky retort of his own for his friend and fellow Gossip Girl costar: “I’m just not a pretty little boy, now am I?”

No doubt the hordes of young women gathered at Bloomingdale’s K-Swiss Classic Remastered dance party Thursday in Manhattan would disagree. Wherever Westwick, 21, goes, fans automatically congregate.

Gossip, too, has a way of attaching itself to the star – be it false rumors of a relationship with Crawford or about his expanding waistline.

Westwick, however, isn’t about to let that any of that get him down. “I’m over everything that’s been in the past, and everything right now is fantastic, man,” he says.

Speaking of fantastic, just how does Westwick manage to look so good? “I’ve never been a real gym guy,” he admits. “I always got my fitness through playing soccer.”