"They really think it's funny," the actor jokes

Gregory Pace/BEImages

When Matt Damon looks in the mirror these days, he knows exactly who’s responsible for his changing appearance.

“I point out all of the gray and I say to my daughters, ‘You did this and you did this and you did this,'” the father of four joked to PEOPLE Tuesday during a dinner for Argo hosted by the Peggy Siegal Company at Porter House steakhouse in New York. “They really think it’s funny.”

Damon doesn’t mind it, though — even if the studios do. “They dyed the gray out for We Bought a Zoo, but I was like, ‘I like my gray hair,'” he said. “I’m proud of it.”

Now sporting a buzz cut for an upcoming role, Damon will have to weather the N.Y.C. winter clean-shaven. But “it’s okay,” he said. “I’ve got a beanie.” Tell us: Do you like Damon best blond, graying or bald?

–Janine Rayford