“I thought it would be fun for them to not just visit the set for half an hour and see me and say hello, but actually see what I do for a living,” Harris, 44, told PEOPLE during Thursday’s season 2 premiere in New York City.
“And they seemed to have a good time,” adds the star, who plays the sinister master of disguise Count Olaf in the darkly comedic Netflix hit.
Want all the latest pregnancy and birth announcements, plus celebrity mom blogs? Click here to get those and more in the PEOPLE Babies newsletter.
The actor, however, admits he was a “worried parent” the whole time his children were on set. “I was very distracted by it all, like wanting to make sure they were okay,” he explains. “I wanted to make sure their experience was okay and that they didn’t get tired.”
While son Gideon and daughter Harper — whom Harris shares with husband David Burtka — enjoyed the acting experience, the Golden Globe nominee doesn’t think his kids want to follow his career path. “No,” Harris says of whether there’s a possibility his children go into acting. “But they’re both extroverts in their own way.”
And the twins aren’t fazed by their dad being on TV. As the former How I Met Your Mother star reveals, “They’re pretty nonplussed by all of that because I’ve been doing it since they were born.”
RELATED VIDEO: Neil Patrick Harris on His Magical New Children’s Book: “I Wanted to Honor the Differences People Have”
The family has been enjoying watching his NBC game show Genius Junior, in which teams of kids duke it out in a battle of the brains.
“[My family] were visiting in Vancouver and before bed, before we started reading books, David mentioned we haven’t watched Genius Junior yet, so we turned it on,” says Harris, who filmed A Series of Unfortunate Events in the Canadian city.
He explains, “Genius Junior is really fun for them to watch because math and spelling is new to them, and they get to watch kids only a year or two older than them excelling so remarkably in it. I love watching them be excited by that.”
All episodes of A Series of Unfortunate Events season 2 are now available on Netflix.