We all remember Elf for its holly jolly hilarity, but there's one moment that stands out to Ferrell for another reason
ELF, Will Ferrell, 2003, (c) New Line/courtesy Everett Collection
Credit: Everett

We all remember Elf for its holly jolly hilarity, but there’s one moment that stands out to star Will Ferrell for another reason.

In an interview with Rotten Tomatoes, Ferrell and director Jon Favreau opened up about some of the holiday film’s most memorable moments, including one that actually brought the 51-year-old comedian to tears during the 2003 Los Angeles premiere.

“I knew it was working at that moment where Buddy is in the back of the sleigh and everyone’s singing in Central Park and there’s enough Christmas spirit to get it lifted off, and he’s waving goodbye,” Ferrell shared. “I’m like, Oh I can’t let everyone see m cry here at my own movie.’ ”

The actor went on to explain that the movie, which was filmed in New York City, was released just a few years after 9/11. Referencing the heartwarming scene, Ferrell shared, “when it came out what we really wanted [was] a movie like that.”

“It was a really challenging time and it was nice to bring a nice breath of innocence to the world and especially to the city at that time. I’m really proud of it,” he added. “If it’s ever in a theater or playing on television, I love to check in on it. And I can tell through social media that it’s something that people have made a tradition of….and that really makes me happy and it’s the best part of the job.”

Amy Sedaris, James Caan, Will Ferrell
| Credit: Alan Markfield/New Line Prods./Kobal/REX/Shutterstock

The pair also reminisced about the standout scene where Buddy the Elf, newly employed as one of Santa’s helpers at a department store, freaks out after learning that Ol’ Saint Nick himself was on the way for a visit.

“I remember the scene in Gimbel’s where Faizon Love makes the announcement that Santa is coming, and he just screams ‘Santa!’” Favreau recalled. “[Will] just loves to commit. He really knows where the laugh is in the scene. And then the reaction of [Faizon] being the manager, looking, thinking his employee is screaming in his face, is probably one of my favorite moments of the movie.”

In a testament to his own comedic chops, Ferrell added that “all of that, ‘Santa, I know him,’ all of that playing around we did, that was all improvised.”

RELATED VIDEO: Will Ferrell Reveals The One Sequel He’ll Never Do

Of course, Favreau and Ferrell aren’t the only people involved in making the Christmas classic that look back fondly on the film.

“It was a nice and pleasant set to be on,” Amy Sedaris, who played secretary Deb told PEOPLE earlier this year. “My time on set was just a blast.”

“Ferrell was always keeping the ball up in the air, always laughing,” she continued. “We just sat around talking a lot. Everybody was always in a good mood; the energy was up.”

Sedaris, 57, went on to share that despite how much fun the actors had, nobody had any way of knowing how much Elf would resonate with audiences.

“You never know when you start,” Sedaris said. “You do a scene here, you do a scene there, and I can never see the big picture. I don’t really think about the final product at all. If I’m having fun and everyone’s having fun, the chances are it’s going to turn out okay.”