Sofia Vergara Opens Up About Simon Cowell's AGT Absence: 'I Wish He Could've Been Here'

Howie Mandel says don't count Simon Cowell out of the whole season just yet

Photo: Tyler Golden/NBC

Sofia Vergara has been learning to go with the flow during her freshman season on America's Got Talent. Not only has the coronavirus pandemic set unprecedented production protocols, but new faces have been joining the judges' panel as Simon Cowell recovers from his bike accident.

"For me, everything is for the first time. This is my first time as a judge, so it's not like I can really compare to anything," Vergara, 48, tells PEOPLE.

"It's been great to have all those guest judges, but I do miss Simon. I wish he could've been here," she says. "For me, it is my first time and I feel comfortable when he is with us. Howie [Mandel] and Heidi [Klum] are amazing, but he's the boss!"

First, in the audition rounds, Vergara's Modern Family costar Eric Stonestreet stepped in for Klum, who fell ill on set in early spring. (Klum tested negative for the coronavirus.)

Trae Patton/NBC

Chris Haston/NBC

And for the live shows so far, Kelly Clarkson and Kenan Thompson have filled in for Simon Cowell, who broke his back at his Malibu, California, home and underwent surgery on Aug. 8. (Cowell's participation for the rest of season 15 has not been determined.)

Vergara's experience as a judge was further affected when the coronavirus pandemic halted production and ultimately altered how filming was done. "It was kind of a little bit upsetting at the beginning. I felt like I was robbed of the opportunity to experience and learn the whole thing," the actress previously told PEOPLE.

Fast forward to the present, Vergara praises the show's producers for adapting to the pandemic and pulling off impressive socially distanced episodes. "I thought we weren't going to be able to finish the season so I am happy and grateful to be a part of the show," she says after Tuesday's show.

America's Got Talent season 15. Trae Patton/NBC

In late 2019, NBC approached Vergara with the opportunity to join AGT's 15th season after season 14 judges Gabrielle Union and Julianne Hough exited their seats from the panel. And since the latest season premiered in late May, Vergara has navigated each modification while finding her voice as a judge.

"What I try to do is to be as honest as I can, to just say what the contestant makes me feel because I try to be the voice of people that are watching TV," she previously told PEOPLE about her approach to judging. "That's what I'm trying to do. I'm trying to have fun and be honest with the stuff that I like and that I don't like."

As for Cowell's recovery, the judges provided an update on the possibilities of his return.

"I wouldn't count him out entirely for the rest of the season," Mandel says.

"He's healing pretty fast a week after breaking your back in four different places. I think he's doing amazingly," Klum says. Speaking of Cowell's "good chance" of being back on the panel, the model adds: "Next week is maybe too fast."

America's Got Talent airs Tuesdays and Wednesdays (at 8 p.m. ET) on NBC.