The new panelist tells PEOPLE, "We want to go back to the roots of it being funny"
Credit: Michael Tran/FilmMagic

Melissa Rivers is stepping into some familiar shoes – her mother’s!

The Fashion Police executive producer will be joining Giuliana Rancic and Brad Goreski when the E! show returns this summer, but the panelist position wasn’t one that Rivers initially set out for.

After her mother’s death, Rivers, 47, told PEOPLE she was unsure of who she was without Joan working by her side. As the remaining half of the Joan and Melissa “comedy act,” Rivers told PEOPLE she was suddenly left as a “straight man” and asking herself, “Now what?”

Unlike Joan – who was very vocal on Fashion Police before her unexpected death last September – Rivers admitted she needed to find her voice because everything was different as a solo act.

“I’m a different person and I’m in a different place,” she said.

Rivers explained, “As a performer it’s terrifying, and as a performer you have to give yourself time to sort it all out.”

Now that the late comedian’s daughter is joining the panel, Rivers is hoping to continue her mother’s legacy on the show.

“We want to go back to the roots of it being funny,” Rivers, 47, told PEOPLE at an event celebrating what would have been Joan’s 82nd birthday earlier this month.

The E! show underwent major cast shakeups in the last year, including the short-lived tenure of exit of Joan’s friend Kathy Griffin and Rancic’s controversial jab at singer/actress Zendaya‘s dreadlocks at the 2015 Oscars. Veteran panelist Kelly Osbourne also left the show earlier this year following Rancic’s comment.

“Just like a family, when the matriarch dies, the sisters started fighting, someone tried to marry in, not a great match – live and learn,” Rivers told Today‘s Hoda Kotb of the panel’s many changes.

Rivers revealed to AARP The Magazine that preserving her mother’s legacy adds “a lot of pressure.”

“I don’t want to blow it,” she said.

Fashion Police returns Aug. 31 on E!