Harry said he feels "really let down" by his father, Princes Charles
Prince harry and Megan and Oprah
Prince Harry, Meghan Markle and Oprah Winfrey
| Credit: Joe Pugliese/Harpo Productions

Prince Harry is opening up about his relationships with family members following his royal exit.

During the Duke and Duchess of Sussex's interview that aired Sunday, Oprah Winfrey asked about Harry's current relationships with his dad Prince Charles and older brother Prince William.

For more on how the Royal Family is reacting to Prince Harry and Meghan Markle's interview with Oprah, listen below to the episode of PEOPLE Every Day.

When it comes to his father, Prince Harry said: "There's a lot to work through there. I feel really let down because he's been through something similar. He knows what pain is like. And Archie's his grandson."

He continued, "Of course, I will always love him, but there's a lot of hurt that's happened. And I will continue to make it one of my priorities to try and heal that relationship."

commonwealth day 2020
The royal family on Commonwealth Day in 2020
| Credit: PHIL HARRIS/POOL/AFP via Getty Images

Harry said his relationship with Prince William "is space at the moment" but he "loves William to bits — we've been through hell together."

"And you know, time heals all things, hopefully," he said.

Get the premiere issue of PEOPLE Royals for glamorous new photos and inside stories royals fans haven't seen or read elsewhere! Subscribe at peopleroyals.com/launch

Queen Elizabeth II, Meghan, Duchess of Sussex, Prince Harry, Duke of Sussex, Prince William Duke of Cambridge and Catherine, Duchess of Cambridge
Queen Elizabeth, Meghan Markle, Prince Harry, Prince William and Kate Middleton
| Credit: Paul Grover - WPA Pool/Getty Images

Can't get enough of PEOPLE's Royals coverage? Sign up for our free Royals newsletter to get the latest updates on Kate Middleton, Meghan Markle and more! 

Despite his exit, Prince Harry said that his bond with his grandmother Queen Elizabeth is strong.

"I have spoken more with my grandmother in the last year than I have done in many, many years," he said. "My grandmother and I have a really good relationship and a good understanding."

Harry added that they've had Zoom calls with the Queen so she can see her great-grandson, Archie, who turns 2 in May.