William recalled remembering "seeing the pigeons and squirrels" at the garden
Britain Princess Diana
Credit: Kirsty Wigglesworth/AP

Prince William and Prince Harry reminisced on Wednesday about their childhood visits to the gardens outside the Kensington Palace home they shared with their mother, Princess Diana.

William, 35, recalled that he remembered “seeing the pigeons and squirrels,” adding, “We used to come here a lot.”

Britain Princess Diana

The princes, along with Princess Kate, visited the Sunken Garden on the Kensington Palace grounds, which has been redesigned as an all-white garden in honor of Diana’s favorite color.

Noting that he and Harry used to feed the fish in the garden’s pond, William said, “there never used to be this many in here” — before being told some of the koi carp are so old that they are the same ones from his youth.

Britain Princess Diana
Credit: Kirsty Wigglesworth/AP

William, Kate and Harry were given a tour of the plants by Sean Harkin, head gardener, and Graham Dillamore, who worked on the Kensington Palace grounds when Diana lived there.

“It does look really different,” Kate, in a floral-patterned Prada dress and L.K. Bennett heels, observed. “I love the semi-wildness.”

Britain Princess Diana
Credit: Kirsty Wigglesworth/AP

Prince Harry told staff that when the summer opening of the White Garden wraps, he would like to keep some of the plants and flowers alive, transporting them to his own Kensington Palace plot to tend them. Flowers include the William and Catherine roses, Forget-Me-Nots, and white roses chosen particularly for their perfume.

“[William and Harry] were very interested in hearing about the flowers,” says Harkin. “Prince Harry is quite green-fingered himself and asked lots of questions about what we were growing. We were talking about the importance of having plants rather than paving over.”

Britain Princess Diana
Credit: Kirsty Wigglesworth/AP

William, Kate and Harry also met with representatives from eight charities closest to Diana’s heart, including the English National Ballet.

Director Tamara Rojo told them many popular productions were only staged because of the fundraising Princess Diana had done in her lifetime, adding: “That’s something we celebrate.”

William said: “We’ve got plenty of pictures. I remember her showing me ballet shoes she had been given [by ENB] and she was so proud of them. She loved dancing, she was a fantastic dancer.

“We’ve been going through her music collection recently and there’s some quite eclectic stuff in there. She was elated by the skill.”

Kate noted that 2-year-old Princess Charlotte is now learning to dance, adding: “She absolutely loves it.”

Meeting representatives from Great Ormond Street Hospital, William recalled how his mother stayed by his side there when he was admitted to the hospital at age 9 after a golfing accident left him requiring 24 stitches.

“I got looked after extremely well,” he said.

Meanwhile, fans who came out on the rainy eve of the 20th anniversary of Diana’s death were given the opportunity to pay tribute — and at least one had an extra-close encounter with Harry.

Eight-year-old Gracie Oxby was holding a bouquet of flowers outside the golden gates of Kensington Palace when the 32-year-old approached.

“Harry came straight over to Gracie and she said, ‘Could you put the flowers down for me?’ ” says her mom, Rhian. “He said, ‘By all means. Where do you want them?’ He was very friendly and funny and very nice.”

Adds Gracie: “It meant a lot to meet a Prince. It was really exciting.”

20th Anniversary of the death of Princess Diana - 30 Aug 2017
Credit: Chris Radburn/REX/Shutterstock

Another fan, Kurt Deneen — an American who now lives in London — came to pay tribute and reflect on Diana’s enduring humanitarian legacy.

“When I was younger, Diana was the first person of celebrity status who made those with AIDS human,” recalls Deneen. “Before I cam out as gay, I would see her on the news meeting all these people who would scare me because I was so much younger then. I was just this young guy in Chicago feeling nervous and watching her. She was amazing.

“I still have colleagues back in the States that I am still sending newspaper clippings about Diana. They are really struck by it. A friend messaged me and said, ‘She was my princess growing up and I really wanted to be her.’ My friend grew up in a small town in Nebraska, which shows you just how far Diana reached into people’s lives.

“That’s part of her legacy. I also think it was because she was so human. She had her flaws like everybody else and we can all relate to that.”

• With reporting by PHIL BOUCHER

For full PEOPLE coverage of the 20th anniversary of Princess Diana’s death:•PEOPLE’s special edition Diana: Her Life and Legacy is available now

Princess Diana: Behind the Headlines is streaming on the new People/Entertainment Weekly Network (PEN). Go to PEOPLE.com/PEN, or download the PEN app on your favorite mobile or connected TV device