Princess Diana's friend and personal designer Paul Costelloe says Prince Harry would have had the full support of his mother in his decision to step down as a senior royal

Princess Diana would have supported Prince Harry in his decision to step back from royal life, says her fashion designer friend.

“She wouldn’t be upset about Harry stepping back,” Diana’s former designer Paul Costelloe tells PEOPLE backstage at his London Fashion Week show on Monday. “She would say ‘do what you need to do,’ definitely, she was not a royalist at all, she was a person of the people and she put her children first before anything else.”

Having worked with Diana for 15 years during the ’80s and ’90s, Costelloe, 73, has fond memories of his time dressing the fashionable royal for numerous events — from the school run to royal tours in India and Australia.

“She was lovely to dress and she was a very honest, sincere person. She was family orientated and she had a sense of humor – she liked to smile,” he says.

Often fitting her at her Kensington Palace home (where her son Prince William now lives with his wife Kate Middleton and their three children), the designer says: “I remember meeting William and Harry — they were so cute. Harry was the little troublemaker and William was always well behaved.”

Diana And Harry Majorca
Princess Diana and Prince Harry
| Credit: Tim Graham/Getty
Harry Sticks Out Tongue
Credit: Tim Graham/Getty

The Irish-born designer first met his royal client back in 1982 after her lady-in-waiting stopped by his Windsor shop and he was asked to visit her at Kensington Palace just before a royal tour. “It was for a trip to India, I think, so we lengthened some sleeves and that was the beginning of it – it was a really lovely experience, she was so humble, there was no bowing and no ‘yes ma’am,’ ‘no ma’am.’ ”

Among other royal engagements, Costelloe dressed her for a 1993 tour of India where a skirt of his caused controversy when it appeared to be see-through. “She was in India and the sunshine shone through it, but we never got any comments or complaints from her.”

Paul Costelloe
Credit: Paul Costelloe

During a tour of Australia in 1988 he remembers fondly a yellow and white dress of his that she wore. “She was surrounded by these amazing looking swimmers on Bondi Beach – she was blonde, tall and she had that lovely olive English skin.”

Diana Down Under

As for her style at the time, Costelloe says she was the perfect ambassador for his growing brand, which he founded back in Dublin in 1979 after training at the Chamber Syndicale de la Haute Couture in Paris and working in Milan and New York.

“She had great shoulders for tailoring so she could carry off any jacket. She loved aqua greens, yellows and pinks – strong colors. I think she liked fashion and she had very strong points of view on what she wanted to wear,” he says.

Showcasing his Autumn/Winter 2020 collection at the Waldorf Hotel in London’s Aldwych, Costelloe, now a grandfather, says he will never forget hearing the news of Diana’s tragic death. By that point the two were no longer working together.

Diana Canada

“She got involved with Elton and Versace and I reluctantly took a bow,” he explains, but the news, like many, hit him hard. “I remember when I heard of the crash at 3 a.m. or 4 a.m. in the morning, I just stayed up all night and cried.”

Can’t get enough of PEOPLE‘s Royals coverage? Sign up for our free Royals newsletter to get the latest updates on Kate Middleton, Meghan Markle and more!

In recent years, Costelloe has dressed Princess Anne and her daughter Zara Tindall, although he is somewhat embarrassed to say Zara’s look was not altogether a success.

“I put Zara in a yellow dress for Cheltenham and somehow it made her look pregnant, so I can publicly apologize to her now. I still make mistakes!”