Prince Andrew Stripped of Honorary Title as Calls Continue for Him to Relinquish Duke of York Title

York councilors voted unanimously to strip Prince Andrew of his Freedom of the City honor

Prince Andrew. Photo: Chris Jackson/WPA Pool/Getty Images

Prince Andrew has been stripped of another title.

At a meeting at York Racecourse on Wednesday night, councilors voted unanimously to strip Andrew of his Freedom of the City honor, according to ITV. Queen Elizabeth's third child, known as the Duke of York, was given the ambassadorial title in a 1987 ceremony shortly after his marriage to ex-wife Sarah Ferguson.

Darryl Smalley, City of York Council's Executive Member for Culture, said the Freedom of York was "the highest honor" the city could bestow.

"The honor is held by many notable and accomplished people who carry it with pride and responsibly. Having been stripped of his military roles and royal patronages by the Queen, we believe that it is right to remove all links that Prince Andrew still has with our great city," he said, according to ITV. "I was pleased to see councilors of all parties support this motion and make it clear that it is no longer appropriate for Prince Andrew to represent York and its residents. The removal of this honorary title sends the right message that we as a city stand with victims of abuse."

Queen Elizabeth and Prince Andrew. Max Mumby/Indigo/Getty Images

The campaign to remove the honorary title came in the wake of Prince Andrew's scandal surrounding his ties to convicted sex offender Jeffrey Epstein and Virginia Giuffre's sexual assault lawsuit against the royal.

In January, the Queen stripped Prince Andrew of his military titles and patronages amid the lawsuit, in which Giuffre alleged she was trafficked by Epstein and forced to have sex with Prince Andrew at the age of 17. Andrew has denied any wrongdoing. The decision to strip Andrew of his titles and affiliations was "widely discussed" within the senior ranks of the royal family, a royal source confirms to PEOPLE — likely meaning talks between Queen Elizabeth, Prince Charles and Prince William.

For more on Prince Andrew being stripped of another title, listen below to our daily podcast PEOPLE Every Day.

Prince Andrew, 62, has retained his title as the Duke of York and keep his place in the line of succession to the throne, which is currently ninth behind Prince Harry and Meghan Markle's two children. He will no longer use the style "His Royal Highness" in any official capacity.

Prince Andrew. STEVE PARSONS/POOL/AFP

However, York councilors say the "next logical step" would be for Prince Andrew to "do the right thing and relinquish his Duke of York title," a dukedom bestowed to him by the Queen on his wedding day. The Duke of York is the title traditionally held by the next sibling in the line of succession behind the Prince or Princess of Wales, the title currently held by Prince Charles.

Smalley said, "If he fails to do so, the government and Buckingham Palace must step in to remove his title to finally end Prince Andrew's connection to York."

Can't get enough of PEOPLE's Royals coverage? Sign up for our free Royals newsletter to get the latest updates on Kate Middleton, Meghan Markle and more!

Queen Elizabeth and Prince Andrew. RICHARD POHLE/POOL/AFP via Getty Images

On March 29, Queen Elizabeth arrived at the Service of Thanksgiving honoring her late husband Prince Philip on the arm of their disgraced son. Royal commentator Robert Jobson told PEOPLE there was uneasiness among senior members of the royal family about the Queen's decision to be escorted by Prince Andrew, "but she insisted."

"It shows she wholeheartedly loves and believes her son," Jobson, the author of Prince Philip's Century. "As she did when she made a statement about Camilla being Queen's Consort, many people will now accept the Queen's word and judgment."