What Queen Elizabeth 'Really' Thinks About Prince Philip's Driving

"He always drove really fast, and she hated it," says veteran royals author Ingrid Seward

Queen Elizabeth and Prince Philip have put plenty of mileage on their relationship during the course of their 71-year marriage — and there have been a few twists and turns along the way.

Even from the high-speed early days of their romance, there was a notable bump in the road: Philip, a daring Naval officer during WWII, likes fast cars.

“He bought a brand new sports car in August 1947, three months before they got married,” veteran royals author Ingrid Seward, whose book My Husband and I examines their enduring marriage.

Prince Philip and Queen Elizabeth in 1960. Daily Mail/REX/Shutterstock

“He always drove really fast, and she hated it, and he would say, ‘If you don’t like it, you can get out.’ ”

In a rare letter Elizabeth wrote to author Betty Shew that was auctioned with royal memorabilia in 2016, she wrote, “Philip enjoys driving and does it fast! He has his own tiny M.G. which he is very proud of he has taken me about in it, once up to London, which was great fun, only it was like sitting on the road, and the wheels are almost as high as one’s head.”

In the wake of the January 17 car crash that saw Philip walking away uninjured from his overturned his SUV, what would his wife say now?

Peter MacDiarmid/REX/Shutterstock

“I’m sure she’d be telling him now to stop,” Seward says in this week’s issue of PEOPLE. “He will listen — but whether he will take any action, I don’t know.”

Philip flipped his Land Rover following a collision with a Kia driven by a 28-year-old woman with a 9-month-old baby in the backseat. The driver suffered cuts to her knee, while a 46-year-old woman sitting in the passenger seat broke her wrist. The baby was uninjured, local police in Norfolk said Friday.

• Can’t get enough of PEOPLE’s Royals coverage? Sign up for our newsletterto get the latest updates on Kate Middleton, Meghan Markle and more!

“People have played the accident down a bit – this could have been a quadruple fatal accident,” former royal protection officer Ken Wharfe tells PEOPLE. “Had his car stayed upright, surely the driver in the Kia would have died.”