The announcements came just hours after Ryan Lochte guaranteed he and Michael Phelps would compete in the next Olympics

Michael Phelps won’t be competing in the next Olympics. For real this time.

The 31-year-old Olympic swimmer, the most decorated Olympian of all time – announced that his 4×100-meter medley relay on Saturday will be the last race of his Olympic career, USA Today reports.

The news came after Phelps’ teammate Ryan Lochte guaranteed that both he and Phelps would return for the 2020 Summer Olympics in Tokyo.

“I will just clarify that Ryan doesn’t know what he is talking about,” Phelps said, according to USA Today. “I am not coming back in four years.”

The 2016 Summer Olympics has been a historic one for Phelps. The new father won his 22nd gold medal (and 26th overall) on Thursday and captured his 13th individual event win, becoming the athlete with the most individual event wins since 152 B.C.

“I have two laps left in my career so I am happy and I’m excited and looking forward to tomorrow,” he said on Friday.

“I saw [what Lochte said] and I was like, ‘Thanks, Ryan, throw me back out there for another four years.’ If he wants to come back it will be great.”

Phelps had hinted that this Olympics would be his last ahead of the Games. The swimmer had previously said that the 2012 London Olympics would be his last, but he came out of retirement in 2014 and went on to easily secure a spot on Team U.S.A. once again.

Lochte told PEOPLE that he remembered Phelps’ first retirement, hence his comments that he would be back.

“He said that in 2012,” Lochte told PEOPLE on Saturday in Rio. “I remember that. And I was the only one who said he was going to come back. So I still guarantee that he’ll come back again. You know, if I’m coming back, I want him to come back, too. It’s really good for the sport. ”

Phelps began his Olympic journey when he was just 15 years old and became the youngest male to make a U.S. Olympic swim team in 68 years.

For more of PEOPLE’s Olympic coverage, pick up our collector’s edition, The Best of the Games, on sale now.

The swimmer was expected to secure his 23rd gold medal on Friday in the men’s 100-meter. But Singapore’s Joseph Schooling surprisingly bested the champion, leaving Phelps and two others in a tie for the silver.

“Watching Joe and what he’s done in the last two years – he’s been swimming great,” Phelps said of the gold medalist after the race. “My hat’s off to him!”

Reporting by STEVE HELLING