Brazilian police allege a confrontation, not a robbery, ensued when Lochte was "very angry because he was intoxicated"
Credit: Michael Sohn/AP

Promising “the real story,” Rio’s Chief of Civil Police said Thursday that Ryan Lochte and three other U.S. swimmers allegedly lied about being robbed at gunpoint early Sunday during the Olympic Games.

“There was no robbery the way it was reported by the athletes,” Chief Fernando Veloso said in a news conference, alleging the swimmers lied to police. “The athletes were not victims of the criminal actions they claim they were.”

Police said the swimmers were confronted Sunday by security guards carrying guns when they stopped at a gas station on their way home from a party.

“This is the real story,” Veloso alleged. “When they left the party they stopped at the gas station. One or all of them started to vandalize the bathroom inside of the gas station, damaging some mirrors and things that were inside.”

Security guards then allegedly confronted the four swimmers and the manager of the gas station allegedly asked the men to pay for the damage, Veloso said.

He alleged a confrontation ensued during which Lochte was “very angry because he was intoxicated.”

Veloso added that the investigation was ongoing and he could not say that if Lochte was the only individual allegedly responsible for the confrontation. But Veloso said witnesses at the scene described the 32-year-old swimmer as “more physically outwardly angry or nervous” at the time.

One of the guards then drew his firearm to “control the situation,” Veloso said. He said he not think this constituted excessive force.

“We have two security guards, we have four very strong men with inappropriate behavior breaking things and showing that they were ready to bring this violence against guards or people,” Veloso alleged. “So the guard is probably justified in this case.”

For more of PEOPLE’s Olympic coverage, pick up our collector’s edition, The Best of the Games, on sale now.

Road to Rio: Ryan Lochte’s Insane Olympic Diet

Someone at the gas station called the police, but by the time the police arrived, the swimmers were gone, Veloso said.

Witnesses, including one who acted as a translator, said the swimmers allegedly gave the manager money before leaving, he said.

Police said this version of events has been confirmed by the security guards, the taxi driver, CCTV footage from the gas station and one of the two teammates, Gunnar Bentz or Jack Conger, who were pulled off a plane headed back to America late Wednesday.

“One of the athletes gave us a statement as a witness saying that there was no robbery against the athletes,” Veloso said.

Veloso said Brazilian police first learned about Lochte’s robbery claim on social media. Lochte reportedly first told his mom about the incident, and when she repeated the story to reporters it made international headlines, according to Associated Press. The controversy has pitted Brazil’s reputation against that of the four U.S. swimmers.

“There were a lot of contradictions from the beginning,” Veloso said Thursday, adding that Lochte’s account was first refuted by a taxi driver who asked not to be identified.

Veloso could not say if any of the athletes involved would be charged as the investigation is ongoing. However, “in theory one or all of them might be charged for false communication of a crime and for damaging private assets at the gas station,” he said.

The FBI is working with Brazilian officials to question Lochte, who returned to the U.S. early Wednesday, about his account of the events, Veloso said.

He added that teammate James Feigen had not been interviewed at the time of the news conference. Bentz and Conger were in process of providing statements to police, he said.

An attorney for Lochte, Jeff Ostrow, told CNN that the all four swimmers’ stories were “95 percent consistent.” (A Lochte source told PEOPLE there is missing security footage from the gas station which “backs up” the robbery claim.)

“He’s the victim. He didn’t commit a crime,” Ostrow said. “Same with the other guys.”

But Veloso said that at the very least, Lochte owed an apology to the Olympic host city.

“They are people, they can make mistakes because probably they were under the influence of alcohol,” Veloso said. “They need to apologize not to the police but to the city of Rio that had its name tainted by a very untruthful story.”