"Everything I do, it's like, 'Ooh, Sexiest Man Alive. He uses a straw. Oh my God! He opened that can. Sexiest Man. So sexy,' " Michael B. Jordan tells PEOPLE of his friends teasing him

Not even Michael B. Jordan is immune to a good-natured ribbing by his pals.

In a recent interview with PEOPLE, the Without Remorse star playfully lamented the fact that those close to him "consistently" rag on him for his reigning Sexiest Man Alive title, especially "when they're trying to be obnoxious and bother me."

"Everything I do, it's like, 'Ooh, Sexiest Man Alive. He uses a straw. Oh my God! He opened that can. Sexiest Man. So sexy.' The harassment doesn't stop," says Jordan, 34.

But he insists he's a good sport about it, taking his loved ones' "fun" comments in stride because he knows they're from a good place: "Every group that I'm in, all my boys, they all take their shots consistently."

Michael B. Jordan
Michael B. Jordan
| Credit: The Tonight Show/YouTube

"But now I know George Clooney has been Sexiest Man Alive a few times, so listen, Clooney, I'm coming for you. Let's try to run it!" Jordan jokes.

Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human-interest stories.

Michael B. Jordan- SMA SWEATERS
Michael B. Jordan
| Credit: Getty

During a Thursday appearance on The Tonight Show, the Black Panther star called his PEOPLE title "a gift and a curse," telling host Jimmy Fallon, "People don't tell you that you instantly have a target on your back."

"All my boys, in every group chat ... constantly roasting me," Jordan said. "Everything I do, it's like, 'Oh, the Sexiest Man Alive is taking off his jacket,' or 'Look at the way he wears a shirt.' 'He eats like the Sexiest Man Alive.' "

"Enough is enough," he continued. "But on the other hand, my mom and my aunts, they have [the magazine] kinda vacuum-sealed, Saran-wrapped, up on the mantel."

"They bought every issue off the newsstand," said Fallon, 46, to which Jordan concurred, "Every one."

Michael B. Jordan SMA
Credit: The Tonight Show Starring Jimmy Fallon

As for who was the most excited for his SMA title, Jordan previously told to PEOPLE the women in his family "are definitely proud of this one."

"When my grandmother was alive, it was something that she collected, and then my mom naturally reads it a lot and my aunts as well," he said. "This is one that they're definitely going to have a special place for."

Jordan will next appear in the Denzel Washington-directed A Journal for Jordan, as well as star in and make his big-screen directorial debut with Creed III, set for a Thanksgiving 2022 release.

Without Remorse is available to stream now on Amazon Prime Video.