The newly public couple seem "totally natural" together at Drake's afterparty

Just days after Sunday night’s MTV VMAs, during which Drake professed his love for Rihanna, the twosome showed off their mutual affection Tuesday night at E11EVEN in Miami.

The 24/7 downtown Miami club played host to Drake’s afterparty following his sold out concert at American Airlines Arena, and Rihanna was right by his side from beginning to end.

“She was in the VIP area the entire time mouthing all the words to every song,” an onlooker tells PEOPLE of the show. Another source adds the duo arrived in the same car and walked in separately, but then left together.

Drake performed five songs for the exuberant crowd, including “Energy,” “Pop Style” and “Jumpman,” before relaxing at a table with Rihanna, where the “Too Good” collaborators “danced, kissed, and acted really coupley,” the source tells PEOPLE.

Rihanna, who wore an oversized white shirt, seemed “totally comfortable and natural” with her beau, “laughing and smiling, and just having a good time,” the source said. The lovebirds stayed at the popular club – a favorite of Miley Cyrus and Leonardo DiCaprio until 4 a.m., partying with friends including Flo Rida, Future, Lil’ Wayne, Bow Wow and DJ Irie, who deejayed the event. “They were standing next to each other all night. He was whispering in her ear and she was rubbing his neck,” the onlooker tells PEOPLE.

RELATED VIDEO: Watch Drake Present Rihanna with the Vanguard Award

This isn’t the first time the singers have enjoyed romance in the Magic City. Earlier this year, Drake was on hand to support Rihanna at her own Miami concert, and the duo got close at that afterparty as well.

It seems Miami being in love brings out Drake’s generosity, as during his set, he took a moment to tell the crowd, “We have a whiskey called Virginia Black here … I’m going to send everyone a bottle!” He added, “It’s always an honor to perform in Miami, always an honor to perform period.”

For those die-hard fans looking to catch the couple in action, Drake will be back for his second show Wednesday – hopefully with Rihanna in tow.

With reporting by JANINE RUBENSTEIN