"They were super cute," an onlooker tells PEOPLE

By Melody Chiu Patrick Gomez
April 12, 2015 09:00 AM

Lovebirds at Coachella!

After PEOPLE exclusively confirmed their engagement nearly two weeks ago, Robert Pattinson and FKA twigs got cozy Saturday at the music festival.

With friends – including close pal Katy Perry – by his side, the Twilight star, 28, supported his new fiancée, 27, as she performed her set on stage just after 9:30 p.m.

The cute couple later hit up Jeremy Scott and Moschino’s late night party, where guests sipped on DeLeén Tequila cocktails, had bites by Eveleigh and checked out Polaris Slingshot motorcycles.

Walking into the party holding hands, Pattinson and twigs (who was not seen sporting an engagement ring) shared kisses while hanging out with their friends.

“They were super cute,” says a fellow guest.

Jack Ü deejayed the bash, which also drew Perry, Zoë Kravitz, Jena Malone and Fergie.

Might babies be on the way for Robert Pattinson and FKA Twigs?