The two stars were also spotted together backstage at SNL over the weekend
First, the two attended the broadcast of Saturday Night Live in Manhattan, which was hosted by Swift’s good friend, Emma Stone.
“They showed up together,” a source tells PEOPLE. “They walked around together backstage, but they were careful not to be seen too close. It was hard to tell if they were together, but everyone was shocked that she brought him.”
On Sunday, Swift, 20 and Gyllenhaal, 29, brunched with another couple at Al Di La in Brooklyn and, says an onlooker, looked “friendly.” They were also spotted together strolling around Brooklyn’s Park Slope neighborhood.
While reps for the two stars had no comment about their relationship, the Prince of Persia star might be just Swift’s type.
Asked what she looks for in a man, the singer, who just released her new album Speak Now, recently told PEOPLE, “Right now the most attractive quality in a guy is a sense of adventure and romance. I’d love to meet someone who would do thoughtful things throughout the relationship to remind me that he was listening to me, knows me and wants to surprise me.”
What else? “Someone who does little things, just sweet little gestures,” she says, “that’s what I’d wish for in a guy.”
And, says Swift, she’s ready to put herself out there to find the right one.
“I love going on dates and getting to know people,” the singer also says. “For me, love is something I occasionally make an exception for if it feels right. I don’t search it out. I just think that if I’m supposed to meet someone, I will.”
• Reporting by JOEY BARTOLOMEO, MICHELLE TAUBER and CHARLOTTE TRIGGS