"The time apart is hard ... but we make it work," Rippon tells PEOPLE of his relationship with Finnish real estate broker Jussi-Pekka Kajaal

The Trevor Project TrevorLIVE NYC 2018 - Arrivals & Cocktails
Credit: Dave Kotinsky/Getty

Finding some time for romance is hard enough when you’re an Olympic medalist-turned-Dancing with the Stars winner-turned-reality TV judge — but it’s even trickier when your beau lives in another country.

Just ask Adam Rippon.

“The time apart is hard,” he tells PEOPLE of his relationship with Finnish real estate broker Jussi-Pekka Kajaala, which Rippon confirmed in April. “But we have a lot going on, so we make it work.”

The pair, who first connected on Tinder last fall, do eventually plan to be in the same city: Rippon, 28, says Kajaala is working to relocate permanently to Los Angeles, where the atlete lives.

And while Kajaala’s dream has long been to move to L.A., Rippon says, he has been “trying to make it happen a little bit faster” since their relationship blossomed in the spring.

• Want to keep up with the latest from PEOPLE? Sign up for our daily newsletter to get our best stories of the day delivered straight to your inbox.

Paperwork is an obstacle, of course — “it’s a little harder than you think,” Rippon says of the bureaucratic process — but for now he and Kajaala are spending time together when they can. Kajaala, who can work remotely, will usually spend a few weeks at a time stateside.

“He’s been going back and forth a lot,” says Rippon, who recently joined the upcoming Dancing with the Stars: Juniors as a judge.

One thing that isn’t on the horizon? Marriage bells. Rippon tells PEOPLE the couple is more than happy going at their own pace.

RELATED VIDEO: Adam Rippon Hilariously Reacts to Photos From His Olympic Performance

When they’re both free, the two have been spending time on the red carpet — they made their couple debut in June — and double-dating with Rippon’s DWTS partner Jenna Johnson, or else (shirtlessly) enjoying the sunny outdoors.

Often Rippon is working, but he brings Kajaala along when he can.

“He can kind of see what I’m doing,” Rippon says of his boyfriend, adding, “I love to have him be a part of that.”