Justin Bieber and Hailey Baldwin Wear Their Wedding Bands for the First Time

The couple wed in secret courthouse ceremony last September

Justin Bieber and Hailey Baldwin may be taking it slow planning their big wedding for friends and family, but they just took a major step forward in their marriage: They were both spotted out this week wearing wedding bands for the first time.

The couple, who tied the knot last September in a secret courthouse ceremony, were seen shopping at Cartier, the luxury jewelry house, over Memorial Day weekend and just debuted their new hardware to the world.

Baldwin, 22, was spotted leaving a workout on Monday sporting her blinding engagement ring with a silver band below the diamond.

On Monday, Bieber, 25, posted a series of modeling shots on Instagram, and in one post he was seen holding his head, showing off his silver band.

Baldwin has been wearing her diamond ring since last July, when the news broke of their engagement. Bieber got the sparkler from Jack Solow, a private jeweler based at New York City’s family-run jewelry store Solow & Co., who helped customize it to perfectly complement Baldwin’s style.

“In the end, Justin wanted something that would gracefully accentuate Hailey’s beautifully shaped hands. We settled on an oval stone that was just exceptional in its length and grace,” Solow told PEOPLE, adding, “Look at her hands — she’s exquisite,” The jeweler said he had a “few weeks” to make the custom diamond sparkler, which is somewhere between 6 to 10 carats.

After saying “I do” at City Hall last year, a source recently told PEOPLE, while that the duo are in no rush to have a bigger ceremony, they do still want to throw a wedding at some point in the future that includes friends and family. For the time being, though, Bieber is tackling his mental health struggles, a source told PEOPLE.

“Hailey keeps supporting him. They will still have a wedding when they feel they are ready,” the source said. “The most important thing for them is that Justin is mentally healthy.”

Bieber began seeking treatment for depression in February and is doing “much better,” the source also said.

“It seems his treatment has helped him a lot. It has changed the way he thinks,” the insider said. “He is much more focused on taking one day at a time. It’s when he starts thinking too far ahead that he gets stressed out and feels pressure.”

A source close to the star previously told PEOPLE that Baldwin was one of the most important factors in the singer’s decision to seek treatment.

“He wants to be the best possible husband for Hailey,” the source said. “It’s one thing to have your issues when you’re single, but when you’re married, there’s two people’s happiness at stake. He’s working on himself so that he can be a good partner to her.”

In March he wrote a lengthy Instagram message explaining that he would not be working on new music anytime soon, but that may be changing. On May 26 he shared a photo with Baldwin in the studio captioning the pic, “Studio vibes.. with my studio chicka.”

Last week Baldwin also shared a sweet pics embracing Bieber captioning the slide show: “Safe place. happy place.”