Although he "feels bad that Selena isn't doing well, that has nothing to do with his gloomy mood a few days ago," a Bieber source tells PEOPLE

By Emily Zauzmer and Linda Marx
October 15, 2018 12:55 PM

Justin Bieber still has love for his ex Selena Gomez — even though he is a married man.

A Miami music source tells PEOPLE of the “What Do You Mean?” singer, who secretly tied the knot with Hailey Baldwin, 21, in September, “He is not over Selena.”

“This was his first great love and while he was young and traveling around the world as a superstar, he learned a great deal from her,” the source explains of Bieber, 24, and Gomez, 26.

Haiely Baldwin and Justin Bieber on Monday
| Credit: Pap Nation/Splash

The source says that Gomez’s health struggle is taking a toll on Bieber. Gomez, who was hospitalized twice in the last few weeks for a low white blood cell count, recently entered dialectical behavior therapy after suffering a panic attack during her second hospitalization.

“He was extremely upset over what has happened to Selena,” the music source explains. “He feels like she is part of his life and he wants her to be happy and healthy.”

Justin Bieber and Hailey Baldwin
| Credit: Snorlax / MEGA

Bieber was seen crying in a car with Baldwin on Thursday as they left their pastor’s home. He also was spotted looking somber as he hugged friends at church in Los Angeles.

A source close to Bieber tells PEOPLE, though, that Bieber’s public displays of emotion were unrelated to Gomez.

“Hailey makes him super happy,” the Bieber source said. “He feels bad that Selena isn’t doing well, but that has nothing to do with his gloomy mood a few days ago.”

In July, Bieber popped the question to Baldwin just four months after splitting from Gomez, who he dated on and off since 2011.

Selena Gomez and Justin Bieber in 2011

RELATED VIDEO: Justin Bieber Serenades Wife Hailey Baldwin Outside Buckingham Palace with One of His Songs

Earlier in October, the Bieber source told PEOPLE, “Justin will always care about Selena.”

The Bieber source added, “It’s hard for him to hear that she isn’t doing well. They have both had their separate struggles over the years and Justin was always very sympathetic to Selena’s. He wants the best for Selena. He wants her to be happy and healthy.”

Credit: Billy Farrell/BFA/REX/Shutterstock; James Devaney/GC Images

Another source told PEOPLE in October that Bieber’s marriage is one of many causes of stress for Gomez.

“Yes, having Justin’s relationship everywhere isn’t easy, but she’s also been dealing with other stressful situations and being back in the hospital was really difficult and brought back tough memories,” the second source said. “It’s not fair to say this caused it or that caused it because depression and anxiety is so layered. She knows it’s something she will be facing for the rest of her life, but she’s committed to tackling her issues head on.”

Justin Bieber and Hailey Baldwin in September
| Credit: UMV/Star Max/GC Images

As for whether the singer’s lingering affection for Gomez could create issues in his marriage to the model, the source close to Bieber says, “It could be a problem down the road.”