Jana Kramer shares son Jace, 2, and daughter Jolie, 5, with Mike Caussin, from whom she filed for divorce last month

Jana Kramer is spending Mother's Day with her two children.

The country singer, 37, shared a glimpse at her celebration on Instagram Sunday, including recent photos with her two kids, son Jace Joseph, 2, and 5-year-old daughter Jolie Rae. Kramer's post also included photos of her kids when they were born.

"The last month I kept saying 'why,' wrote Kramer. "Well now I know the why. It was so I could be a mommy to these two incredible babies. Being chosen as your mom will always be the greatest chapters of my life. Happy Mother's Day to all the mommas out there however that looks."

Kramer received a flurry of supportive messages in the comments section of her post from her followers, including from model Jessica Hall who wrote, "Thinking of you ❤️."

Kramer announced her split with Mike Caussin, with whom she shares her kids, last month. In the One Tree Hill alum's divorce filing, she cited "inappropriate marital conduct, irreconcilable differences and adultery" as grounds for divorce.

The filing also revealed the pair officially separated on April 20, just a day before PEOPLE learned that Kramer decided to file for divorce due to infidelity.

Want to get the biggest stories from PEOPLE every weekday? Subscribe to our new podcast, PEOPLE Every Day, to get the essential celebrity, entertainment and human interest news stories Monday through Friday.

Jana Kramer
Jana Kramer and family
| Credit: Jana Kramer/ Instagram

Earlier this month, Kramer opened up about feeling "grateful" for her son and daughter amid the divorce. Alongside a snapshot of herself sitting with her kids outside, the mom of two detailed the happy moment on Instagram.

"Sunshine is good for everyone," Kramer wrote. "I gotta be honest 2 weeks ago the sun was the last thing I wanted to see...but through grieving you start to see the light...even if it's a dim light it's still something I'm gonna be grateful for. "

"But I'll forever be grateful for these 2 and the forever light they add to my life," she added.

Mike Caussin and Jana Kramer
Mike Caussin and Jana Kramer
| Credit: Todd Williamson/E! Entertainment/NBCU/Getty

Infidelity has been an issue the couple has been candid about in the past. They previously separated in 2016 due to similar issues and documented some of their past marital problems in their book, The Good FightWanting to Leave, Choosing to Stay, and the Powerful Practice for Loving Faithfully.

According to the pair, Caussin filed for divorce in 2016 when Kramer was on Dancing with the Stars and he was seeking treatment for sex addiction. (They ultimately decided not to get divorced at the time and went on to renew their wedding vows in December 2017.)

"Mike was in rehab and I was in an all-out war with my emotions. Most days, I would push Mike out of my brain and tried to live in the world as if he didn't exist," Kramer wrote in a passage that she later read on their Whine Down podcast. "Other days I was too weak to push and I missed him, and I hoped he would be in the audience watching me perform, rooting for me."

Kramer added, "The only times that gave me calm was when I looked at our baby girl. I knew that regardless of the outcome, I wanted the best for her."