The couple took the photos with their newborn Jace Joseph after welcoming him on Nov. 29.
The proud parents — who recently welcomed their second child Jace Joseph together — posed for photos at their Los Angeles home alongside their family’s newest addition.
In the gorgeous shot, Caussin, 31, cradles his son’s head in his hands while Kramer, 34, watches over, admiring her baby boy who is wrapped in a white, fuzzy blanket.
The intimate moment comes less than two weeks after PEOPLE exclusively confirmed that the country singer and former NFL player’s son had arrived. Born in Los Angeles on Nov. 29 at 7:53 a.m., Jace weighed 7 lbs., 12 oz., and measured 20.5 inches.
“Welcome to the world Jace Joseph Caussin. Our hearts are so full. Thank you to all our friends and family and all of you who have supported Mike and I and our growing family,” Kramer said in a statement to PEOPLE.
The couple also shares daughter Jolie Rae, who turns 3 in January.
Want all the latest pregnancy and birth announcements, plus celebrity mom blogs? Click here to get those and more in the PEOPLE Parents newsletter.
A day after giving birth, the singer revealed the special meaning of her son’s name, who arrived following the couple’s struggles to conceive.
“Jace. My sweet baby boy. When I found out Jace meant ‘healing’ I knew right then that was his name,” the singer revealed on Instagram on Nov. 30, along with an adorable new photo of Jace in the hospital.
“He has been every bit of healing since I found out we were being blessed with another baby. I love you so much baby boy,” added Kramer.
In June, the One Tree Hill alum spoke to PEOPLE exclusively about her pregnancy news, opening up about how the couple — who reconciled after Caussin’s cheating scandal — endured three miscarriages and two chemical pregnancies before becoming pregnant naturally following a failed IVF cycle and subsequent loss that February.
“It’s hard because it’s like ‘Have hope,’ but when you’re in that moment, hope is the hardest to find,” she said. “It’s so hard not to [stress] when you want a family so bad.”
“Babies and having other kids don’t save a marriage but for us, this baby truly has been a savior for us in a way that we’ve really done our work and gotten dirty,” the Whine Down with Jana Kramer star added.
“We’re bringing a child into this world [so] we have to be there for each other and continue to do our own recovery work for him and me and be in a place where we’re bringing [our son] into the best environment possible,” she said.
And even though she’s been surrounded in baby bliss, Kramer recently took time to clap back at some of the hateful comments directed towards her after she posted a photo in which she wore form-fitting clothing.
RELATED VIDEO: Jana Kramer Expecting Second Child with Husband Mike Caussin After Multiple Miscarriages
“I wanted to post this photo (showing my stomach) because I wanted to show my journey back to health and my goodness I’m so glad I didn’t which is why this photo is now cropped,” she wrote on Instagram alongside an image of the star lifting her shirt to reveal her stomach. “It’s amazing the comments and how rude some people be.”
“Why do we women have to compare ourselves to each other and then shame?” the actress asked. “Our bodies are all beautiful and created differently, they heal different, they react different, they simply look different. Why do we need to shame someone for not looking a certain way? Or feel bad about ourselves for looking a certain way?”
“Can we be kinder to ourselves and know that every woman has a different journey but yet we are all beautiful? Can we lift someone up but not tear yourself down in the process with comparing? Let’s give that a try,” she remarked before concluding her post by telling her followers, “I love y’all. Back to my baby.”