Mother of two Jana Kramer discussed her decision to forgo breastfeeding on the latest episode of her podcast with husband Mike Caussin

Jana Kramer is opening up about her decision to not breastfeed.

In Monday’s episode of her podcast, Whine Down with Jana Kramer, the country singer and One Tree Hill alum discussed her choice to not breastfeed — and the reactions she has received because of it.

Kramer, 35, welcomed her second child, a son named Jace Joseph, with husband Mike Caussin on Nov. 29. Her newborn joins big sister Jolie Rae, who turns 3 this month.

“I’ve had so many people reach out to me on DMs … it’s been really great,” Kramer said on the podcast of the reactions after she revealed she wasn’t breastfeeding, “but it also [is] just really frustrating.”

Want all the latest pregnancy and birth announcements, plus celebrity mom blogs? Click here to get those and more in the PEOPLE Parents newsletter.

Credit: Kara Coleen/
Jana Kramer babyCredit: Courtesy Jana Kramer
Jana Kramer with newborn Jace and husband Mike Caussin
| Credit: Courtesy Jana Kramer

While the breastfeeding questions are more often directed at his wife, Caussin can still “sense that tension” when he’s there. But he can also understand why the inquiries arise in the first place

“I get why [people ask] because that’s the way humans have done it for so long, so anything other than the way you’re biologically created to feed your child, I can see why it’s still a lasting stigma,” he said.

“I totally get that. It’s just frustrating when women [are told] breast milk is the best,” Kramer continued. “I understand that — [they] don’t need to tell me something that I [already] know.”

RELATED VIDEO: Emily Maynard Johnson Admits She Was Ashamed to Stop Breastfeeding Her Second Son Early

“But I also don’t need to explain to [them] why it’s either not working or we’re not doing it or we chose not to. I don’t know why we feel like we have to explain,” she said.

“I feel like I have to hide the bottle when I’m in public because people are judging me. I feel judged,” added the “I Got the Boy” singer, revealing that she was not breastfed as a child.

“I actually don’t even think we should or need to [explain] because then it’s like you feel like you have to defend why we’re not,” Caussin agreed. “We’re the only two people that matter in this entire situation.”

“I guess I’m saying this to, also, the moms out there that feel this way too,” Kramer continued. “Let’s not feel like we have to defend our decision. Let’s just be the best moms that we are.”

Jana Kramer and family
| Credit: Ari Michelson

Shortly after welcoming Jace, Kramer told PEOPLE in December that she “had zero patience before kids.”

“Our marriage, you think would be more stressful,” she added, referencing her relationship with Caussin, “and don’t get me wrong — there are times when we are sleep-deprived and short with each other because we’re so tired and you take that out on the people you love, but we have a quicker rebound now.”

In June, Kramer opened up to PEOPLE about how the couple endured three miscarriages and two chemical pregnancies before conceiving naturally following a failed IVF cycle and subsequent loss the previous February.

“It’s hard because it’s like, ‘Have hope,’ but when you’re in that moment, hope is the hardest to find,” she said. “It’s so hard not to [stress] when you want a family so bad.”