"I remember getting so much shame and so much hate," Jana Kramer recalled of her decision not to breastfeed in Monday's episode of her Whine Down podcast

Products in this story are independently selected and featured editorially. If you make a purchase using these links we may earn commission.

Jana Kramer is riding the “fed is best” train when it comes to feeding her 13-month-old baby boy Jace Joseph, but it took some trial and error for her to accept it.

On Monday’s episode of her Whine Down podcast, the actress and singer recalled that she initially “tried” nursing her and husband Mike Caussin‘s daughter Jolie Rae, 4 this month — but in truth, she “didn’t want to breastfeed.”

“I had no desire. I wasn’t breastfed,” said Kramer, 36. “I didn’t feel like I had to have that connection with my baby. Personally, I don’t like my nipples being touched. I don’t like it. I don’t want someone sucking on ’em.”

“But it was really important to Mike so I said, ‘Okay, I’m gonna try,’ ” she continued. “[But] my milk didn’t come in and then they wanted me to supplement and they wanted me to pump, and it was so stressful. I was just crying and I was like, ‘Oh, my gosh, my baby lost even more weight than she should’ve.’ ”

“And then my milk eventually came in but we had already made the decision to formula feed,” said the One Tree Hill alum. “I remember getting so much shame and so much hate.”

Jana Kramer
| Credit: John Shearer/Getty Images

Never miss a story — sign up for PEOPLE’s free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human interest stories.

So for the couple’s second child, Kramer made the formula choice before he was born — but with that decision came some backlash, due to a medical complication early on in Jace’s life.

“He got [Respiratory Syncytial Virus] when he was just a couple weeks old and everyone was just like, ‘Breast is best, this is how he’s not gonna get sick,’ and I was like, ‘Oh my God, they’re right. If my kid ends up getting worse, it’s because of me, because I didn’t breastfeed,’ ” she said.

“And so I had two of my girlfriends come over … my boobs still had the milk in ’em but they were just so engorged and just gross, so they had the pump on me. I literally had my friends Ashley and Julie massaging my breasts to try to get milk out, and I’m just bawling,” the mother of two added.

“And then we’re trying to get Jace to connect [with] my nipple and I didn’t know how to do it ’cause I never breastfed, so I didn’t know how to latch it and how to do all these things,” Kramer said. “Mike walks in the room, he’s like, ‘Why are you doing this to yourself?’ And I was like, ‘Because I have to — because everyone says that he’s gonna get worse if I don’t breastfeed!’ ”

Caussin, 32, then reminded his wife that her nurse told her it “doesn’t matter” if she uses breast milk or formula and that as long as Jace gets the nutrients he needs, he was “going to be healthy one way or the other.”

RELATED VIDEO: Emily Maynard Johnson Admits She Was Ashamed to Stop Breastfeeding Her Second Son Early

Kramer previously got candid about breastfeeding on her podcast a year ago, discussing with her husband the reactions she has received because of her decision to use formula instead,

“I feel like I have to hide the bottle when I’m in public because people are judging me. I feel judged,” said the "I Got the Boy" singer, also revealing then that she was not breastfed as a child.

“I actually don’t even think we should or need to [explain] because then it’s like you feel like you have to defend why we’re not,” Caussin agreed. “We’re the only two people that matter in this entire situation.”

“I guess I’m saying this to, also, the moms out there that feel this way too,” Kramer continued. “Let’s not feel like we have to defend our decision. Let’s just be the best moms that we are.”