Shawn Johnson and husband Andrew East talk baby No. 2, sex during pregnancy and more on the latest episode of Hot Marriage Cool Parents

Shawn Johnson East and Andrew East are officially not not trying to have another baby.

The couple appeared on Sunday's episode of Jamie Otis and Doug Hehner's Hot Marriage Cool Parents podcast, where East joked that they had "pulled the goalie" in their journey toward giving their daughter Drew Hazel, who turns 1 later this month, a sibling.

But Johnson East is hesitant to come out and say definitively that they are trying to conceive, as she still has residual complicated feelings over her past miscarriage and doesn't want to set herself up for disappointment if having a second child ends up taking them a while.

"I mentally don't want to say, 'We're trying,' because I don't want to put myself in the position where we get a year down the road and it's like, 'Oh, we've been trying for a year and it hasn't happened,' " said the athlete, 28.

Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human interest stories.

Shawn Johnson, Andrew East
Andrew and Shawn Johnson East with daughter Drew
| Credit: Shawn Johnson/Instagram

Johnson East admitted she's "already protecting my heart because of everything you go through as a mom." (The former Olympic gymnast has been open in the past about her loss, as well as her experiences with mommy shamers over everything from formula-feeding her daughter to her methods of dancing with baby Drew.)

"I'm like, 'Oh, no, we're not trying, we're just playing around. And then six months down the road, we'll start trying.' It sounds dumb but it's just, I think, an emotional protectant," she added.

The mother of one and East, 29, previously revealed she had suffered a miscarriage in October 2017, before going on to announce they were expecting again in April 2019. Drew was born the following October.

"[After a loss,] it is [scary]," Johnson East said on the podcast. "And I feel like it never gets easier, which is funny. It's like, 'Oh, once I have our kid, it'll be fine and I won't be worrying.' But then you have a child and [it's] anxiety all day, every day. [You always think] something's gonna go wrong. It never gets easier but it's always worth it, obviously."

RELATED VIDEO: Shawn Johnson Opens Up About Body-Image Struggles, Drug Use and Going from "110 Lbs. to Pregnant"

Having suffered multiple pregnancy losses of her own before welcoming son Hendrix Douglas, 5 months, and now-3-year-old daughter Henley Grace with Hehner, 37, Otis, 34, asked her guests whether they were "scared" to have sex while expecting Drew, "for fear of losing the baby."

"No," said East, with his wife chiming in, "We didn't, only because we miscarried early enough on that that was never a thought in our mind."

As for postpartum intimacy, the former NFL player joked, "We walked through the desert there for a little stretch of time." And according to Johnson East, it was "like PTSD" getting back to a place where she felt comfortable having sex, as she "went through a lot of different experimental things trying to get Drew out."

"It just took a while to find that comfort again," she added.