At their home in Los Angeles, the "Daisies" singer, 35, "is still decorating the nursery and having fun with it," a source close to the actor, 43, tells PEOPLE in this week's issue, on newsstands Friday.
During a May appearance on The Graham Norton Show, Perry said she was "learning to be a mom fast" while social distancing at home amid the coronavirus pandemic thanks to the family members staying with her, including her 9-month-old nephew, 3- and 6-year-old nieces and sometimes Bloom's 9-year-old son, Flynn, whom he shares with ex-wife Miranda Kerr.
"Kids gravitate towards me through my music, so I am used to them," she said. "But I am not used to being woken up really early in the morning by them staring at me from the side of my bed!"
Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human interest stories.
When they're not at home in L.A., the couple can typically be found in Santa Barbara — Perry's hometown — as of late.
"Orlando and Katy both love the beach," the source says. "They enjoy the drive up the coast from L.A. It's a beautiful change of scenery for them."
Perry, who started dating Bloom in 2016, first announced her pregnancy in March in her "Never Worn White" music video. With just "several more weeks" to go until her due date, Perry "seems less stressed but is definitely a bit nervous about giving birth," says the source, adding, "Orlando tries to be as supportive as possible."
RELATED VIDEO: Pregnant Katy Perry Shows Off Growing Baby Bump and Says Daughter Has Been Kicking: 'It's Painful!'
Though their plans for a babymoon and a summer wedding in Japan were nixed because of the pandemic, the silver lining has been the amount of time the couple have gotten to spend with each other.
"Katy loves that Orlando is not working and that he is around all the time," the source says. "She is super excited about her baby girl."
Next, Perry will co-headline Rock the Vote's live virtual event with the Black Eyed Peas on Thursday night. "The young people of America are speaking loud and clear on the streets and online, and come November, it will be more important than ever to fight for justice and equality, and against systemic racism, with our ballots," she said in a statement.
For more on Katy Perry and Orlando Bloom's count down to baby, pick up the latest issue of PEOPLE, on newsstands Friday.