"We spice things up a bit and keep it fun," Kelly Rowland exclusively tells PEOPLE

Kelly Rowland is bringing the heat to her new music, marriage and steamy sex life.

In this week's issue, the star behind the hot new tune "Coffee", 39, opens up about both the serious and sensuous tricks she uses to keep the romantic spark alive in her marriage to husband Tim Weatherspoon. Rowland and her husband, 38, wed in 2014 and are parents to their 5-year-old son Titan.

"As far as sex is concerned, I'm like, 'Well, if I have to play dress up and do role-play, honey, if I need to be Alicia tonight and give you a surprise in the middle of the night or something then it needs to happen. We spice things up a bit and keep it fun," Rowland exclusively tells PEOPLE. "When we met each other, we had fun. Two years from now, we’ll have fun. I think if you hold that close to you, it can always show itself in your relationship.”

kelly rowland and tim witherspoon
Tim Weatherspoon and Kelly Rowland
| Credit: Charley Gallay/Getty

Sexual liberation is an important theme for the Grammy-winner, who sings about a sexy morning rendezvous in her latest single "Coffee."

“It's really about owning your sexuality," says the star. "You should already know how sexy and great you are without someone having to tell you. As women, for so long we'd do something to be regarded by others and this song says, ‘I really don’t need to hear what anyone else has to say.’"

The accompanying music video to the flirty bop has yet another powerful message. Rowland says she wants the visual to show off "the different tones, textures and colors of beauty that there are, and how rich that is, and how unified we are as women. That really meant a lot to me as well. You see, ‘I’m just one person, but I’m even better with other women next to me.'"

EXCLUSIVE: Go Behind the Scenes of Kelly Rowland's New Music Video

The Destiny's Child alum says that black women "for so long, we’ve been trying to be seen. For so many generations, we were fighting to be seen. This video is like, ‘No, you’re gonna see us in all our splendor!' For me, that was something I really wanted people to grasp when they saw the video. That meant a lot to me."

When she isn't busy empowering women or homeschooling her son in their Los Angeles abode, the singer is tending to her husband's needs within their marriage with the goal of keeping an honest dialect in their relationship.

“If I wasn’t taking the time to get to know myself better in this crisis, we would be struggling. We go to therapy every week and we’re so proud of it," she tells PEOPLE. "It’s just to keep the relationship and the space of communication open, [so we're] ready for each other. I have a husband who supports me and is there for me and loves me and I’m the same for him. I met my best friend. There’s nothing that we can’t talk about. When I meet an older couple, I always ask them, ‘What’s the one thing that can break up a marriage?’ and they’re like, ‘Money and communication.’ With every challenge, there is a solution and I think he and I have grown to talk about everything." 

RELATED VIDEO: Kelly Rowland's Most Embarrassing Parenting Moment Is So Relatable

Rowland tells PEOPLE she is happy and willing to do just about anything to maintain her loving marriage — that effort is one of the secrets to her and Weatherspoon's everlasting love.

"Before I was married, I was like, ‘Well, if it doesn’t work, you just get a divorce," she reveals. "[But now,] that’s not an option for me. That’s not an option for me because I know I have a good man. I know I have someone that loves me unconditionally and we’re gonna sort this thing out, we’re gonna figure it out. Above anything, we’ve got faith and we’ve got each other. Marriage means a lot to me. It’s something that I hold very sacred.”

For Rowland, whose late mom Doris separated from her father — a heavy drinker — when she was a child, healing some of her deepest wounds is at the forefront of her new approach to love.

“I didn’t grow up with my father, so everybody’s like, ‘Oh yeah, you got daddy issues.' It’s deeper than daddy issues," she explains. "That’s just a surface way to try to say what it could be that somebody is dealing with. I think when you get to the real stuff that causes the pain — which is abandonment and someone actually not being there — [you can understand] what you're basing your relationships off of and what you subconsciously are wanting from a lover or mate."

kelly rowland and family
Tim Weatherspoon, Titan and Kelly Rowland
| Credit: Stefanie Keenan/Getty

For more of Rowland's hard-won lessons about marriage, motherhood and her unbreakable bond with fellow Destiny's Child alum Beyoncé and Michelle Williams, pick up the latest issue of PEOPLE, on newsstands Friday.