May 11, 2016 03:00 PM

After seven years living in the Playboy Mansion with her former boyfriend Hugh Hefnerand his various other girlfriends, Holly Madison couldn’t wait to start life on her own terms. But when she finally left the Mansion in 2008, she discovered her past would follow closely behind.

“I didn’t have my own identity,” explains the author of The Vegas Diaries. “For so long I wasn’t myself. I was only known as somebody’s girlfriend.”

Madison, then 29, moved to Las Vegas to start a new career as a burlesque dancer but admits, “there would be other things I wanted to do and people wouldn’t give me a chance because I was a Playboy bunny.”

Holly Madison
Photographs by Gregg Segal

Continues Madison: “When you have a lot of baggage and it comes from a seedier place, sometimes people just want to write you off. The hardest part was knowing there were so many haters waiting for me to fall. The public consensus was, ‘Okay, this dumb girl is leaving her rich boyfriend, she’s going to fall flat on her face.’ ”

Still, Madison persevered. “I was starting from scratch,” she says. “And I had all of this heavy baggage and a scarlet letter on my forehead.” Dating proved to be an even bigger challenge. “I was definitely embarrassed [by my past],” she says. “It was the elephant in the room.”

The Vegas Diaries

For exclusive photos and much more from Madison, pick up the new issue of PEOPLE, on newsstands Friday

After a string of failed relationships, Madison says she was afraid she might never find “the one.” “I wondered, ‘Is this because I spent my 20s at the Mansion?’ I worried I missed my chance because I was there the whole time.”

Continues Madison: “I got to a point where I was trying to adjust my mindset and think, ‘Okay, am I going to be happy without that picture of marriage and family?’ The hardest part was just feeling really alone.”

Kelly Ripa on the cover of PEOPLE

Now happily married to events company owner Pasquale Rotella with a 3-year-old daughter, Rainbow Aurora, and a son on the way, Madison has no regrets about how life turned out.

Holly Madison
Photographs by Gregg Segal

“I love looking back and seeing the silver lining in the clouds,” she says. “Everything happens for a reason.”

You May Like

EDIT POST