Kal Penn on His Childhood, Racism (and Progress) in Hollywood and Obama's 'Uplifting' White House

In his new book You Can't Be Serious, the actor details his unique career path, the obstacles he's overcome and why he's hopeful about the future

Photo: Miranda Barnes/The New York Times/Redux

Kal Penn's personal and professional successes are the stuff that dreams (and good reads) are made of.

The New Jersey native has starred in a beloved film franchise and acclaimed TV shows, and also spent two years working in the White House for the Obama administration, all of which he details in his new book You Can't Be Serious.

Penn wrote the book "for both the 20-year-old version of me and for anybody else who has ever been told that that crazy thing they want to do is impossible," he tells PEOPLE in this week's issue.

Born Kalpenn Suresh Modi to mother Asmita Bhatt, a chemist, and father Suresh Modi, an engineer, Penn says growing up in ethnically diverse Montclair, New Jersey, where he spoke his parents' native language Gujarati at home, offered him invaluable insight into and comfort with American and immigrant culture (Penn's parents immigrated to the U.S. from India in the '70s).

"I grew up bilingual, but never felt like that made me either more Indian or less American," he says. "Although I totally understand that a lot of people do feel that way when they grow up in an immigrant household."

For more from Kal Penn, pick up the latest issue of PEOPLE, on newsstands Friday, or subscribe here.

Courtesy Kal Penn

The future Harold and Kumar Go to White Castle star (seen above with his grandparents Bapaji and Ba, who joined Gandhi in his peaceful protests against British colonialism) had no idea he was "different" until a playground bully called him the N-word in kindergarten, an unforgettable memory that opens his book.

"I didn't even know what that word meant," he says. "But I knew that he thought I'm supposed to feel different and that was somehow bad. But I didn't feel that way in real life."

That incident was unfortunately the first of many, particularly when Penn was trying to break into Hollywood.

After graduating from UCLA, where he says he was "the only Indian kid in the entire theater department," Penn writes that he decided to adopt the stage name Kal Penn (rejecting a friend's suggestion of Kal Pacino) and continued pursuing his dream of acting despite mounting racist encounters within the industry, such as: a casting director being visibly and vocally upset that he was neither mixed-race or "Latin," sitting with a white actor wearing brown makeup at a final casting call for an Indian role, and one manager who said he was a good actor but might only "play a cab driver once or twice."

"I've said this before, that working in entertainment at the time that I started was one of the more racist places I've ever worked," Penn says. "I always think of Seinfeld and Friends, two really funny shows that I really enjoyed watching, but you have to try real hard to exclude people of color from New York City — they did and they both made that choice."

Now, with his own successes — the Harold and Kumar movies, a compelling two-season arc on House, roles on Designated Survivor, How I Met Your Mother and running his own short-lived passion project Sunnyside — Penn's outlook on the entertainment industry is far more hopeful.

"Hollywood was [once] where I felt the most excluded," he says. "It's also where I feel the most home and opportunity now. I'm excited about that."

"There's no denying that 2021 looks a heckuva lot different than 1998, in terms of diversity of content and characters," Penn continues. "Streaming platforms deserve a lot of credit for that because they're not beholden to advertising revenue in the same way and it's proven that audiences just want good content. They don't care what color you are, who you date, where you're from, which is how it should be. There's room for all of us."

Pete Souza

Coincidentally it was Penn's acting career, specifically his role on House, that put him on an indirect path to the White House.

In 2007, House costar Olivia Wilde brought Penn as her plus-one to a reception for Barack Obama's 2008 presidential campaign, which inspired him to volunteer on the campaign trail.

"It felt like I was living in a far-off idealistic future where people are uplifting, inclusive, encouraging and also don't ask you where your turban is," he recalls in the book.

When Obama was elected, Penn left House to take a job as a liaison to Asian Americans, Pacific Islanders, young Americans and the arts community in the White House.

"The majority of work you do isn't either sexy or vitriolic enough to end up on cable news," he says now, though he's proud to have worked on the Affordable Care Act and the repeal of the U.S. military's Don't Ask, Don't Tell policy while spearheading smaller items like an executive order helping Vietnamese American fishermen gravely impacted by the BP oil spill in 2010.

"It was really meaningful," he says of the work he was able to do. "What can seem like the most mundane, bureaucratic things have such an impact for so many families. That was the real humbling part of walking in there everyday."

Penn's book You Can't Be Serious is out now.