Gaffigan's new comedy album, Cinco, will be released just in time for Father's Day

Credit: Courtesy Comedy Dynamics

Though Jim Gaffigan may take the stage solo, the comedian insists cracking up audiences is a family affair.

The stand-up veteran tells PEOPLE while announcing the upcoming release of his fifth album, Cinco, by Comedy Dynamics, that he and wife Jeannie Gaffigan have made comedy an important part of their children’s lives.

“Humor is of incredible value in our family,” Jim says.

The 50-year-old and Jeannie share five kids: daughters Marre and Katie, and sons Jack, Michael, and Patrick.

As always, the couple worked together on Jim’s latest special, with Jeannie serving as co-writer, director and executive producer of Cinco. She also joins Jim on the commentary that accompanies each track of the upcoming album – which PEOPLE is exclusively previewing.

“Our relationship has always been – ever since we first started dating – it’s been collaborative,” Jim explains. “So we’ve always had this similar work ethic where we don’t really view it as a job. And it doesn’t happen in a set or even a scheduled time, it’s an ongoing discussion.”

He adds, “And having five young children it would be impossible to do that. You know, years ago we used to put the kids to bed and then we’d drink wine and talk about stand up, but those days are a fond memory.”

As their kids grow up, Jim says they’re starting to understand more and more what mom and dad do.

Credit: Robert Trachtenberg

“My oldest is 12, so I’m still holding on to her but I have an expectation that I’ll eventually lose all of them and none of them will think I’m funny,” Jim says, adding, however, “[Home is] definitely a funny environment.”

In Cinco – which is currently available to stream on Netflix – Jim hilariously touches on parenthood. He says, however, that all of his kids aren’t aware they sometimes feature heavily in his specials.

“They’re not watching my stand up – my younger ones, if you asked what I did, they would say, ‘He works on his computer,’ ” he explains. “The three older ones, they know what I do.”

Credit: Courtesy Comedy Dynamics

Adds Jim, “It’s interesting because, my 12-year-old and an 11-year-old, they’re at the age where some of their friends are really into stand up. And because I’m not filthy or anything like that, it ends up being the type of album that their parents would let them listen to.”

So will the two-time Grammy nominee’s brood follow in his footsteps?

FROM COINAGE: See Where 6 Stars Were Before They Were Famous

“I would say they’re definitely interested [in entertainment], but specifically what aspect, I don’t know,” he says. Yet, the kids are already honing their sense of humor.

Jim shares, “They’re figuring out sarcasm, but sometimes their sarcasm is just cruelty to me.”

The Cinco album will be released on CD, iTunes, Apple Music, Google Play, Spotify and other digital audio retailers on June 13th. It is currently available for pre-order on iTunes.