T.J. Miller has denied the allegations, saying, "I'm not high when I work because it gets in the way of the comedy"
In a Hollywood Reporter cover story published Wednesday, various sources associated with the series admitted the decision to cut ties with Miller, 36, was “a long time coming,” making thinly veiled references to his “demons” and alleging his use of drugs and alcohol interfered with his ability to do his job. (Miller has been outspoken about these vices as well, referencing them in his comedy.)
Multiple show sources told the magazine that there had been stretches when Miller “looked to have things under control, and others when he’d show up seemingly under the influence, if he showed up at all.”
“There are a lot of different ways you can find out somebody doesn’t want to do the show anymore,” said series creator Mike Judge. “And it’s not fun to work with someone who doesn’t want to be there, [especially when] they’re one of the main people and you’ve got however many crewmembers and extras and people who are [not paid as well] and they’re all showing up before 7 a.m., and then are just like, ‘Oh, OK, we’re not shooting today.’ “
According to the report, Miller would allegedly show up late to table reads, typically without having read the script in advance. Sources told the magazine Miller would fall asleep between takes on set, “leaving the cast and crew to nudge him awake.”
Miller defended himself and denied the accusations, telling THR that unlike his character, entrepreneur Erlich Bachman, in real life he’s “not always high.”
RELATED VIDEO: T.J. Miller Reveals the Last Emoji He Used Texting
“And this will blow your readers’ minds, but I’m not high when I work because it gets in the way of the comedy,” he continued. “I also am not a guy who’s blackout-drunk, bumping into things on set. … What was occurring was I was out doing stand-up all the time, even if it meant I only got three hours of sleep. So, the thing I have a problem with? It’s pushing myself to do too much.” (PEOPLE has reached out to Miller for further comment.)
But according to Judge, “it just wasn’t working.” In consultation with his producers and the network, he offered Miller the opportunity to return for three episodes in the series’ fifth season, a sendoff for his character.
Miller declined and decided to leave after season four. At the time, he cited his film career as the reason behind his exit.
As for how the show will continue without Miller’s central character, creator Alec Berg said he’s excited about the clean slate.
“These guys are the Golden State Warriors of comedy,” he said. “It’s like, yeah, we’ve lost Andre Iguodala but we still have Steph Curry and Kevin Durant and Klay Thompson and some other guy on the Warriors whose name I don’t know. But I don’t feel like we can’t win championships anymore.”
Switching figurative teams, he added, “T.J. wasn’t LeBron.”
“Oh, that’s great,” Miller told the magazine with a laugh in response to Berg’s statement. “And it makes me like him more [because] he’s so good at being an a—hole.”
Season 5 of Silicon Valley premieres March 25 on HBO.