"It was very, very strange," Jon Cryer said

Jon Cryer is reflecting on his days starring alongside Charlie Sheen on Two and a Half Men.

During a recent appearance on PeopleTV’s Couch Surfing, Cryer, 55, said he only has fond memories of his time working with Sheen on the hit CBS sitcom — despite Sheen’s tumultuous exit from the show.

“Charlie and I just hit it off. We had a great first few years on that show,” he said. “It was incredibly smooth, we had a lovely time, it was working really well.”

While Cryer said Sheen, 54, was sober when the show first premiered in 2003, he believed that the actor’s split from wife Denise Richards in 2006 led to a relapse, prompting problems on set.

“At first he could handle it, and he was still incredibly professional — and still lovely, by the way, to everybody on set — but you could just see that stuff was wearing on him,” he said.

Credit: E.J. Camp/CBS via Getty

A rep for Sheen did not immediately return PEOPLE’s request for comment.

Two and a Half Men ran for 12 seasons until 2015. However, Sheen’s contract was terminated in 2011 after the star entered a rehabilitation center and verbally attacked the show’s creator, Chuck Lorre, in a number of interviews.

“It became one of the first internet s—storms, plus there’s the fact that all of our jobs are tied into this guy, and we’re all his friends,” he said.

After Sheen’s exit, Ashton Kutcher stepped in for the remainder of the sitcom’s run as a new character.

Looking back, Cryer said he doesn’t hold any hard feelings against Sheen.

“It was very, very strange and [there were] aspects of it that I absolutely hated, but you know, I’m thankful for the years that were great,” he said.

Two and a Half Men
Jon Cryer and Charlie Sheen
| Credit: Tony Esparza/CBS/Getty

In 2018, Sheen campaigned for a Two and a Half men reboot after Roseanne was canceled in the wake of a racist tweet by star Roseanne Barr.

“adios Roseanne! good riddance. hashtag NOT Winning. the runway is now clear for OUR reboot,” he wrote on Twitter, adding the hashtag #CharlieHarperReturns and a photo of a script.Cryer even retweeted the post with a message: “What could possibly go wrong?”