"He was like a brother to me and I'll miss him terribly," Clark Gregg tells PEOPLE
Credit: Kelsey McNeal/ABC via Getty Images

Bill Paxton is being remembered by more loving costars.

Clark Gregg, who worked with Paxton on the Marvel show Agents of S.H.I.E.L.D., tells PEOPLE that the late actor was the type of guy to befriend everyone — and mean it.

“We just hit it off like old pals,” Gregg said about their time together. “I think he had that affect on a lot of people. We stayed friends. You say you want to stay close but this is a nomadic business and you hit it off and but may not see them again despite any promises.”

Paxton died due to complications from surgery, a family representative told PEOPLE on Sunday. He was 61.

Gregg says the actor was so serious about their friendship that he even surprised Gregg on set the season after he was done on the show. “He left the show after season 1 and I showed up for work next season and they said Bill is in your trailer,” says Gregg. “People don’t do that. That doesn’t happen. He ended up doing Training Day on the same lot and I’d get to work and they’d say Bill is in makeup he’s waiting for you. He was just the most incredible friend. He would come by all the time and we would just hang out.”

RELATED VIDEO: Bill Paxton Dies at 61

Since Paxton’s death, his many costars and others in Hollywood have taken to social media to remember the beloved actor. Gregg is joining them with his praise of the Paxton’s loving demeanor. “He had more great stories and more humility for a guy who had been more responsible for more quotable lines than probably anyone else,” Gregg said. “He was like a brother to me and I love him and I’ll miss him terribly.”

And Gregg reveals he last saw his good friend just a month before his death. “He was his usual great self. He’s an art collector and a writer and creator and an amazing dad,” Gregg said. “He spends a lot of time going to galleries and buying art. We were talking about how I wanted to buy a piece and I asked him to go with me after this season was over to help me pick something.”