"It's just clutter," said the athlete of his many trophies and memorabilia
Charles Barkley is cleaning out his house in order to build homes for others.
Earlier this month, the NBA Hall of Famer and sports commentator, 57, announced that he intends to clear his stock of athletic memorabilia for auction, with the proceeds going toward constructing affordable housing in his hometown of Leeds, Alabama.
According to CNN, Barkley is selling his 1996 Olympic gold medal (he also earned one in the 1992 games, which he’s saving for his daughter to keep), his 1993 NBA MVP trophy, among other accolades and items from his storied career.
On Tuesday, Barkley explained why he’s “getting rid of all the crap trophies” he’s ever won while visiting The Late Show with Stephen Colbert, saying that his small hometown is in need of about 30 new houses.
“I’ve got a bunch of good stuff I’m gonna sell,” he told host Colbert, 55. “Man, it’s just clutter.”
He added: “You know, I tell people, when I’m 70 I can’t be saying, ‘Hey guys, come over and look at my trophies’ — I’m not that crazy.”
Barkley was the leading scorer for the U.S.A. basketball “Dream Team” at Barcelona in 1992 and in Atlanta in 1996, according to the official Olympics website. He played alongside Scottie Pippen, John Stockton, Karl Malone and David Robinson, among others.
Speaking to WJOX-FM radio show The Roundtable earlier this month, Barkley said he wasn’t attached to the material objects, and would rather give back to a community in need.
“That stuff’s not that important to me. I’ve had an unbelievable life,” he told the radio show, according to the Washington Post. “I’ve been in Leeds a lot and we’ve probably got 30 eyesores, as I call them, where houses used to be when I was growing up. Either a rotted-out house or there’s just weeds that have overgrown.”
He added: “I want to work with the City of Leeds; I want them to give me the spaces, number one. I want them to give me the houses, and I’m going to use my own money, selling my memorabilia.”
Never miss a story — sign up for PEOPLE’s free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human interest stories
Revealing that all the memorabilia is stashed at his grandmother’s house, Barkley said he’s content with parting ways with the historical objects — and that they could garner a six-digit selling point.
According to the Post, Barkley said the sports memorabilia company Panini — which the athlete is working with to auction the goods — projected his MVP trophy could attract nearly $400,000.
“I got an autograph deal with Panini and I was talking to the guys,” Barkley said, “All that stuff is at my grandmother’s house — I don’t even keep it, to be honest with you — and I said: ‘How much could I get for my MVP trophy and my two gold medals? And I got an autographed plaque signed by the  Dream Team.’ He says, ‘Oh, I can get you a lot of money for that stuff.’”