"He was a genius ... he had a photographic memory or something," Alfred Jackson says of his little brother

By Char Adams
April 29, 2016 03:30 PM

Musical genius Prince died April 21 at the age of 57. Subscribe now for an inside look into his private life and shocking death, only in PEOPLE.Prince‘s half-brother is breaking his silence, revealing details about the childhood of the reclusive music legend.

Many unknown details about the mysterious singer‘s personal life have come to light in the days following his death at his Paisley Park home in Minnesota last week. And now, Alfred Jackson, one of Prince’s five half-siblings, is speaking out about growing up with the icon.

Alfred Jackson

“When he was younger, my father and mother would put him on stage, and he was dancing on stage and you should have seen him out there – he wasn’t but 5 years old,” Jackson told Entertainment Tonight of his little brother, noting that Prince was a big fan of James Brown.

Paisley Park Films/Entertainment Pictures/Zuma

According to Jackson, it was he who influenced Prince’s music when he was developing his sound at an early age. He recalled playing the piano with Prince in their mother’s basement, and Prince copying everything he did but “faster” and “better.”

“He was a genius,” Jackson said. “Wherever he heard music, he could copy it just like that – like he had a photographic memory or something.”

Jackson said he hadn’t seen or spoken to Prince in 15 years, but always relished seeing his little brother on TV and reading about him in magazines – ” ‘There’s my brother, Prince.’ I was so happy he was making it for himself,” Jackson told ET.

Although Prince is known for his humble-yet-self-assured, confident nature, Jackson said his brother was once self-conscious about his 5’2″ stature.

“I think he wanted to be taller because he wanted to be a basketball player,” Jackson said, noting that he and Prince would shoot paper into a basket in the basement. He added that Prince soon joined his school’s basketball team.

“They did him a favor because he was the smallest on the team,” he said. “He did better than some of the rest of them.”

Jackson, along with Prince’s other half-siblings and lone full sister, Tyka Nelson, is named in a petition to appoint a special administrator in the case of Prince’s estate. The singer has no will in place to declare the distribution of his assets.

Prince: the Private World and Shocking Death of a Musical Genius

For more on Prince’s private world and tragic death, pick up this week’s issue of PEOPLE, on newsstands now.

On Wednesday, a Minnesota judge appointed Bremer Bank to oversee the singer’s massive estate, which is estimated to be worth $250 million.

Jackson and his lawyer were expected to head to a meeting with “all the beneficiaries and the heirs” on Thursday to discuss the situation.

“I miss my brother, because my brother was everything in the world to me,” Jackson said. “He’s a legend, he really is.”