Inside Simon Cowell and Howard Stern's Feud – and What Ended It

"There was a misunderstanding. That's all," an America's Got Talent insider tells PEOPLE

Photo: Eamonn McCormack/WireImage; Michael Loccisano/Getty

What feud?

Those who listened to Howard Stern‘s SiriusXM show on Monday were treated to a lengthy – and heated – segment regarding Simon Cowell.

Stern had read that Cowell had attempted to take the radio personality’s spot on the America’s Got Talent judge’s panel – and his radio co-hosts were not happy about it.

And an AGT insider confirms to PEOPLE that NBC did discuss replacing Stern with the former X Factor mentor in the event that Stern did not want to return for the show’s 10th season.

“There were a lot of conversations going on because Howard didn’t look like he was coming back,” the insider says of the negotiations that went on last fall. “It’s a huge show and you have to make alternative plans if one of the key players isn’t sure if he’s returning. Simon was an obvious call to make. But even before Howard made his decision to return, Simon had already said, ‘Thanks, but no thanks.’ He’s already committed to both his British shows, which film back to back. It just wouldn’t have worked.”

Catching wind of the misinformation Stern had received, Cowell – who owns and is an executive producer on the top-rated NBC summer talent show – quickly reached out to remedy the situation.

“There was a misunderstanding. That’s all,” says the insider. “Simon called Howard’s manager to make sure he knew the real situation and Howard sent a message back that he now understood and now they’re friends again. They’ve always got on well.”

In fact, Cowell – who is currently a judge on Britain’s Got Talent and the UK version of X Factor and will be bringing La Banda to Univision later this year – could have ended up on the panel with Stern if he had wanted.

“Simon turned down an offer from NBC four years ago out of loyalty to Fox,” says the insider. “And even in the last few weeks, he’s been called on to appear as a guest judge on this summer’s AGT but has declined the offer.”

“Simon loves the Got Talent format. And since he owns it, he approves all the judges, including Howard, but he wouldn’t appear on both the British and American version,” continues the insider. “He’s fully committed to [British TV network] ITV and Britain’s Got Talent for the foreseeable future. Howard’s been reassured he’s got his judging job for as long as he wants it as far as Simon is concerned!”