He says he wants the court to eliminate his ex's "ability to abuse and harass us"

Charlie Sheen says the FBI was “aware of” ex-wife Denise Richards‘s allegations that he viewed child pornography, and has taken no action.

Responding to Richards’s most recent court filing in their custody battle, Sheen, 41, says in a statement released Monday that for 18 months Richards herself filed “not a single complaint” about the welfare of their daughters – and that her attacks only began after he asked to pick his own nannies.

“I have asked that the court eliminate Ms. Richards’s ability to abuse and harass us any longer,” said Sheen, who has accused Richards of using the nannies as spies on his personal life. “One can only imagine what we’ve withstood thus far.”

In her filing, Richards accuses Sheen of “an attraction to underage women,” an allegation she made earlier, providing numerous printouts of porn Web sites he allegedly visited during their marriage. The filing has not been released publicly, but a copy has been reviewed by PEOPLE.

“The FBI was aware of these issues,” Sheen states. “I provided them with every computer I owned. The computers were returned two weeks later. Haven’t heard from them since. Period. The end.”

As of now, a childcare provider must accompany Sheen and fiancée Brooke Mueller whenever they’re with daughters Sam, 3, and Lola, 2.

Richards, 36, also alleged in her filing that Sheen’s engagement to Mueller is fake and a media stunt, based on what Mueller told her.

Mueller, denying Richards’s claims, responded alongside Sheen’s statement Monday, saying: “There are huge concerns about how Denise’s emotions are impacting her judgment.”

Richards had no comment to Sheen’s latest statement.

Who’s turning 40? Which celebs are high school dropouts? What’s hot and definitely not in fall fashion? Check out PEOPLE’s new digital magazine now!