Kelly Rutherford Says Ex Won't Let Her See Kids Until She Turns Over Their Passports

Rutherford has been fighting to bring her two children back to the United States from Europe

Photo: Cindy Ord/Getty; Rukhsana Hamid/Bloomberg News

Since 2012, Kelly Rutherford‘s two children have been living with their father, Daniel Giersch, in Europe. The exes technically share custody, according to legal papers, yet she flies back and forth for visits based on a court-mandated parenting plan.

But the Gossip Girl star tells PEOPLE that Giersch would not allow her to see their kids after arriving in France for her latest trip Thursday – and claims that the German businessman won’t let her be with them unless she turns over their U.S. passports.

His attorney, Fahi Takesh Hallin, tells PEOPLE Giersch is asking Rutherford to give the passports to a third party.

“Any statement to the public suggesting that if someone were to bring the children back to America (which is in violation of the custody orders and criminal) that he or she would be a hero, is unacceptable on many levels. It creates a very real and present risk to the children’s safety and well-being that must be taken seriously,” Hallin says in a statement. “As a result of such statements by Kelly, Daniel has had to take necessary precautions in accordance with the parties’ Judgment and custody orders, which includes requesting that the children’s passports be held by a neutral person when the children are not traveling.

“Daniel has not requested that the children’s passports be turned over to him, and that is a false claim. Daniel has always been and will continue to be in compliance with the Court Orders and ensure the children s safety and well-being.”

But Rutherford claims Giersch asked her to give them to another one of his attorneys. According to Rutherford, this person is someone she does not know and has not been approved by the court, and the exes failed to agree on a neutral party for this purpose years ago.

The couple’s son Hermes, 8, and daughter Helena, 5, moved with Giersch to France and Monaco after a California judge allowed him to relocate because his U.S. work visa had been revoked. According to court papers, a former lawyer on Rutherford’s team caused this by reporting him to the State Department; she has steadfastly denied any involvement. Giersch was supposed to apply for a new visa, but the State Department confirms he has not yet done so.

“Where is his visa? Or efforts to get one? That was the point of all of this,” Rutherford tells PEOPLE. “I have only been doing press to get help from my country. How else will I ever see my kids? We are all U.S. citizens.”

Giersch’s team is firing back. Hallin released a statement earlier Thursday, speaking out against the actress’s public cry for help in bringing her children back to the United States.

“Daniel’s goal is to continue to protect these children and shield them from any media exposure, which is against court order,” Hallin said. “We are taking Kelly’s statements to the media very seriously.”

“Kelly’s statement that Daniel was deported from the U.S. is false and should not be used to entice danger to the parties’ children,” she added. “Daniel has taken and will continue to take all necessary security measures to ensure that the children are not wrongfully removed from Europe and remain safe, healthy, happy and protected from any such threats to their safety and well-being.”

Rutherford, 46, appealed the California judge’s decision in New York last month and lost, but she has told PEOPLE she plans to keep on fighting. Last week, her petition asking the Obama administration to intervene reached the 100,000 signatures it needed to merit a response after celebrities like Kim Kardashian West and Melanie Griffith voiced their support.

“I’ve always wanted my kids to have a relationship with him, and I’ve really only encouraged that all along. When I went into court, I never asked for money, and I never asked for full custody,” Rutherford told PEOPLE in April. “I just wanted what was right for them and their well-being moving forward. And what happened over the next six years was the most surreal, bizarre, unimaginable thing.”

ABC legal analyst and Mediaite founder Dan Abrams – who has written articles in support of Rutherford’s cause – tells PEOPLE that if Giersch is really refusing to let Rutherford see the kids, it may be the evidence she needs to make legal headway.

“If true, this is not just disturbing but could – and in my view should – become the turning point that leads a California court to say it’s time to have these American children sent back home,” he says. “There is no question that Daniel has no legal right to prevent Kelly from seeing their children under either the California or Monaco court rulings.

“It is not for him or his lawyer to decide that they are suddenly so concerned about her somehow whisking the children out of France that they will refuse her access to the kids unless she turns over their American passports. If he wants to get the visitation order amended, he would have to go to court to do so, but I can’t imagine any court would allow him to prevent her from having at least the access to the children that has already been ordered.”

RELATED VIDEO: After Parents’ Deaths, 18-Year-Old Fighting for Custody of Younger Siblings

As for Giersch’s attorney’s statement, Abrams acknowledges that Rutherford could face ramifications for talking to the media.

“The risk Kelly runs by speaking out about her case is that she could say something that is used against her in court,” he explains. “On the other hand, by not speaking out about it, it’s harder to see how the State Department or any other political body gets involved in what, in my view, is a miscarriage of justice.”