Jessica Gibson "had a change of personality when she was working for them," her sister tells PEOPLE

Credit: NBC; Kathy Hutchins/Hutchins Photo

The family of Rob Lowe’s former nanny describe the 24-year-old woman at the center of the nasty sexual harassment case as caring, cautious and concerned about the actor’s two sons.

“I was shocked when I heard about everything [and] I worry about her capability to cope with things,” says Jessica Gibson’s father Jim, 63, a technology executive.

In a lawsuit, Jessica Gibson claims Lowe repeatedly exposed himself and inappropriately touched her. Lowe counters that Gibson made up the allegations in an extortion scheme.

But Gibson’s father tells PEOPLE: “She felt being a nanny was perfect for her. She liked it.” The Lowes, he says, “were working her ridiculous hours” as Jessica looked after the couple’s sons, Matthew, 14, and John Owen, 12.

Loving and Discreet

Jessica’s mother, Eileen Gibson, a teacher who lives in Lake Elmo, Minn., finds it “absurd to say she tried to extort from the family. She is very compassionate, very caring and honest. She has a gift for working with kids. She loves them. She’s very gentle and kind and values her family very much.”

When it came to Jessica’s professional life, “She has been absolutely discreet about her job, and we did not discuss that,” says her mother, who is divorced from Jessica’s father.

Recalls Jessica’s older sister, Kelly Gibson, 28, “She was always the babysitter growing up. She’s Auntie Jessie to my daughter Alicia. I always told her she would be the best mother and wife.”

During her employment by the Lowes, “Jess had a change of personality when she was working for them and when she wasn’t,” says her sister. “She loved those boys. She looked out for them and worries about them. That was the hardest thing about leaving.”

For more on this story, pick up this week’s issue of PEOPLE, on newsstands Friday