Much like her This Is Us alter ego, Kate Pearson, Chrissy Metz is no stranger to bullying

Much like her This Is Us alter ego, Kate Pearson, Chrissy Metz is no stranger to bullying.

In her new memoir, This Is Me, the 37-year-old actress opens up about her painful history of being ridiculed and harassed, judged throughout her childhood and teens for her weight.

“By fifth or sixth grade, I started becoming aware of being different in the sense of my size,” the star tells PEOPLE in this week’s cover story. “I started to examine everybody else’s body and my own and pick myself apart. …I was wanting to wear a T-shirt when we would go swimming because I didn’t want to expose myself. Emotionally, I was so down on myself.”

F:PHOTOReady RoomActionsInsert Take48799A1#Yu TsaiPeople_ChrissyMetz_01_0125_C copy.jpg
Chrissy Metz
| Credit: Yu Tsai

When it came to boys, “they would flirt,” says Metz, “but they’d never want to be in a relationship or date me because I was the chubby girl.”

Perhaps the most heartbreaking treatment came from the first boy Metz fell for, Derek, brother of her friend Mya.

“Let me tell you, my first kiss with Derek was — to this day! — one of the most amazing kisses, if not the best of my life,” writes Metz in her book. “The next time we were all at Mya’s, I waited for him to acknowledge me in a real way, not just through his secret signals. When he didn’t, I decided this was what I needed to settle for — a wink here and there. He couldn’t let anyone know he liked the fat girl. Got it.”

  • For more from Metz, pick up this week’s issue of PEOPLE, on newsstands Friday

“If I got too comfortable around him in front of our friends — meaning I said, ‘Hi, Derek’ — he curled that perfect upper lip and looked away, exhaling in disbelief and disgust,” she adds. “The night before, he’d kissed me and told me I was beautiful.”

Says Metz: “When you’re 14 years old, it’s devastation. Your world is upside down.” But now, Metz wonders, “How in the world did I put up with that behavior?”

RELATED VIDEO: Chrissy Metz Says Her Stepfather Beat Her and Forced Her to Do Humiliating Weigh-Ins as a Teen

Finally happy in her own skin, Metz says she’s learned valuable lessons about self-acceptance, and what she’ll no longer tolerate when it comes to relationships.

“I don’t need to beg somebody to like me, to call me, to text me,” she says. “I’m your girl or I’m not. I know what I want now and there’s no reason to settle for anything less.”