"I needed to get knocked down and to build up a thick skin," Kelly told Seventeen of competing on American Idol

By Jeff Nelson
January 05, 2016 11:45 AM
Tom Schirmacher/Seventeen

Before Tori Kelly was a Best New Artist Grammy nominee, she was a contestant on American Idol – and prone to the wrath of Simon Cowell‘s blunt critique.

The “Hollow” singer competed on the ninth season of the FOX talent competition when she was 16, and Cowell once dubbed her voice “almost annoying” before she was eliminated.

Kelly, now 23, didn’t take the British label exec’s comments to heart too much, though – because they actually fueled her debut album.

“Deep down, I knew I was good, and I knew I could do it. I needed to get knocked down and to build up a thick skin,” the performer – one of PEOPLE’s 2015 Ones to Watch – said in the February issue of Seventeen. “Who knows what I would have been singing about otherwise? You have to go through things in order to inspire others with your story.”

And though she’s up for one of the top awards at music’s biggest night, she’s still faced people in the industry trying to change her, including her manager who once called her boring.

Tori Kelly
Tom Schirmacher/Seventeen

“He told me, ‘People are saying you are vanilla and that you have nothing to say.’ I was like, Whoa,” she told the glossy. “I remembered record-label guys telling me I was really reserved when I was younger. Like, ‘Wake up, you should be bubbly and happy!’ And I was thinking, But this is my personality; I’m just chilling.”

Turns out, that interaction inspired her songwriting, too.

“So I went right back to being that little girl who was just so freaked out,” she added. “I acted as if I didn’t care, but I went home that night and just poured all the feelings that I was holding back into my music. That’s how ‘Unbreakable Smile‘ came to be.”