"In hindsight, I should not have done that," says Ballerini of going a little too far trying to meet Kelly Clarkson

Credit: Imeh Akpanudosen/Getty; Cindy Ord/Getty

Kelsea Ballerini is a rising star in country music – she made history when her debut “Love Me Like You Mean It” topped the charts in June and she’s nominated for Female Vocalist of the Year at the CMA Awards on Nov. 4. She’s also a bit of a super fan.

Kicking off the Rock the Room Tour! benefitting Musicians on Call on Wednesday night in Nashville, the singer shared that she may have crossed a line the first time she met Kelly Clarkson.

Kelly Clarkson is my favorite artist of all time,” explains Ballerini, 22. “I actually stalked her after a show and knocked on the door of her tour bus. She came out and was very kind. In hindsight, I should not have done that. Hopefully, I get to meet her again tonight and make a better second impression.”

Though Clarkson was on the night’s bill, the pregnant star is on vocal rest and could not attend. The show did go on in fine form with Martina McBride and Reba McEntire as headliners.

They each delivered a song with messages particularly relevant to the cause: McBride’s “I’m Going To Love You Through It” and McEntire’s “I’m a Survivor.” McEntire also performed one of her first hits – “Somebody Should Leave” – joking that she hoped the audience liked old country, because “I’m old and I’m country.”

Musicians On Call brings live music to the bedsides of people in nursing homes, hospitals and hospice facilities. Nashville‘s Charles Esten described why it was such an important cause.

“Our daughter was diagnosed with leukemia when she was just 2 years old,” said Esten. “One night she was in the hospital and in such pain, she was inconsolable. I was desperate to help her and went to my iPod and found ‘In the Arms of an Angel’ and by the time it was over, she was asleep on my chest. I knew then the power of music. Addie is now 16 and I am a proud and new member of Musicians On Call.”

The Rock the Room Tour! will also make stops in Los Angeles, New York and Washington, DC.

Don’t miss a beat of country music news, photos and videos! Click here to get all this and more in the PEOPLE Country Newsletter.