To celebrate the arrival of her newest grandchild, Caitlyn Jenner also shared a sweet throwback photo of daughter Kylie Jenner when she was a baby

Caitlyn Jenner has a new grandchild in her life!

The star, 68, celebrated the arrival of daughter Kylie Jenner‘s baby girl on Instagram Monday with a throwback photo of the Keeping Up with the Kardashians reality star, 20, when she was just an infant.

“My daughter just had a daughter. It’s amazing to be by your side through this journey. She’s SO beautiful already. Can’t wait to watch her grow. Throwback to my baby, as a baby @kyliejenner,” Jenner captioned the picture.

The new mom, who welcomed her first child with rapper boyfriend Travis Scott on Feb. 1, expressed thanks for the sweet message by commenting, “Love you.”

Though Kylie has yet to share a photo of the newborn, a source told PEOPLE Monday that the baby “is very cute with chubby cheeks.”

Caitlyn’s Instagram post follows congratulatory messages shared by Kylie’s sisters on Sunday, the same day the Lip Kit entrepreneur announced her baby’s arrival.

Want all the latest pregnancy and birth announcements, plus celebrity mom blogs? Click here to get those and more in the PEOPLE Babies newsletter.

Kylie Jenner CaitlynCredit: Caitlyn Jenner/Instagram
Caitlyn’s Jenner Instagram post on Monday
| Credit: Caitlyn Jenner/Instagram

“Cute mommy,” Khloé Kardashian, who is currently expecting her first child with NBA star boyfriend Tristan Thompson, said while retweeting a video sweetly titled “To Our Daughter” that documented Kylie’s pregnancy.

Kendall Jenner and Kim Kardashian West, who welcomed her third child, a girl with husband Kanye West via surrogate early this year, shared the youngest KarJenner sibling’s tweet.

Momager Kris Jenner was also thrilled to welcome another grandchild, writing, “God is SO good!! I am so excited and thrilled to welcome another precious baby girl to our beautiful, ever growing family. We are blessed beyond belief!”

Caitlyn and Kylie Jenner in November 2015
| Credit: Nicholas Hunt/Getty

Also on Monday, Kardashian West, 37, also shared Kylie’s baby news on her website, writing, “Mommy Kylie!!! You did it!!! I’m so incredibly proud of the woman you have grown up to be and I know that you are going to be the best mom to your new baby girl!”

Adding, “Our girls are two weeks apart and I can’t wait for them to be besties! I am so proud that you do what you want to do and live your life the way you want, you will pass down all of those amazing qualities to your baby girl. You were made to be a mom! I love you!”

WATCH: Pregnant Kylie Jenner Has a Baby Name Picked Out for Her Daughter

Back in September, when multiple sources first confirmed to PEOPLE the news that Kylie was expecting a girl, an insider close to the family said Caitlyn was ready to support her child through first-time motherhood.

“She’s ready to be there for Kylie and will be a big part of the baby’s life,” the insider shared with PEOPLE. “She’s happy about all the babies coming into the family,” the insider also added about Kim and Khloé’s new additions.