Credit: John Shearer/WireImage

Wisteria Lane is losing one of its longtime residents.

Shawn Pyfrom, 22, who plays Desperate Housewives‘ Andrew Van De Kamp, the troubled son of Marcia Cross‘s character, won’t be coming back full-time, PEOPLE has learned exclusively.

“Shawn’s not returning for a full season next fall,” a source tells PEOPLE. “But he’ll come back and make a few guest star appearances.”

The source says it was a tough call for the actor to make. “His decision didn’t come easy,” the source says, “but he wants to pursue new television and film opportunities.”

Desperate Housewives returns to ABC on Sept. 27. –David CaplanJohn Shearer/WireImage