By Doris Lilly
Updated April 07, 1975 12:00 PM

From the beginning, Alfred (“Don’t call me Al”) Vanderbilt has had considerable luck with horses, and very little with women. When he was 19 he bought his first filly, Sue Jones, for $200, and since then has owned such money winners as Native Dancer and Social Outcast.

When Vanderbilt was 25 he took his first wife; at 32 he married his second; at 44 he gamely tried a third time. Then late last month it was made known in Manhattan Supreme Court that Vanderbilt had suffered another loss in the marriage derby. His wife’s lawyer filed a memo stating that the couple had reached a separation agreement, which presumably will lead to his third divorce.

Vanderbilt’s racing silks are cerise and white but, more properly, they should be Mint green. On his father’s side there was the New York Central: railroader Cornelius Vanderbilt died in 1877, the richest man in America, leaving $105 million. On his mother’s side was Bromo Seltzer. As a young man Vanderbilt made enthusiastic use of his inheritance. He dated a procession of handsomely shellacked glamor girls until in 1938 he made California socialite Manuela Hudson his bride. Her uncle owned Seabiscuit. Four years and one child later they were divorced, and Vanderbilt began making the rounds again. His mother, married four times herself, came to his defense when he was criticized for dallying. “He wouldn’t be a son of mine if he didn’t,” she snapped.

Vanderbilt’s subsequent wives were strikingly similar—sleek, slim and aristocratic. His 11-year marriage to his second wife, socialite Jeanne Murray (he courted her mostly at the Stork Club) ended in 1956 after two children. He married willowy, chestnut-haired Jean Harvey a year later—she was 20, a budding actress and heiress to the Harvey Restaurant system and Cudahy meat packing enterprises. Alfred lavished small attentions on his new wife, calling her each day from his office, bringing her daily gifts and, once, thoughtfully presenting her with a horse on her birthday.

Neither Jean nor the sometimes acidulous Alfred (he once described himself as “a Fred Allen without jokes”) was speaking last week about their legal sundering. Jean was consoling herself with their three children (ages 8 to 17) and doing nothing socially that might jeopardize what is expected to be a handsome settlement.

For some time now, the 62-year-old Vanderbilt has been seen, both professionally and socially, in the company of jockey Robyn Smith, who is 29, small and tough. Vanderbilt has reportedly been urging fellow stable owners to hire Smith to ride for them. Not long ago the two were together at the Saratoga annual horse sales and caused some aristocratic shudders when Robyn appeared in blue jeans and bare feet. “I’ve never seen her in a dress,” sniffed one horse owner. “And you should see them holding hands in the infield.”

Robyn wasn’t saying much either, although when asked what kind of horses she preferred, she answered without hesitation: “the older, more experienced ones.”