In 1953, he was sentenced to 15 months in prison for owing back taxes and was forced to sell the team. Although he received offers of $4.5 million from other cities, he instead sold the team to August Busch Jr., chairman of Anheuser-Busch, for $3.75 million in order to keep the Cardinals in St. Louis. After selling the Cardinals, Saigh began buying stock in Anheuser-Busch until he owned more stock than any other person outside of the Busch family. However, he always resented the fact that, to the people of St. Louis, August Busch was portrayed as the man who saved the Cardinals rather that Fred Saigh as the man who sold them for a lesser amount to keep them here. He never attended a Cardinal game while the Busch family was involved in its ownership.

Fred Saigh never married and died a very wealthy man at the age of 94. {"Fred Saigh Jr., Former Owner of the Cardinals, Savvy Investor, Dies at 94", St. Louis Post-Dispatch, December 30, 1999, http://thedeadballera.crosswinds.net/Obits/SighFredsObit.html].