(The). The vest of penitence. It was a coarse yellow tunic worn by persons condemned to death by the Inquisition on their way to the auto da fé; it was painted over with flames, demons, etc. In the case of those who expressed repentance for their errors, the flames were directed downwards. Penitents who had been taken before the Inquisition had to wear this badge for a stated period. Those worn by Jews, sorcerers, and renegades bore a St. Andrew's cross in red on back and front.
Source: Dictionary of Phrase and Fable, E. Cobham Brewer, 1894
More on San Benito from Fact Monster: