David says (Ps. li. 7): “Purge me with hyssop, and I shall be clean.” The reference is to the custom of someone who was ceremoniously “clean” sprinkling the unclean (when they came to present themselves in the Temple) with a bunch of hyssop dipped in water, in which had been mixed the ashes of a red heifer. This was done as they left the Court of the Gentiles to enter the Court of the women (Numbers xix. 17).
Source: Dictionary of Phrase and Fable, E. Cobham Brewer, 1894
More on Hyssop from Fact Monster: