The best cloth for men's clothes. So called from its great breadth. It required two weavers, side by side, to fling the shuttle across it. Originally two yards wide, now about fifty-four inches; but the word is now used to signify the best quality of (black) cloth.
Source: Dictionary of Phrase and Fable, E. Cobham Brewer, 1894
More on Broadcloth from Fact Monster: