All our numerals and ordinals up to a million (with one exception) are Anglo-Saxon. The one exception is the word Second, which is French. The Anglo-Saxon word was other, as First, Other, Third, etc. Million is the Latin millio (-onis).
There are some other odd exceptions in the language: Spring, summer, and winter are native words, but autumn is Latin. The days of the week are native words, but the names of the months are Latin. We have daeg, monath, gear; but minute is Latin, and hour is Latin through the French.
Source: Dictionary of Phrase and Fable, E. Cobham Brewer, 1894
More on Numerals from Fact Monster: