I am going down town, i.e. to the business part of the town.
Down the country
properly means down the slope of the land, or as the rivers run. We
say “I am going up to town” when we mean out of the country into the
Source: Dictionary of Phrase and Fable, E. Cobham Brewer, 1894
More on Down Town from Fact Monster:
- Samuel McMILLAN - McMILLAN, Samuel (1850—1924) McMILLAN, Samuel, a Representative from New York; born in County ...