The custom of turning to the east when the creed is repeated is to express the belief that Christ is the Day-spring and Sun of Righteousness. The altar is placed at the east end of the church to remind us of Christ, the “Day-spring” and “Resurrection”; and persons are buried with their feet to the east to signify that they died in the hope of the Resurrection.
The ancient Greeks always buried their dead with the face upwards, looking towards heaven; and the feet turned to the east or the rising sun, to indicate that the deceased was on his way to Elysium, and not to the region of night or the inferno. (Diogenés Laertius: Life of Solon, in Greek.)
Source: Dictionary of Phrase and Fable, E. Cobham Brewer, 1894
More on East from Fact Monster: