While rooted in the past (see Halloween History), Halloween as we know it today dates back to the early 20th century.
The holiday was relatively obscure in late 19th century America. It was brought to the country by Irish and Scottish immigrants, combining the features of the Celtic and Christian holidays, and celebrated with feasting, divinations, and mischief making.
People had been carving gourds or pumpkins and using them as lanterns long before this practice was associated with Halloween. In 1850, for example, poet John Greenleaf Whittier mentioned the practice of his boyhood in "The Pumpkin": "When wild, ugly faces we carved in its skin, / Glaring out through the dark with a candle within!"
We don't know exactly when and why these lanterns became associated with Halloween in particular, though we do know it was in North America. But by the start of the 20th century, the connection was firmly established.
The name "Jack-o'-lantern" has changed in meaning several times. It was first recorded as a nickname for a night watchman, dating back to 1663. Around the same time, it was used as another name for a will-o'-the-wisp. It began to be used for pumpkin lanterns sometime in the 19th century. It's possible that the name simply went from the night watchman (a man holding a lantern) to the lantern itself.
On the other hand, an Irish legend tells of a miserly man named Jack who, while alive, tricked the Devil into agreeing not to take him into Hell. Upon his death, St. Peter wouldn't let Jack into Heaven, because he had been too stingy and sinful. The Devil wouldn't let him into Hell because of the deal they'd made. Jack was condemned to wander between Heaven and Hell with his lantern, looking for a place to rest and never finding one.
In late 18th century America, Halloween was a night for mischief and pranks. Boys would make "tick-tacks," cutting notches in the ends of a wooden spool and winding string around it. The spool would be placed right up against a window, with a nail serving as an axle. When the string was pulled, it made a loud and rapid "tick-tack" noise. Other noisy and startling practices involved throwing corn and decaying vegetables at houses.
While this was considered innocent fun, some pranksters began to go too far, especially with the move from country life to city life. News stories tell of students being expelled from schools, gangs of youths roaming through town covering people in flour, buildings being blown up, and so on.
The general practice of going door-to-door for treats is clearly similar to a much older practice, "souling," in which the poor would go from house to house begging for alms or food. However, the specific practice of "trick-or-treating" dates to around the 1930s. It is possible—though by no means certain—that it evolved as an antidote for the increasingly rowdy and costly Halloween pranks. It provided a healthier activity for the young and gave them an incentive not to play tricks.
We could easily make something up about the origins of Halloween costumes. We could say that people originally dressed as ghosts and witches to scare people, and that the practice eventually spread to include costumes of all sorts. But the fact is that we don't actually know where Halloween costumes came from, only that the practice, like trick-or-treating, appears to have begun in early 20th century America.