One thing to consider is why we celebrate the eve, rather than the day. Eves are also celebrated for May Day and for Christmas, although in those cases the day too is important. I remember many years ago reading (and believing at the time) that Halloween represents a festival for evil before it's banished by the holy day.
This is a load of nonsense, of course. Night-time festivals like Samhain certainly represent the dark part of the light-dark cycle, but the idea of identifying dark with evil is relatively modern. To our ancient ancestors, light and dark were simply two essential sides of the same cycle. Dark had its dangers, to be sure, but that didn't make it evil. It was also the time for hunting.
The actual explanation is that many ancient calendar systems (most notably today the Jewish calendar) count the day as running from sunset to sunset. This means that the activities of the night of 31st October start the festival that continues into the 1st November.
So is Samhain really a Celtic festival? Well, that depends what you mean by Celtic. Properly speaking, the term refers to a family of languages within the Indo-European group, and by extension to the ancient tribes who spoke them. It's applied, somewhat inaccurately, to the inhabitants of the countries or regions where Celtic languages are still spoken — Ireland, Scotland, Wales, Brittany — but in reality these are all racially and culturally mixed countries.
The Celtic languages and the associated cultural elements were the last of a series of waves to sweep over western Europe and into Britain before the Roman conquest. Or, possibly, the last three waves, representing the Goidelic (Gaelic), Brythonic (British) and Belgic groups, although the actual relationships between these three isn't clear.
In any case, the Celtic languages arrived in Britain during the 1st millennium BC, replacing whatever languages were spoken before. Nothing of these has survived, and speculation that some may have been related to Basque, the odd-one-out of European languages, are no more than an intelligent guess. It used to be believed that the Pictish language of north-east Scotland, which survived till well after Roman times, was one of these, but the prevailing view now is that it was probably Celtic in origin.
Nevertheless, this doesn't mean that the older populations, or their cultures, simply disappeared. The old model was of successive waves slaughtering, driving out or enslaving their predecessors, as European colonialism has done so efficiently in recent centuries. However, DNA profiling has shown that, aside from obvious recent arrivals, most people in Britain are actually descended from the hunter-gatherers who followed the retreating ice northwards. On the matrilineal side, at least, which is easier to trace.
This suggests that all these new arrivals, including the Angles, Saxons and Jutes, may well have been relatively small numbers, mainly male — perhaps restless young men — who bred with native women. They might sometimes have become leaders, or acquired the cool factor in some other way, and begun influencing both the culture or the language, in the same way that most of the "Romans" in Britain were actually Britons who'd adopted Roman language, dress and customs.
The older customs would have survived, though, alongside the new imports, just as both English and the modern Celtic languages almost certainly include elements from the unknown ancient languages. It's convenient to refer to Samhain, Beltane and the rest as Celtic, but there's no reason to suppose they don't go back thousands of years before the Celtic influence began to be felt.
So what was Samhain? Well, for one thing, it was the ancient New Year's Day, marking the official beginning of winter in the agricultural calendar. Other elements have to be largely deduced from later folk traditions, but two elements seen paramount: burning bonfires, often with a human effigy on them, and the presence of spirits, ghosts and the otherworld.
There are suggestions that Samhain may have been the occasion of human sacrifice. In the ballad of Tam Lin — one of the masterpieces of the ballad tradition — the eponymous hero has been kidnapped by the faerie queen and is growing distinctly nervous as Halloween approaches, since at the end of seven years she pays a tithe to Hell, and I so fair and full of flesh am feared it be myself. Although we don't get to see how exactly she pays her tithe — Tam Lin is, of course, rescued by his True Love — this seems to indicate a ritual sacrifice, perhaps of the Summer King to be replaced by the Winter King.
Or maybe not. It's notoriously hard to be sure whether to take descriptions of religious rituals literally or symbolically, since adherents tend to talk of them in literal terms regardless. One of the major prejudices against Christianity in the Roman world was that it was a cannibalistic religion, since the rituals involved eating the flesh and drinking the blood of its dead leader. If we step away from what everyone in the modern western world knows instinctively, whether or not they're Christian, it's easy to see how such a misunderstanding could be possible.
In the same way, the sacrifice might always have been a symbolic one, using an effigy to represent the dying god. That's pure speculation, of course, and both Roman writers and archaeology suggest that the Britons performed human sacrifice. On the other hand, the Romans had a vested interest in presenting the conquered people as barbarous, while the finds are inconclusive. A handful of pre-Roman human remains have been unearthed that appear to have been ritually killed, but very few, and it's not clear from the context whether these were sacrificed or ritually executed — for sacrilege, for instance.
As for the otherworldly aspects, the folk traditions strongly suggest that Samhain was a festival of the dead. Archaeological finds show that prehistoric homes often had bodies buried under the floor, suggesting that dead ancestors were seen as still present. Samhain seems to have been a time when your ancestors might well visit you and expect a decent welcome. This wouldn't necessarily have been something to fear, but it must, at best, have been spooky.
In any case, a night when the dead could return implied a night when all kinds of beings could get through the veil into our world, and not all of them would be friendly. From the fae-folk to malevolent demons, they had to be discouraged, scared off or placated, and many Halloween traditions, from the fancy-dress to the scary stories, may have originated for this purpose.
The early Church simply appropriated Samhain, as it appropriated many pagan festivals and pagan sacred sites, eventually reserving it only for the most exalted of the honoured dead (All Saints) and shunting the ordinary folk (All Souls) to the following day. Beyond that, little was done to discourage the pagan traditions. With the exception of occasional bouts of zeal, the mediaeval Church tended only to care that the peasants attended Mass on Sundays and major saints' days. As long as they fulfilled their obligations, they could get up to whatever they chose in the woods or on the hill above the village.
The puritans, though, took a dim view of a celebration that was half pagan, half popish. After 1605, the perfect excuse was presented to hijack Halloween when the failure of the Catholic Gunpowder Plot began to be celebrated in a suspiciously similar way on its anniversary, 5th November. The figure of special hatred became Guido (Guy) Fawkes, although he was actually only the hired gunpowder expert. The plot's leader, Robert Catesby, and his cronies were mostly midlands gentry — several were related to Shakespeare on his mother's side. Nevertheless, Guy Fakwes became the one whose effigy was burnt on bonfires all over the country, although sometimes it was the Pope.
This was close enough in date to take over from the fires of Samhain, and many Halloween traditions died out, though they lasted longer in villages and more out-of-the-way parts of the country. When I was a child, in the 1960s, Halloween was scarcely a blip in the UK, except that there might be a ghost story on TV. Instead of trick or treating, we went around a few days later with the "guy" that was to be burnt — this was actually the origin of the word guy meaning a man — asking for "a penny for the guy".
I say "we" — my parents didn't approve of this, since they considered it begging. We had a bonfire — technically a balefire, since a bonfire should burn bones — and fireworks, but there was only one year when we made a guy. We didn't have the heart to burn him, though, and propped him up against the house to watch the fireworks. It's probably just a child's imagination, but I'm sure he looked relieved.
One tradition that survived for a long time in some villages was Souling, in which the children of the village went round to each house giving it blessings and good luck for the next year in return for specially baked soul cakes and other goodies. A surviving Souling Song gives the wish-list:
A soul a soul a soul cakePlease good missus a soul cake
An apple or pear
A plum or a cherry
Or any good thing to make us merry
Not quite candy, but it still sounds familiar.
Whether it was descended precisely from this or from a similar tradition, the custom seems to have survived and flourished in America, turning into trick or treating. I don't know where the trick aspect came from, but I suspect it was a graphic illustration of the consequences of not accepting the blessing. Or maybe it was actually a kindness: getting the bad luck out of the way quickly and ritualistically.
When I was a kid, trick or treating was barely known in the UK. We became gradually familiar with it through American films and TV, and I suspect it was after ET that it began to take off — I seem to recall that it was the 80s when I began noticing children doing it. Even now, though, it's much more random and disorganised than in the US.
It's a long way from human sacrifice to candy, but the joy of true traditions is their variety and adaptability. Whether you dress up in scary costumes, put on a horror film, go the church to give thanks for the saints, or head off to the woods to reconstruct a pagan ceremony, tonight has something for everyone.