Let's face the facts. Christmas is a pagan holiday. It's a pagan celebration so strong and so popular that it defies every effort to modify or suppress it. You can change the name and tack on all the baby Jesuses you like, but it's still the feast of the Winter Solstice, and most of the popular observations can be traced back to pagan origins. We've got the sacrificial tree of Wotan, the feasting of the Roman Saturnalia, the sacred libations of Dionysius, the Yule log that represents the burning corpse of the god of the dying sun, sacred rowan wreaths and mistletoe to ward off evil spirits, a Santa who's a rehashed pagan sky god, and plenty of monetary sacrifices to Mammon.
December 25th certainly isn't the day Jesus was born. That was likely much earlier in the year. The date was originally selected and made a holiday by the Romans as a consolidation of several pagan winter feasts, and then the early Church just just appropriated it, over some fairly strong objections, and those objections have continued ever since. It seems like the joyous spirit of Christmas is inherently incompatible with the dour righteousness of fundamentalism.
We hear a lot about the "War on Christmas" that secularists are waging, but the truth is that there are two wars on Christmas, with the religious right attacking it just as fiercely as the atheistic left. At the very least the Christians are at war to take the holiday away from the people who just want to celebrate their old paganistic traditions without having to worry about the larger implications. Most concerned Christians just want to "put the Christ back in Christmas" - never mind that there was precious little of him there in the first place. However, some of them want to go even farther and get rid of Christmas entirely.
In the late Middle Ages the Catholic scholar Polydore Virgil wrote against Christmas in de Rerum Inventoribus:
Dancing, masques, mummeries, stageplays, and other such Christmas disorders now in use with Christians, were derived from these Roman Saturnalian and Bacchanalian festivals; which should cause all pious Christians eternally to abominate them.
And opposition to the pagan traditions was high on the list of things the puritans wanted to purge from the Church. In the 1640s the puritan parliament tried to ban Christmas celebrations in England. Later that same century there was a similar ban in Massachusetts and Increase Mather expressed what were largely objections to the joyous nature of the holiday as being unchristian in his broadside A Testimony Against Several Profane and Superstitious Customs:
The generality of Christmas-keepers observe that Festival after such a manner as is highly dishonourable to the name of Christ. How few are there comparatively that spend those Holidays (as they are called) after an Holy manner. But they are consumed in Compotations, in Interludes, in playing at Cards, in Revellings, in excess of Wine, in mad Mirth; Will Christ the holy Son of God be pleased with such Services? Just after this manner were the Saturnalia of the Heathen celebrated. Saturn was the Gaming God. And the Feast of Christ’s Nativity is attended with such Profaneness, as that it deserves the name of Saturn’s Mass, or of Bacchus his Mass, or if you will, the Devil’s Mass, rather than to have the Holy name of Christ put upon it.
Mather clearly reaffirms the popular perception that Puritans really objected to people having any fun at all.