There are mathematical techniques intended to counter over-fitting in data. Imagine a stockmarket analysis program. If it's trained on a year of data when, that year on June 23, all tech stocks dropped, it might start to recommend selling (or shorting) all its tech stocks just prior to June 23 every year. Obviously this isn't smart; the program has overfitted. What you might do is take the training data and slice it up and restack it in some new way - different sectors, different months - and the program might start finding more meaningful relationships.
This is analogous to dreaming, and has been rediscovered (or re-engineered, if you prefer) in other settings for some years now. We still don't really know why we sleep, let alone why we dream. But if we assume that brains are doing the same thing - avoiding overfitting - the explanation makes sense, and is consistent with the characteristics of dreams. Suddenly you're at the beach you used to go to every summer as a kid, but then in your school; your deceased grandmother is there, at the same time as your asshole coworker from the last company, and then you're driving down a steep mountain road with no brakes. Obviously they never met, and they especially didn't meet in some bizarre hybrid beach-school-mountain place. By mixing them, you're trying to avoid overfitting. In contrast, if you have PTSD, you do dream literally the same concrete traumatic experience over and over - and your waking behavior is overfitted - you avoid trucks under overpasses, or that one street corner, or movies about fire, based on whatever experience you had that you can't "digest", integrate with the rest of your life's experience, and move past.
Returning to computers, Andrej Karpathy on Twitter says "...in some sense, hallucination is all LLMs do. They are dream machines. We direct their dreams with prompts. The prompts start the dream, and based on the LLM's hazy recollection of its training documents, most of the time the result goes someplace useful." Similarly, the data stored and sliced and diced in our dreams influences how we perceive the world. It provides the top-down filter for the bottom-up sense data pouring in.[1] Without this, we veer off into hallucinations and delusions within days.[2]
But sometimes we humans' dreams intrude into our waking hours, drawing attention to themselves in full form, like stars briefly visible in the daytime if you know where to look[3], or if you like, a laundry dryer opened mid-cycle to let a sock or a shirt fly out. In the shower, you start laughing when a joke you heard in eighth grade pops out of nowhere. While driving to work somehow you're suddenly thinking of your recently deceased cat, and you're sad.
It seems to me that fiction is a more elaborated version of this, committed to a less ephemeral form, one that produces fossils of our overfitting-avoidance. Something has to explain the reason that we write stories that never actually happened, that we know no one will ever think actually happened, and that we (mostly) never even show to anyone else. (Out of all the short stories ever completed in history, what percentage of them was even intended for publication, let alone did the author think had a real chance of being read by someone else?) So why do we bother? Why are we so compelled? We're avoiding overfitting the things that are important to us. We write character-driven stories examining the psychology of people similar to those we know, or we write alternate history what-ifs about events that we find interesting - all of which we are trying to understand better and connect to our other experience, as we turn it over, and slice and dice it and rub it up against our other experiences.. Even the writing process itself is consistent with this - the ideas somehow just appear automatically, along with some scenes and images and events, that the writer has to organize (often laboriously, decidedly non-automatically) into a coherent narrative.
[1] In psychosis, the top-down part of the process dominates and you're lost in a waking dream of hallucinations and delusions. Autism is sometimes thought of as a diametric opposite to psychosis, when the bottom-up sense data dominates and unfiltered and the overwhelming cacophony of sounds, lights, or textures become intolerable.
[2] Waters F, Chiu Vivian, Atkinson A, Blom JD. Severe Sleep Deprivation Causes Hallucinations and a Gradual Progression Toward Psychosis With Increasing Time Awake. Front Psychiatry. 2018; 9: 303.
[3] When Venus is on the same side of the Sun as Earth, you can actually see it with the naked eye during the day. When I first located it in a blue daytime sky, I found this almost disorienting.
Sunday, January 7, 2024
Thursday, January 4, 2024
A Brief Sketch of History: Subdivisions of the Iron Age
Large sedentary civilizations emerged where groups of people were forced to, and then rewarded by, central organization of labor - often in marginal environments (dry river valleys requiring irrigation, or in the special case that generalizes the rule, rice farming.) Because of the more rapid diffusion of ideas, the Silk Road regions (Asia, Middle East, Europe, North Africa) were advantaged over the rest of the world. (Hence, the description here is focused on the Silk Road macro-region.) Europe, in turn, was advantaged over China because geography predisposed the formation of multiple small states which acted like incubators for cultural selection; and Greece was advantaged over the rest of Europe given its peninsulas, mountains, and islands. This arrangement still allowed eventual cultural diffusion, overtaking China only with the printing press. The other factor which allowed Europe's situation to obtain was the separation of religious/moral authority and secular authority - a Pope, and kings (as opposed to the unity of these institutions throughout much of Islamic history, and the relative marginalization of religion in East Asian history.)
States demonstrate a natural cycle of 200-250 years. Even if a nation by the same name, on the same territory, with the same people, lasts longer than this, typically there is a transition period. The natural experiment of a large state on a fertile plain showing this pattern is China, but Rome also demonstrated the Principate and Dominate periods, the Ottomans had a similar pattern, and it can be seen elsewhere as well. In these rough divisions I am focusing on the Silk Road regions of the Old World - China, the Middle East, North Africa and and Europe.
THE IRON AGE: 1200 BCE-1800 CE
Early Iron Age: 1200 BCE-500 CE. Began in Europe with the Bronze Age Collapse, saw the rise of multiethnic administrative empires and coinage, and thus the Axial Age. In China, this contains the end of the Shang Dynasty, as well as the rise and fall of the Zhou and Han Dynasties. In Europe this can be further divided into Early Iron Age 1 (1200-600 BCE) featuring palace economies, and Early Iron Age 2 with the later development of oligarchic rule and early market economies.
Middle Iron Age: 500-1500 CE. Roughly co-extensive with the medieval period. in Europe this begins with collapse of Western Roman Empire, the weakening of the Eastern Roman Empire, the collapse of Persia and the spread of Islam. In China, it begins earlier with the end of the Han Dynasty. The Middle Iron Age is characterized by the spread of supra-ethnic philosophies and the dissolution of large empires, which can be thought of as an ecological model of cooperation within empires no longer exceeding competition within empires; oligarchies quarelled amongst themselves, and social or ethnic outsider groups benefiting from cultural diffusion (Germanic tribes in Europe, or the Yellow Turban Rebellion in China.) This period is marked by states and peoples developing a sense of identity if not patriotism, and especially by nomads occasionally overwhelming established states, with the Mongols as the military high water mark of nomads in history, with their decline signaling the end of this period.
Late Iron Age: 1500-1800 CE. In the West this contains the Renaissance, Protestant Reformation, Age of Discovery and Enlightenment. In Russia, it starts with the Great Stand on the Ugra and eastward expansion. In China, it's the parallel end of the Yuan Dyansty. Ultimately it begins with the spread of gunpowder, as seen in the gunpowder kingdoms, as well as the printing press, which had its greatest impact in Europe both for the good (Europe's domination of the world starting in this period) and the bad (religious civil wars as Northern Europe could communicate more easily.) At this time, the technological advantage of sedentary societies began to overwhelm that of nomads. Simultaneously, the benefit of technological innovation in the crucible of a sort of geographically-enforced natural federalism in Europe allowed Europe to outstrip China and colonize the world. The use of gunpowder as a source of energy more powerful than human or animal muscle anticipates the Industrial Age. Like the Mongols, Napoleon was the high water mark of Iron Age warfare, and was ultimately undone by the home of the Industrial Age, the United Kingdom.
Though not the focus of the post, the Industrial Age comes with its own subdivisions: the first wave in 1790-1830 with steam and water power leading to factories, materials extraction and textiles, an interim with three "transition wars" in the West (the Crimean War, American Civil War, and Franco-Prussian War; the Taiping Rebellion still appeared very much like a late Iron Age religious War, like a Thirty Years War compressed into half the time) with the second industrial revolution 1870-1910 converting industrial power into consumer goods; this culminated in World War I, the first industrialized war.
States demonstrate a natural cycle of 200-250 years. Even if a nation by the same name, on the same territory, with the same people, lasts longer than this, typically there is a transition period. The natural experiment of a large state on a fertile plain showing this pattern is China, but Rome also demonstrated the Principate and Dominate periods, the Ottomans had a similar pattern, and it can be seen elsewhere as well. In these rough divisions I am focusing on the Silk Road regions of the Old World - China, the Middle East, North Africa and and Europe.
THE IRON AGE: 1200 BCE-1800 CE
Early Iron Age: 1200 BCE-500 CE. Began in Europe with the Bronze Age Collapse, saw the rise of multiethnic administrative empires and coinage, and thus the Axial Age. In China, this contains the end of the Shang Dynasty, as well as the rise and fall of the Zhou and Han Dynasties. In Europe this can be further divided into Early Iron Age 1 (1200-600 BCE) featuring palace economies, and Early Iron Age 2 with the later development of oligarchic rule and early market economies.
Middle Iron Age: 500-1500 CE. Roughly co-extensive with the medieval period. in Europe this begins with collapse of Western Roman Empire, the weakening of the Eastern Roman Empire, the collapse of Persia and the spread of Islam. In China, it begins earlier with the end of the Han Dynasty. The Middle Iron Age is characterized by the spread of supra-ethnic philosophies and the dissolution of large empires, which can be thought of as an ecological model of cooperation within empires no longer exceeding competition within empires; oligarchies quarelled amongst themselves, and social or ethnic outsider groups benefiting from cultural diffusion (Germanic tribes in Europe, or the Yellow Turban Rebellion in China.) This period is marked by states and peoples developing a sense of identity if not patriotism, and especially by nomads occasionally overwhelming established states, with the Mongols as the military high water mark of nomads in history, with their decline signaling the end of this period.
Late Iron Age: 1500-1800 CE. In the West this contains the Renaissance, Protestant Reformation, Age of Discovery and Enlightenment. In Russia, it starts with the Great Stand on the Ugra and eastward expansion. In China, it's the parallel end of the Yuan Dyansty. Ultimately it begins with the spread of gunpowder, as seen in the gunpowder kingdoms, as well as the printing press, which had its greatest impact in Europe both for the good (Europe's domination of the world starting in this period) and the bad (religious civil wars as Northern Europe could communicate more easily.) At this time, the technological advantage of sedentary societies began to overwhelm that of nomads. Simultaneously, the benefit of technological innovation in the crucible of a sort of geographically-enforced natural federalism in Europe allowed Europe to outstrip China and colonize the world. The use of gunpowder as a source of energy more powerful than human or animal muscle anticipates the Industrial Age. Like the Mongols, Napoleon was the high water mark of Iron Age warfare, and was ultimately undone by the home of the Industrial Age, the United Kingdom.
Though not the focus of the post, the Industrial Age comes with its own subdivisions: the first wave in 1790-1830 with steam and water power leading to factories, materials extraction and textiles, an interim with three "transition wars" in the West (the Crimean War, American Civil War, and Franco-Prussian War; the Taiping Rebellion still appeared very much like a late Iron Age religious War, like a Thirty Years War compressed into half the time) with the second industrial revolution 1870-1910 converting industrial power into consumer goods; this culminated in World War I, the first industrialized war.
Subscribe to:
Posts (Atom)
