Monday, February 17, 2025

Integers Are a Useful Fiction, But Math Is Real

Integers have no existence independent of our brains; they are a useful fiction humans have developed, extending on how our nervous system operates, but not pointing to something real about the universe outside of our skulls. I am not a mathematician or logician, and I concede that this is not a proof, but rather a collection of circumstantial arguments. I may of course not be aware, or not have processed the implications of, work already done in the field that supports or negates any of these arguments or has relevant implications. If you are aware of such, please comment below.

There are several positions on the objective reality of math, "out there", in the objective universe separate from human experience. The tension is this: math seems "real" in that it works, and is useful, and yet, it is very unlike other things in the world. Is it "real" or a human creation? It's commonly taken that there are three positions to take when considering the reality of math:

  • Platonism (or realism) – mathematical objects are real but abstract and outside of time and space, and our ability to apprehend them in the manifest world reflects this.

  • Nominalism – math is empirically derived from our experience of the world. (This is agnostic to the question of whether abstract mathematical objects exist.)

  • Fictionalism (or formalism) – math is a useful trick, employed by entities with specifically-constructed nervous systems; it is ultimately a language game that we use to make sense of the world. Some fictionalists would say this means that math is false, while others say this means it is meaningless, and/or that the property of truth or falsehood does not apply to math.


While typically these accounts seek to address the reality of mathematical objects generally, here I'm only interested in one. Indeed, in Wigner's words, mathematics is indeed unreasonably effective if fictionalism were true for all of math. I hope this is part of a continuing discussion about the reality of mathematical objects in general, and the meaning of their being a different kind of entities, which I address at the end of the post.

  1. Numbers are derived from the logic of Peano arithmetic. Peano arithmetic assumes the existence of the unit; that is, by positing the successor function, it assumes that the unit exists. The idea of the "fiction of the unit" is at the core of this claim about integers being invented - even in the formal construction of numbers, the unit is just assumed.

  2. While not all numbers are integers, we do use a system of integers to represent the digits of all numbers, including transcendental ones. In fact, most numbers are not just irrational, but transcendental, which cannot be constructed algebraically. This means that most numbers cannot even in principal be accurately represented by integers (non-transcendental irrational numbers can be algebraically constructed using integers, transcendental numbers cannot.) At least some of the numbers that describe nature like pi and e are transcendental. It's worth pointing out that the inability to represent transcendental numbers with integers is not merely a trivial artifact of their infinite length - despite not being able to represent pi, the Kolmogorov complexity of pi is very much finite.

  3. Historically, each time there is an innovation in numbers, it is first rejected as an absurd fiction, then accepted as a useful tool. This is well-illustrated by the history of negative numbers and then irrational numbers. This is exactly the pattern we should expect if in fact numbers are a useful fiction - except for integers, since our nervous systems had this quirk built in, and we never had to produce them effortfully in a rule-based system; we never had to get past the stage of disbelief.

  4. Over a century ago, set and number theory encountered difficulties, most famously with Goedel's theorem. - "one cannot prove consistency within any consistent axiomatic system rich enough to include classical arithmetic": which is to say, integers necessitate inconsistency. When asked what consequence we should expect in the real world from integers being a convenient fiction, this is the answer - for millennia we used units without trouble but as soon as we attempted to ground them in logic around the turn of the nineteenth to twentieth centuries, we ran into difficulties.

  5. You might object to my initial claim that if our nervous systems apprehend a thing, it must be real. This is incorrect, or rather it's not clear enough about what "real" means in this context. No, it's not that numbers are a hallucination - even if you hallucinate a dog, this doesn't disprove the existence of real dogs; if you hallucinate an abstract color pattern, this doesn't disprove the existence of colors. Rather, numbers are a kind of thing that exists only as subjective experience. We experience pleasure, pain, and emotions. They are subjectively real, they correspond to events in the world outside our nervous system, but cannot exist separate from a nervous system. Integers are the same category of entities.

  6. Expanding on instinctive human knowledge of whole numbers - it's a universal that for millennia humans have numbering systems with units (usually one, two, and many) without any formal grounding. Some animals (including non-primate ones, e.g. crows) can count at least this high. Note that biology having evolved representational tissues (nervous systems) capable of representing mathematical objects is certainly not an argument against the reality of those entities - indeed, if we live in a universe where there are abstract entities unidirectionally causing things in the physical universe, you would expect some of those rules to appear in the brains of replicators that evolve in that universe, who are will be imbued with some innate instrumental math sense for survival. The way we do this is using neurons which convert the input of continuous reality into a discrete fire/don't fire output. This is why the world seems to have discrete, countable objects when it is actually composed entirely of fuzzy gradients. Therefore, animals do not even need language to delude ourselves into believing in integers - this is where the "conceit" of integers is rooted, not in some philosophical mistake our ancestors made, but in the basic mechanism of how we perceive the world. (To extend integers indefinitely, we do need language.)

  7. If the claim that integers are an invention of the human brain is correct, then we would product prediction is that intelligent aliens would have a mathematics that did not use integers. Of course there are no aliens to talk to about this, but we have the next best thing: increasingly sophisticated computer systems. In at least two cases, computers have developed mathematics without integers as a primitive concept. In the first case I'm aware of, Wolfram Alpha "rediscovered" math without creating integer. Later, computer scientists showed how large language models use trigonometry to do addition. There are intriguing hints that even in our nervous systems, integers are not primitive but rather an adaptation of spatial reasoning. Gerstmann syndrome results in a stroke in (typically) the left parietal lobe; symptoms include left-right confusion and loss of arithmetic abilities. In OCD, there can be a sensitivity to both orientation (lines and angles) as well as certain numbers. In second-language speakers, otherwise-fluent speakers often must still resort back to their native language for both numbers and directions.

  8. I've deliberately left this point at the end of the list as it is the most metaphysical eyebrow-raising. If there is no such thing as a unit, only gradients, then there are no boundaries in the universe, and everything is the same object. This argument has even been applied to subatomic particles (see the one-electron theory.) If everything influences everything else, there are only gradients, and in reality there is only one unity (the universe as a whole) - then the idea of a unit beyond the universe itself is logically inconsistent. The unit of the universe would not actually be a unit either - if the only binary is existence and non-existence, then there is only one thing, and nothing that is not that thing. With no boundaries, it is meaningless to talk about a "unit".


The Making of a Dualist? Or at Least a Platonist

Initially I set out to show that math itself was a provincial tool of human cognition, and shrank my argument to just integers. And I do think based on the observations here that there is good reason to believe that integers are not a coherent mathematical object in the same sense as other abstract math and logic concepts. But I'm not making a claim about other objects, even about transcendental numbers (which is most numbers.) But I have not attacked abstract objects and have no suspicion that they are invented in the same way that I think integers are. It seems to me that if those of us who would call ourselves materialists in the philosophical sense, believe in both the material world, and the reality of at least some mathematical and logical objects - objects which appear eternal, unchanging outside of space and time as it were - we are clearly dualists. Obviously this shouldn't sit right with us, which is what motivated my interest. And here I am, left in the surprising position of agreeing with the Platonists, just quibbling over which types of mathematical objects that have meaning outside of a human nervous system.

The possible solutions both seem incredible:
  1. True Platonic dualism. These objects exist outside space and time but are real because they affect us, but they are causally asymmetric. It is this causal asymmetry that is key to their fundamentally different nature. They are a true uncaused cause, a true prime mover. Mathematical objects affect the material world, but the material world cannot affect them.

  2. Platonic monism. These objects are real, but are not causally asymmetric. They can change, and there is some kind of feedback from the universe to math and logic. It is worth recognizing that the question of whether physical constants change over time and space remains an open one, with recent evidence in its favor (See for instance Murphy et al 2003 for evidence of change to the fine-structure constant over time, which has stood up to scrutiny for a quarter century at this point.) But this is more fundamental: how can the nature of a triangle change? Or the nature of identity? The spatial dimensions? One interesting solution to Platonic monism is that advanced by Tegmark, who argues the "hyper-Platonist" monist argument that there are only mathematical objects. (Notably, Tegmark is the senior author on the LLM paper I linked to above.)

Monday, February 3, 2025

Is There Such a Thing as Color Harmony?

Some colors seem to go together; others do not. The same goes, even more strongly, for notes. Some form pleasant harmonies, and others are dissonant. In the case of sound, it's more mathematically obvious what's different: harmonious combinations of sound frequencies (like a major third) are constructively interferent, and dissonant combinations are destructively interferent. Light waves have frequencies as well, though we usually think of them in terms of wavelengths.

Another interesting difference in our perception of sound is that our nervous system automatically does a Fourier transform for us, before the sound reaches conscious awareness. This is why you can listen to a chord, and hear the individual notes, instead of a mess of superimposed frequencies. That doesn't happen when we're seeing multiple colors - others, there would be no such thing as non-spectral colors like pink, and your screen couldn't fool you into thinking you're seeing more than the three frequencies it's actually sending to your eyes. There are also biological reasons that certain colors are more important to us just by themselves, without context (like red, which could be fruit or blood), as opposed to certain isolated frequencies of sound (what cause would our ancestors have had to be really attuned to a sound at 640 Hz?) Still, I've always wondered if colors "going together", or pleasant mixtures of pigments used in famous paintings, are actually making some kind of harmony, and we're attracted to those harmonies without realizing.




Above: what comes into your ear. Below: what your brain receives, post-transform. This is a major chord.



I've considered doing a proper color harmony experiment. As I just said, computer monitors don't allow this - I can't just make images online and ask people which they find more pleasing, because computer monitors just emit red, green and blue, and we're not actually looking at those pure light frequencies. I would have to buy specific LEDs and have people do the test in person. The expense of this hardware exceeds my interest in the result; and, if you think somehow subconciously we can tell the difference between the fake RGB hue, and the perceived hue, you're already agreeing that we are actually doing a Fourier transform with light perception.

So what I did, using the fake color imitations coming out of our screens, was to look at a simple artwork with just a few colors, and alter them so that they are making harmonious or dissonant chords. Below is Piet Mondrian's Abstact Cubes. Since he uses three colors, that means we can make a chord.



My approach was as follows.

  • I found a website that would convert a wavelength into a perceived color (here.)
  • I took the red spaces as the root note (e.g., the C in a C chord.) I chose the red spaces as the root of the chord because red does tend to be dominant, analogous to the way the root note establishes the basis of the chord, and because red is the lowest frequency. (Why didn't I choose tones for the white and black? While we're stretching the analogy, let's say that's black and white are the spatial equivalent or rhythm, an atonal drum beat.)
  • I started with "C" (arbitrarily) at 780nm, which means the high C octave will be at 390nm. (For musicians this is counterintuitive. These are wavelengths, and as we go up the scale, the frequency gets higher but the wavelength gets smaller.) Also notice that humans can just barely see one visual octave.
  • I used equal temperament, calculating the frequency as frequency = root frequency * 2^((number of half steps up)/12.)
  • My hypothesis, which I expect will be falsified: individuals will display the a consistent preference for each color harmony (i.e., will consistently like major, minor, or discordant best.)
  • As a side note: to highlight the likely inaccuracy of the colors (remember, we're looking at RGB monitors - you're not really looking at real shade) - I use two monitors when I work and even between the monitors, the colors are noticeably different. I hoped that the ratios of the wavelengths would remain the same, but on one of my monitors there was a difference between colors that looked the same on the other. Further complicating things, our color vision is not equally sensitive across the spectrum - it's much easier to discrimate a 1nm difference in the middle than at the edge of the frequency band we can see.


Above is the "scale" that results from this. Below are chord compositions based on the arbitrary C at 780nm. Major and minor are obvious enough; the discordant one is the root, a diminished second, and a diminished fifth. Do any of these compositions look more or less pleasing to you? And is this in accord with the harmony (that is, is major more pleasing than minor, minor more pleasing than the discordant one?) For reference, below each composition is each respective chord, made from the Szynalski tone generator. On the sounds, click to pop out if you want to hear them.


To avoid any special effect from these particular colors, I did another set of three (major, minor, and discordant) starting on the arbitrary E. If there's a harmony effect, the order of preference should be the same for both sets.

(Here I kind of like the discordant one, but I'm also a metalhead who likes diminished fifths.)

Maybe Mondrian knew what he was doing when he chose the root note for his composition. What would happen if I started with that one as the root, that is, normalized his red to the C below A440, and built chords from that? Here is the Mondrian color scale:



Here are the three harmonies, with the root note as his red, and the three same chords. Which one do you like most? Also, converting colors back into sounds - here instead of a piano I used an online tone generator to compare the major, minor, and discordant chords to the chord made by the actual colors in Mondrian's painting. The reason I used a tone generator instead of a piano is that, going in this direction (from light to sound) the color frequencies are likely to land between the notes,, and I wasn't about to de-tune my piano.







Interestingly, of these, the major chord is the one that most resembles the original work, though the original chord (the harmony from the colors of the original work, normalizing the red to the C below A440) is not a major one:



I hope you've escaped this without developing color-sound synesthesia. For next steps I may put up these harmonies (blinded) for a vote to see if people choose them consistently. If you want more, at some point I'm going to automate the pixel-counting of some famous paintings and assign wavelength values to each pixel (the Bridge at Giverny, the Scream, Starry Night, School of Athens) and see what kinds of chords those make.

Dimensions of Art




Above: a catalog of scents from the Aftel Archive of Curious Scents in Berkeley, California, USA.

Sometimes, innovations in art occur because artists have conceived of, and/or technology has allowed, novel ways of creating an artificial object/event purely for the purpose of experiencing it. For example - the development of perspective in the Renaissance; or the evolution of the institution of the artist as a person who we connect with; or photographs and recordings that let us experience a live performance at our leisure; or pictures that move and fool us into experiencing a solid moving object.

Breaking down artistic experience into its basic dimensions might reveal "holes" in artistic possibility-space where no one has made an effort to create, and for too long I've been thinking about what these basic dimensions are, and where the holes might be. Of course, at least some of those holes will exist for good reasons. By creating a list of these characteristics, we could create a matrix and more easily see where there are artistic innovation deserts. Originally I thought I would just make a chart, and look for structure in the N-dimensional space ("art forms with X usually don't have Y") and that would reveal the remaining holes in art-space. But the list is longer than I thought it would be, with approximately 10^69 possible combinations, so I just eyeballed it rather than looking for holes systematically. I include these first, along with the list of dimensions, then a final section containing thought experiments exploring the role of our connection to the artist in our experience of art.

This project amounted to a quick-and-dirty survey of the entire human artistic enterprise, and to summarize: visual art tends to be representative, sound art tends to be abstract. Using other senses in art is limited by technology, safety, and capital. Representative art usually attempts to include a coherent narrative. These are the defaults, possibly driven by biology, and departures from these tendencies are usually deliberate choices and/or driven by changes in technology. As photography drove a move to more stylized, more abstract art with a focus on concept more than execution, the same may be happening in the near future to sound art, and I predict the innovative sound forms I list here are those most likely to be explored in the next 1-2 decades.


New Forms, Based on Blank Areas in the Map of Art Space

Some of these likely exist, and I just haven't encountered them. Some of these have likely been tried, but they're just not interesting. Is that a certainty for all of them? Art varies in its capital intensiveness (movies at the apex, vs a single scribbler typing on a machine that everyone now owns) which might further limit experiments. I include only forms possible with current technology - no neural interfaces.
  1. Silent films - in the sense that they do have sound, but no language (including text or signs.) Some of these exist (1981's Quest for Fire), but not as a genre.

  2. Art with intentional synesthesia - associating certain sensory experiences or concepts intentionally, before experiencing art. If you want to fully experience it, you must train before you go! Some people study a musician's work before going to a concert. We do this unintentionally by accumulating knowledge about the world and art so that when we encounter an artistic reference, we can appreciate it (if you don't know the classics, the Bible, and Shakespeare, you're having a very different and more confusing or impoverished experience of Western art than you could be.) You recognize this when you start watching parody comedies with children who haven't yet had the time to store information about their culture.

  3. Interactive electronic paintings; musicians who put out all the sound files used to make an album (including ones not included) for other musicians to use.

  4. Scent sculptures - each object has its own fragrance.

  5. Video game with switched "skins". I've avoided trying to define art, but still, a boundary case that I truly can't decide if I would include is video games. Obviously there is art in video games, but would I call the video game itself art? At football games, certainly there is art decorating it, but the game is at the core, and does not fundamentally change if the uniforms, the theme of the stadium, or the logos change. This is less clear in video games and possibly even some board games. Imagine a video game with switched skin, or randomly shifting underlying mechanics. This could be done either for video or tabletop games.

  6. Paintings with color harmony rules similar to 12-tone serialism (using the frequencies of light of each color; note, we cannot see a full octave.)

  7. I was about to inlcude surrealist or abstract photography, then I discovered Aaron Siskind.

  8. Sound-art with recordings or imitations of natural sound, and minimal language or abstract (tonal) sound. I once saw Ikue Mori perform with John Zorn and Mike Patton, and that sounds she made were so bizarre and original that I found myself only able to connect it to plumbing sounds and the sensation of Pop Rocks.

  9. A "blind" gallery. There are no labels with titles, years, media, or certainly artist names. I thought I would actually find this quite distressing, but then realized that this was my experience at Burning Man, and I'm only bothered by it, now that I reflect (and want to credit an artist I reference in this article.) What if I see something I like and want to learn more about the artist? It almost feels like a waste of time. But what if you were given a notebook to write down what you thought the artist was like and what they were trying to do? I'm not a professional artist, so I don't have a sense of whether they might be more or less interested in such an exhibit, but if it causes the audience to think more about them as an artist,

  10. An actor who attends galleries as the "front end", a fake artist persona, with a made-up back story. Everyone knows this is the case but plays along. Gradually the actor gets involved with actually producing the art, and eventually it is suspected that she was the artist the entire time.

    I also asked Perplexity, and like many humans, it thinks of possible forms without thinking too much about the execution (I removed the ones with no clear means to implement.)

    Synesthetic Reality Sculptures would be a groundbreaking art form that merges representational and abstract elements while engaging multiple senses in an innovative way. This art form would create three-dimensional sculptures that transform based on sensory input from the viewer, blurring the lines between representation and abstraction. Key features of Synesthetic Reality Sculptures:

  11. Proprioceptive Interaction: The sculpture would respond to the viewer's body position and movement, changing its form and color as the viewer moves around it.

  12. Sound-Induced Textures: The sculpture's surface would change texture based on ambient sounds or viewer-generated noises, creating a tactile representation of auditory input.

  13. Temperature-Sensitive Visuals: The sculpture would alter its colors and patterns in response to the viewer's body heat or touch, bridging thermal sensation with visual
    perception.



    WHAT ARE THE DIMENSIONS OF ART? The dimensions of art - its possible elements - break into two categories - sensory, and cognitive. The sensory (or innate, or physical) dimensions are those you could measure with a simple instrument - like color, time, or volume. The cognitive (or extrinsic) dimension require a human audience. The colors and shapes and sounds in a film are just projections on a screen or vibrations in the air - those are measurable physical, sensory properties of the movie. But, your brain knits together the sounds into coherent objects, assigns the sounds to them, understands the language and culture and emotions and intentions of the characters and actors and director - and you're now creating a narrative and using your theory of mind and this is central to your experience.

    Initially I thought this would be a simple task, but the intrinsic physical sensory dimensions are fairly limited. What the audience - the human brain perceiving the thing - brings into the work really is quite complex.

    While I'm trying to be as inclusive as possible, the emphases and omissions here will inevitably reflect my own tastes. (Imagine if everyone made such a list and it was nearly the same, or so different as to be alien - I think either possibility is interesting.) I also have next to no formal education in art history, so some of the holes I think I've found have likely been explored.


    1. INTRINSIC SENSORY PROPERTIES OF ART

    1.1 Time (change over time vs static)
    1.1.1 Interactiveness (how does it respond to audience)
    1.1.2 Rate of change
    1.1.3 Is there coordination between different senses or within the same sense
    1.1.4 Length

    1.2 Vision
    1.2.1 Number of spatial dimensions - one, two (image) or three (sculpture)
    1.2.1 Color
    1.2.2 Form
    1.2.3 Intensity and contrast
    1.2.4 Size

    1.3 Sound
    1.3.1 Volume
    1.3.2 Pitch
    1.3.3 Timbre
    1.3.4 Rhythm and Tempo

    1.4 Tactile
    1.4.1 Temperature
    1.4.2 Movement
    1.4.3 Smoothness
    1.4.4 Hardness
    1.4.5 Angularity, sharpness, or roundness
    1.4.6 Density
    1.4.7 Phase (gas/liquid/solid)
    1.4.8 Coherence (crumbly, intact)

    1.5 Olfactory

    1.6 Taste

    1.7 Proprioception/Vestibular - 4D films provide this and they've become more common in amusement parks.

    1.8 Passive or Interactive


    Notes on the sensory dimensions:
    • These dimensions are both more primitive, and for that reason more difficult to consider. Of course I wasn't able to fully remove the human element even here. I suspect that this is where the most productive "mining" could happen.

    • I haven't expanded on olfactory and taste for a reason. Visual and sound arts, and the coordination of the two (plays, movies), are the kings of the sensory world. This is because there are physical challenges to the others. For olfactory art - control of scent in space and time is difficult. One scent at a critical point in a play would not be technically difficult, or associated with an image (remember Scratch n Sniff stickers?), but how to make an olfactory symphony with multiple scents changing over time? How to ensure the scents disperse quickly enough and don't mix? This might be something for the first artists working in pure neural interface, where we don't have to worry about the experiences having an Ex Neuro origin. There's also a contagion concern - do you want to sniff something lots of other noses have been near?

    • Concerns similar to olfactory, but even greater, apply to taste-art. Food is mostly not taste-art. Yes, we "decorate" our food with extra flavors, but ultimately the reason we eat is for the physiological need for calories and nutrients, so I wouldn't count that as art. (A massive cake in the shape of the Taj Mahal that can be eaten after it is viewed? Principally for the entertainment effect - that would count as art because the fact that it is food makes it more interesting. You probably wouldn't make it at home and eat it by yourself; if you did it would be for the reward of seeing it, with - again - the thing being more interesting as a result of being edible.) Taste art would be something where the taste is mostly divorced from nutritional function. (Not e.g. diet soda, which mimics the nutritional function.)

    • Taste and tactile experiences are often destructive, therefore a work of art relying on these will have small audiences, especially taste.

    • I did not include all the tactile experiences humans can have (e.g. pain, vibration, itch) because those are more "invented" by our nervous system, others less so, like temperature. When something burns us, there is a measurable temperature of the object and our skin where it touches us, but the pain requires a central nervous system. That is to say: there is something hot on your foot, and your foot is in a shoe, so there is something hot in your shoe. But if there is a pain in your foot, and your foot is in a shoe, most people would not understand the pain as being in the shoe.

    • Size is an underrated aspect of art with a visual dimension; quantity is indeed a quality all its own. Probably the biggest surprise I've had, and strongest emotional reactions, from experiencing something in person I'd experienced only in other media, was seeing Goya's Saturn Devouring One of His Children in the Prado - because of its size.



    2. EXTRINSIC DIMENSIONS OF ART THAT HUMAN AUDIENCES ADD - while we bring these elements to art, they are inevitable to our experience. Daniel Dennett commented, in his comment on the old question "how do I know the red you see is the same as the red I see", that they cannot be the same, because of the associations we have with even basic sense experiences - you can't pry apart red from your other senses, memory, and thoughts about it.

    2.1 Interpretation - these are more subjective than the sensory elements, so some clarifying comments are helpful. Here, interpretation means the effect of the art on the the audience separate from the persona of the artist.

    2.1.1 Emotional experience - many of the other elements in this list are mostly a means to generate emotions, even the appreciation of abstract aspects.

    2.1.2 Representative vs abstract - In visual art, an image can be either a representation of the world (or at least of coherent discrete objects in the world) or completely abstract. Surrealist or cubist paintings are still representative, though e.g. Kandinsky or Matta are not. The obvious contrast here is between visual art, that usually represents something you could see, and music, which is non-representative sound - you could argue in fact music is abstract sound-art. The nearer to pure abstraction, the more difficult to avoid being unsatisfying. You might not enjoy a performance where you're handed the sheet music to Beethoven's fifth, any more than asking to hear a song and being handed a vinyl record (most of us don't have the hardware to translate the notes into any auditory experience, any more than we can run our fingers over the grooves of the record and hear it.) Some interesting examples of pure abstration are John Cage's 4'33", or the Museum of Non-Visible Art.

    2.1.3 Need for active interpretation of implied meaning - Orwell's Animal Farm has a political message and the characters are obvious parallels for (to him) contemporary political figures. Can be frustrating or tiring when the work seems to demand interpretation but you can't "solve" it.

    2.1.4 Language - are there words?

    2.1.5 Narrative coherence - is it a story, and does it have clear cause and effect (even if backwards like Memento), vs an abstract palette of shifting colors, or a surrealist movie with total lack of causality (Un Chien Andalou) or partial (Eraserhead.) Anything with any kind of narrative must either have the standard story elements of plot, setting, and character, or deliberately choose not to include them, and have the audience notice.

    2.1.6 Identifying with the performance - imagining your own body moving in the way the dancers are moving, or how you as an architect would have chosen materials for the building.

    2.1.7 Permanent or temporary - sand paintings; or Agrippa A Book of the Dead by William Gibson, which physically deteriorates after being read once.

    2.1.8 Tone - serious, comical, whimsical, dark.

    2.1.9 Status/community validation - basking in the glow of the Mona Lisa, knowing you will be able to tell others you did so. The crowds in front of this, versus at any other location in the Louv're reveal its importance in most humans' actual experience.

    2.1.10 Live or recorded performance (or not applicable) - would you watch a composer putting together a piece of music? Maybe, if she was also the performer, and was improvising jazz. It's harder to imagine watching a novelist typing or an artist painting, rather than their final effort. Then again, people pay to say symphony practices.

    2.1.11 Medium or instrument - I include this in the cognitive aspects rather than physical, because part of the experience here is not just the physical properties of the medium, a known medium the experiencer recognizes - a convention. (That's a trombone; that's burlap; that's tinsel and bronze.)

    2.1.12 References (in reverence or parody) - sometimes referred to as "literary reference" in media outside writing. Borges carried this idea to its extreme in "Pierre Menard, Author of the Quixote", about a writer who reproduces exactly the work by Cervantes.

    2.1.13 Style (and collision of style) - is it an Aztec codex? A Victorian dress? Some cross-fertilization of the two? In visual art, thanks to AI, we're flooded with these.

    2.1.14 Simultaneous experiencing (fellowship) - laughing at a comedy in a crowd is very different than watching it at home.

    2.1.15 Source of inspiration

    2.1.16 Driving medium in the creative process - is the music a score to a film? Or is it a music video where the music was primary? A novelization of a movie or a film of a novel? (Or created in parallel like 2001?)

    2.1.17 Innovativeness - (To keep our interest, art must continually walk a line between habituation to the familiar, and incomprehensible chaos.)



    Above, Triumphant Scale by El Anatsui, a Ghanaian/Nigerian artist who creates art objects (images? sculptures? textiles?) from found materials. Below, Frontier Psychiatry by The Avalanches, built out of samples, with the video made from found materials as well.



    2.1.18 Use of new technology or technique - separate from innovation. Without naming the specific technology or technique, there is an experience of seeing an early adopter at work - e.g., continuing of action in the Great Train Robbery, or Uccello's use of perspective.

    2.1.19 Exploration of limits and advantages of medium - Disneyland has a light show using water jets, which are projected onto as a screen, or in some cases directly illuminated from underneath (it's a nice way to create a 3D light saber.) Magritte's This Is Not a Pipe draws attention to the uncrossable gap between image and object. Extreme musicians try to affect their audiences physiologically with the "brown note". Twentieth century writers exploring and cleaving aspects of the authorial role and voice were using this element.

    2.1.20 Space/time/cultural context - is it in a museum? Someone's home? Is the building a former factory, or prison, or near a political leader's office? Is it a brightly colored object in a gray place, or is it made from local stone?

    2.1.21 History of the individual artwork, its history within the medium, and history of the medium itself. This is separate from merely reference to other works. For instance, capoeira as a form of dance emerging from slaves trying to conceal martial arts training.

    2.1.22 Expectation - are the actors remaining on the stage, or running through the theater? Are objects in a museum that you thought were purely functional actually part of the display? For the last one, at the Mattress Factory in Pittsburgh, a railing on a stairwell was actually an installation that looked like a railing and served that function, but furtively also included warm running water on the surface where you'd place your hand.



    2.1.23 Moral judgment, disgust, or shock - (Above, gold-plated poop by Terence Koh.) I give this its own category apart from other emotional impacts because it tends to be a brief experience. Even horror films are not end-to-end torture porn. Dada was an early incarnation of the genre. An example, where I wish I knew the artist's name: at Burning Man in 2000, I was riding around looking at art. It was nearing sunset, and I approached a structure made from a dozen ten-foot-tall glass columns arranged in a circle, about ten feet tall, stabilized and connected where each column met a horizontal ring about every two feet. Coming closer, each column was actually a stack of jars, each a subtly different shade of yellow or orange, some almost clear, sparkling in the evening light. Still closer and I could see that it was not the glass itself that was colored, but a liquid in each jar. And each jar had a person's name on it. They were each filled with someone's urine. I try to tell the story in this way over the years so others can share the full experience - the curiosity, the revulsion, then a shift to slow-clap congratulations to the artist. I lingered at the spot for some time, watching others approach, inspect with furrowed brow, then shout "Eeewww" and cover their mouths, exactly as I had (and if I did my job, as you just did.)

    2.1.24 Cleverness - considerable overlap with abstraction. There is a Bay Area artist (who I cannot recall or find online; if you know, please comment) who once had an apparently popular exhibition where attendees paid five dollars for the experience of giving him five dollars.

    2.1.25 Intentional or serendipitous creation - da Vinci's experiment with tempera on stone for the Last Supper makes for an interesting story, but since it was flaking off within his lifetime, probably not his intention. Sometimes objects are created without the intention of being an art object, but people who went to college in the 90s will recall the beautiful blue Arizona Iced Tea bottles that found their way to people's dorm rooms as vases. And finally, outsider artists often seem to have a different experience of what they create, or do not predict or understand their effect on an audience. Wesley Willis does not appear to have been trying to make comedy music that made people laugh, but to many people, it's funny. This made some people uncomfortable, but he kept selling records and touring.




    2.2 Characteristics of Creator - As time passes, this has become more important, and I suspect will only increase with the use of AI to make art.


    2.2.1 Persona and life history - feeling you have an understanding and connection of the person, and can see the influence on what they produce.



    2.2.2 Body of work, style, media worked in, consistency with other work - (Above, an early work by Mondrian. Yes, he drew curved things at one point.) The thing I found most remarkable about the Dali museum is the number of media he worked in. Auto body detailing, jewelry, it was hard to think of one where he hadn't tried his hand, and this now influences my experience of his work.

    2.2.3 Sense of mastery - Vermeer's curtains are famous for their realism; on the other hand, you can't help but notice mistake in an attempt at a realistic portrait; or you see a statue, and wonder if it posssibly could been intentional. Close inspection of the Velasquez paintings in Vienna's Kunsthistorisches Museum shows that much of the background were just dashes of dark color, which initially I found disappointing, but then stepping back to a proper viewing distance, I realized it produced exactly the represeentative effect he probably intended. In some ways mastery and innovation are opposing forces - it's too easy to explore an interesting new angle than just try to improve your technique, and in a competitive environment, comparison is easier, much like competition on price in economics (and the resultant race to the bottom, or some sort of shrouding behavior to avoid this.)



    Above: a Strandbeest by Theo Jansen.



    2.2.4 Authenticity of artist's life experience - imagine if you had bought turquoise jewelry from a Native American artist in Winslow, Arizona. Later you click on a story about an artist who was found to fraudulently claim he was Native American but was in fact Italian, and the picture in the story is the artist you saw. Do your feelings change about the jewelry? It's still the same object. (Similarly, once at a Bad Religion show, the singer made a comment about having signed to a major label after one of their albums came out. The crowd booed this development, at which point the bass player asked the crowd whether the quality of the already-recorded album somehow changed after they signed.)

    2.2.5 Artist's position in history and relationship to other artists and historical figures - did Verdi write a part specifically for her? Did she make a dress for a Persian shah? Was she arrested as part of a protest or revolution? Were any of her family members famous?

    SIDEBAR: THOUGHT EXPERIMENTS ON THE ROLE OF THE CREATOR IN THE EXPERIENCE OF ART ART



    Above: the No Name Art Museum in Jiaxing, China. This museum displays pieces by the artist collective No Name which consisted of Zhao Wenliang, Yang Yushu, Shi Zhenyu, and Zhang Da’an.


    Section 2.2 is still certainly incomplete. Much of what we get out of the experience of art is the experience of knowing about another person's inner life. An art museum with no name plates by any of the pieces might be quite frustrating to many people. AI is forcing a reckoning of how much knowing about the creator, the creator's characteristics, and even the existence of a creator matters to our experience. It's worth compiling some boundary cases about what it means to create something, and who or what can do, to as a vehicle for exploring this question.


    The Role of Personhood in the Experience of Art, Case 1: Intention versus Randomness in Method If you found out that Jackson Pollock had actually just used a fan to randomly spray paint onto canvas, rather than deliberately placing it, how would this affect your experience of it?


    The Role of Personhood in the Experience of Art, Case 2: The Influence of Species

    Form your impressions first:




    Top was painted by Congo, a chimpanzee. Second is the Makapansgat Pebble, at least 2.5 million years old, a natural object found at an australopithecus site - not modified, but carried miles from its initial location. Bottom is art, unmistakeable cave art! Made by Neanderthals, a different species.


    The Role of Personhood in the Experience of Art, Case 3: Taylor Swift Does Not Exist

    (Title stolen from a hilarious piece by Sam Kriss.) Taylor Swift's fans feel they identify with her, they understand her, they connect with her. Imagine that you find out that she has actually been at the cutting edge of technology, a kind of marketing experiment that was successful beyond the executives' wildest dreams: from the beginning of her career, there was an music-composing AI system (like the ones we can all play with now) writing her songs. She's a real person, she's just singing lyrics and melodies composed for her by a machine. This would profoundly change the way her fans react to her and her music. Of course, the disaggregation of roles in art production is not new. Why should we assume the best composers are the best performers? Holly Knight wrote hits for Pat Benatar, Tina Turner, Bonnie Tyler, and Aerosmith among many others, yet she's not a household name, for exactly this reason.


    The Role of Personhood in the Experience of Art, Case 4: Proximity of Agency and Consciousness of a Human to Your Experience of Art

    From the AI art test results on Astral Codex Ten. Worth clicking through to try it yourself. The caption for this one is: "One of these two pretty hillsides is by one of history’s greatest artists. The other is soulless AI slop. Can you tell which is which?" Try looking at both and imagining the person behind it - or, if youre like me, once you know one is AI and one is not and you can't tell which, suddenly it's just a blank.





    Above, Aditya Ramesh, lead engineer on DALL-E 3. Below, the Wintergatan marble music machine, with Martin Molin, the machine's creator. If you think Ramesh is NOT creating art, but Molin is (or at least performing it), why?




    The Role of Personhood in the Experience of Art, Case 5: Deliberate Creation as Art

    Above, we wondered about the proximity of agency to the artistic experience, how directly causal it was; what if there unambiguously was no agent causing it? Imagine we lived in a universe where Mark Rothko had never lived. You're hiking in a desert area, on the Earth of this Rothko-less universe. You see a pattern like this on a rock, composed of lichen and mineral deposits.

    Orange, Red, Yellow, Mark Rothko (1961)

    "Pretty, and how strange that it made that geometric pattern," you might think. You might take a picture of it, but it may or may not be your favorite thing you saw on that hike. (There are lots of areas on Earth that look manmade, many of them wall-like.)

    This is a very different experience from the one you have in this world, where Mark Rothko did live, and you see what he did deliberately placed on a canvas for others to look at, likely in a high-status place and knowing that other people have viewed it and gained pleasure from doing so. You place it in the context of Rothko's life experience and his other work.

    Even if, in our Rothko-ful universe, you went for a hike and saw that same pattern, you might think, "How strange that it made that geometric pattern, it looks like a Rothko." You might take a picture of it, but it may or may not be your favorite thing you saw on that hike.




    A picture I took as I flew south over California's Central Valley during a late winter's afternoon. On social media I referred to as a Rothko sunset. I actually don't even like Rothko, but I do like status, and comparing the sunset to it lets other people know I am the sort of person who knows Rothko.

Monday, January 13, 2025

Two Absurd Music Videos and the Importance of Human Connection

Cult of the Snowman, Doopiidoo (AI), 2025:




Hell, Clown Core, 2019:




The first one, a sort of silly nightmare, approaches video surrealism far more than any absurdist play or comedy ever has. The second is reminiscent*, in a good way, of John Zorn. (As an ex-saxophonist myself I'm proud that the most bizarre cerebral work in jazz tends to come from saxophinists. See e.g. Eric Dolphy.) But what I'm most interested in, is how much more interested I am in the second one, seemingly because humans created it.

I make no argument about AI Not Being Able to Produce Real Art - I think that debate seems quite over by now, and there's no innate, essential trait differentiating AI and human-generated art outside of our skulls.If you told me I misread the date, and in fact Clown Core is all AI-generated as well, I would shrug, then having learned that, my interest would diminish. It's not just the machine-vs-human aspect of it: if I found out that say, the band Carcass (one of my favorites) had for all these years outsourced their songwriting to someone at their record label, my reaction would be similar. When I see or hear art that appeals to me, I would to know about its creators, what kind of people they are, about their backgrounds and training and what it was like to produce the work. (Note in the case of Clown Core, they've never officially revealed their identities, which makes them more compelling still.)

There may, very soon, be a status value to "artisan novels" or "artisan symphonies" produced manually by human. This troubles me only because I don't at all understand the appeal of artisanal products (soap, bread, bicycles) and in fact I might be a little too vocal about rejecting their value. Being honest, the people who get excited about them usually strike me as trying a little too hard to signal their taste and authenticity, and sometimes as old farts pining for a long-lost age. I may grudgingly be joining their ranks.



Granted, I'm not aware of any AI engines that are proactively churning out surrealist videos without human supervision, entirely motivated by their own programmed-in utility function. There are still humans involved in the decision tree to create these things, if only at a very abstract level. What if I found out that Dali would open a dictionary, roll dice, and this would determine what he painted? No, and in fact that might deepen my appreciation - because he, a fellow human being, conceived of it, and executed it. Though I'm not aware of any such experiments, I'm sure artists have tried randomizing their art in some form. In fact, the automatic painting some surrealists did strikes me as the opposite of random - trying to produce "inscapes" (like Vertigo of Eros by Roberto Matta, above) that were even more authentically the product of the artist's mind than something he deliberated over and sicklied over with the pale cast of thought. Supposing there is some deep subconscious that large language models can expose with their own products - would I care more? No - so why do I care when humans do it? The answer must be that as a social animal, I am compelled by connection with others of my species, even with someone I can never meet.


*Clown Core also uses the technique of An*l C*nt, who pioneered the lyrical technique of clearly not saying any of the printed lyrics and just screaming; possibly they make up the words after they've finished recording.

Saturday, December 21, 2024

Shocked Pikachu: Gambling Is a Moral Hazard

Recently Zvi, a smart and principled person whose writing can frequently found in the rationalist blogosphere, wrote an analysis about online gambling. In this analysis he showed that online gambling on net harms its customers and in general makes the world a worse place.

The article is interesting, particularly where he discusses what actual smart bettors do to disguise the fact that they are smart (why would they do this? Read the piece.) My reaction to the article was twofold:
  1. HOW CAN HE POSSIBLY BE SURPRISED BY THIS?

  2. Likely because he is a smart and principled person (read: who has good ego integration and impulse control) and his model does not take into account that he is nears one corner of the distribution of humans in these dimensions.
To expand on this: cultures tend to have norms, both formal (written law) and informal (morals, taboos, traditions) around activities that engage or interfere with powerful primitive biological drives, and therefore can cause interpersonal conflict and future discounting. While cultures do this differently, the items chiefly among those things are psychoactive substances, weapons (increasingly as technology has made them better and states intensified their monopoly on violence control), sex, and gambling. Of these, gambling is in some ways the most interesting because it's very differently regulated in otherwise similar cultures (see: UK vs US) and not often a central vice or sin in religion. That may be because, while gambling is not new, gambling as a business activity that can simultaneously involve lots of people, is new. (I suspect if we're still here in a few centuries, social media will be the "new gambling" on this list.)

My claim is that any enterprise which traffics in the aforementioned activities a) necesssarily entails moral hazard and b) from a pareto principle argument, will get a major chunk of its profits from a slice of the population who are irresponsible with their drugs, guns, sex and gambling, harming themselves and others in the process. This seems trivially obvious. It also seems like this should be obvious to everyone else, in some form, even if lacking explicit reference to some of these concepts. (I may be near another corner of the distribution in the cynicism of my opinions about humans and our impulse control.)

You might object that we could avoid moral hazard by allowing only those who are competent to make good decisions in these arenas, participate in these activities - and indeed, we are trying to do this in a very low-resolution way by setting a legal drinking age, an age of consent for sex (low-resolution because there are certainly people above those ages who prove repeatedly they make bad decisions about when/how much to drink or when/how/with whom to have sex.) But this is exactly the opposite of what online gambling sites do. (If you win too much in brick-and-mortar casinos, they find ways to get you permanently out the door, and they don't care if they can figure out how you're doing it or whether it's legitimate. Again, as a smart and moral person Zvi may not have known much about how casinos operate.) Scott Alexander links to Zvi's piece, adding (emphasis mine):
I try to err on the side of liberty when it’s at all plausible, but I think Zvi makes a convincing case that this has destroyed too many lives for too little gain (it doesn’t even encourage people to be better at understanding risk and probability; the betting sites ban anyone who doesn’t seem like a rube).
Like Scott, I too have tried to err on the side of liberty but every advance in information technology is asymmetric in that it incentivizes increasing accessibility to the taboo domains (sex, drugs, weapons, gambling) for everyone, including (especially!) the people who should least have access to those things. People are manipulated into addictive behaviors via the components of happiness: pleasure, meaning, and flow (examples are porn and junk food, cults, and video games respectively.) That's why this is an interesting example - because increasingly, technology incentivizes liberalization of these domains. If you're reading this article, you probably would have found life in 1950s America intolerable for many reasons, many of them reducing to rigid uniformity - that is, not having the ability to make risky out-of-bounds choices to better yourself. You just weren't allowed many degrees of freedom in how you chose to live your life. But for people with poor impulse control, it kept them on the rails, and we've created a world that's great for Zvi, but not for his customers. This leads to the confession of a fallen libertarian: we harm people by liberalizing taboo activities, and we can't pretend that extending liberty does not give more room for those at the bad end of the impulse control spectrum, much more room to go off the rails.

I don't know what the solution is - jokingly (?) a drone that follows each person, and shocks them if they do one of the taboo activities that they are particularly vulnerable to - the problem being that, if this is in a democracy, the people that need the most protection are least likely to know they should consent to it.

Wednesday, July 24, 2024

The Decisive Fraction: What Percentage of the Population Wins or Loses a War?


In close elections, it's often pointed out (usually by the losing faction) that a relatively small portion of the country chose the whole country's fate, by choosing the country's next leader. For example, in the United States 2016 Presidential election, 42.3% of the total population of the country voted. But an even smaller fraction determines a country's future where the method used is state-organized mass violence rather than uncoerced choice. Here is a table of the most decisive battles of each war, and the fraction of the defenders' populations involved in each.

BattleWarYear Fought% Host Country Population
HastingsNorman Conquest1066 0.41
Spanish ArmadaSpanish Invasion of England1588 0.43-0.51
Plains of AbrahamSeven Years War (in North America)1759 0.29
YorktownAmerican Revolution1781 0.41
GettysburgAmerican Civil War1863 0.40
D-DayWorld War II1944 0.40


Cherry picking? I chose the the most decisive battle in six conflicts in Anglophone history. Notably, the major decision I had to make was, for D-Day, do I use the combined population of the Allies (in which case the percentage is 0.063%) or, as I did here since it seemed to make more sense, the defending country hosting the battle, which was France. (Only a minority of troops in that battle was French.)

Is it possible that the most decisive battle is usually the biggest battle? Yes, I would think there aren't many most-decisive battles that are small - in fact the Battle of Plains of Abraham is famous for being small and quick among important battles. Looking at the Wiki article for the list of American Civil War battles, and at all the battles listed as having "A" importance as considered by historians, the mean and median come out to 0.21% and 0.18% of the population, with a range of 0.0004% to 0.58%. Clearly Gettysburg was bigger than all but a few of even the important battles (all but 6 out of 42 others.) If you'd like to add up all the "B" battles to look for mean and median go ahead - at a glance, they are clearly smaller on average.

The biggest land battle in history in terms of number of combatants is probably the Soviet counteroffensive at Stalingrad, which saw 0.59% of the Soviet population participate in the battle. We might reasonably consider this an upper bound of the amount of a civilization that could participate in a battle. This is also considered by many historians the most important battle of the war.

It is also possible that historians, lacking counterfactuals, merely pick the largest battles of any war since they have objective numbers of participants and casualties. However the counterargument there is that if the generals at the time did not consider a battle important, they likely would not have contested it with a large number of troops, and there would have been no large battle.

Assuming we're converging to 0.4 for most of the critical battles, what determines this number? My guess is it's a function of percentage of willing able-bodied men, transportation, and "surface area" between combatants. These of course are all pre-ICBM numbers, and now that we're all combatants, when a nuclear conflict occurs, the numbers will all be orders of magnitude higher, though it's hard to imagine a family sleeping in their home as combatants.

Sunday, January 7, 2024

Fiction Is Dreaming In Print

There are mathematical techniques intended to counter over-fitting in data. Imagine a stockmarket analysis program. If it's trained on a year of data when, that year on June 23, all tech stocks dropped, it might start to recommend selling (or shorting) all its tech stocks just prior to June 23 every year. Obviously this isn't smart; the program has overfitted. What you might do is take the training data and slice it up and restack it in some new way - different sectors, different months - and the program might start finding more meaningful relationships.

This is analogous to dreaming, and has been rediscovered (or re-engineered, if you prefer) in other settings for some years now. We still don't really know why we sleep, let alone why we dream. But if we assume that brains are doing the same thing - avoiding overfitting - the explanation makes sense, and is consistent with the characteristics of dreams. Suddenly you're at the beach you used to go to every summer as a kid, but then in your school; your deceased grandmother is there, at the same time as your asshole coworker from the last company, and then you're driving down a steep mountain road with no brakes. Obviously they never met, and they especially didn't meet in some bizarre hybrid beach-school-mountain place. By mixing them, you're trying to avoid overfitting. In contrast, if you have PTSD, you do dream literally the same concrete traumatic experience over and over - and your waking behavior is overfitted - you avoid trucks under overpasses, or that one street corner, or movies about fire, based on whatever experience you had that you can't "digest", integrate with the rest of your life's experience, and move past.

Returning to computers, Andrej Karpathy on Twitter says "...in some sense, hallucination is all LLMs do. They are dream machines. We direct their dreams with prompts. The prompts start the dream, and based on the LLM's hazy recollection of its training documents, most of the time the result goes someplace useful." Similarly, the data stored and sliced and diced in our dreams influences how we perceive the world. It provides the top-down filter for the bottom-up sense data pouring in.[1] Without this, we veer off into hallucinations and delusions within days.[2]

But sometimes we humans' dreams intrude into our waking hours, drawing attention to themselves in full form, like stars briefly visible in the daytime if you know where to look[3], or if you like, a laundry dryer opened mid-cycle to let a sock or a shirt fly out. In the shower, you start laughing when a joke you heard in eighth grade pops out of nowhere. While driving to work somehow you're suddenly thinking of your recently deceased cat, and you're sad.

It seems to me that fiction is a more elaborated version of this, committed to a less ephemeral form, one that produces fossils of our overfitting-avoidance. Something has to explain the reason that we write stories that never actually happened, that we know no one will ever think actually happened, and that we (mostly) never even show to anyone else. (Out of all the short stories ever completed in history, what percentage of them was even intended for publication, let alone did the author think had a real chance of being read by someone else?) So why do we bother? Why are we so compelled? We're avoiding overfitting the things that are important to us. We write character-driven stories examining the psychology of people similar to those we know, or we write alternate history what-ifs about events that we find interesting - all of which we are trying to understand better and connect to our other experience, as we turn it over, and slice and dice it and rub it up against our other experiences.. Even the writing process itself is consistent with this - the ideas somehow just appear automatically, along with some scenes and images and events, that the writer has to organize (often laboriously, decidedly non-automatically) into a coherent narrative.


[1] In psychosis, the top-down part of the process dominates and you're lost in a waking dream of hallucinations and delusions. Autism is sometimes thought of as a diametric opposite to psychosis, when the bottom-up sense data dominates and unfiltered and the overwhelming cacophony of sounds, lights, or textures become intolerable.

[2] Waters F, Chiu Vivian, Atkinson A, Blom JD. Severe Sleep Deprivation Causes Hallucinations and a Gradual Progression Toward Psychosis With Increasing Time Awake. Front Psychiatry. 2018; 9: 303.

[3] When Venus is on the same side of the Sun as Earth, you can actually see it with the naked eye during the day. When I first located it in a blue daytime sky, I found this almost disorienting.