[Previously in sequence: Fundamental Value Differences Are Not That Fundamental, The Whole City Is Center. This post might not make a lot of sense if you haven’t read those first.]

I.

Thanks to everyone who commented on last week’s posts. Some of the best comments seemed to converge on an idea like this:

Confusing in that people who rely on lower-level features are placed higher, but the other way would have been confusing too.

We need to navigate complicated philosophical questions in order to decide how to act, what to do, what behaviors to incentivize, what behaviors to punish, what signals to send, and even how to have a society at all.

Sometimes we can use theories from science and mathematics to explicitly model how a system works and what we want from it. But even the scholars who understand these insights rarely know exactly how to objectively apply them in the real world. Yet anyone who lives with others needs to be able to do these things; not just scholars but ordinary people, children, and even chimpanzees.

So sometimes we use heuristics and approximations. Evolution has given us some of them as instincts. Children learn others as practically-innate hyperpriors before they’re old enough to think about what they’re doing. And cultural evolution creates others alongside the institutitions that encourage and enforce them.

In the simplest case, we just feel some kind of emotional attraction or aversion to something.

In other cases, the emotions are so compelling that we crystallize them into a sort of metaphysical essence that explains them.

And in the most complicated cases, we endorse the values implied by those metaphysical essences above and beyond whatever values we were trying to model in the first place.

Some examples:

People and animals need a diet with the right number of calories, the right macronutrient ratios, and the right vitamins and minerals. A few nutritional scientists know enough to figure out what’s going on explicitly. Everyone else has evolved instincts that guide them through this process. Hunger and satiety are such instincts; when they’re working well, they make sure someone eats as much as they need and no more. So are occasional cravings for some food with exactly the right nutrient – most common in high-nutrient-use states like pregnancy. But along with these innate heuristics, we have culturally determined ones. Everyone has a vague sense that potato chips are “unhealthy” and spinach is “healthy”, though most people can’t explain why. Instead of asking ordinary people and children to calculate their macronutrient and micronutrient profile, we ask them to eat “healthy” foods and avoid “unhealthy” foods. There’s something sort of metaphysical about this – as if “health” were a magic essence that adheres to apples. And in fact, sometimes this goes wrong and people will do things like blend a thousand apples into some hyper-pure apple-elixir to get extra health-essence – but overall it mostly works.

EXPLICIT MODEL: Trying to count how many calories and milligrams of each nutrient you get EMOTIONAL EXPERIENCE: Feeling hungry or full REIFIED ESSENCE: Some foods are inherently healthy or unhealthy ENDORSED VALUE: Insisting on only eating organic foods even when those foods have no quantifiable benefit over nonorganic

Every society has some kind of punishment for people who don’t follow their norms, whether it’s ostracism or community service or beheading. There’s a good consequentialist grounding for why this is necessary, with some of the most academic work being done in the field of prisoners’ dilemmas and tit-for-tat strategies. But again, we don’t expect ordinary people, children, and chimpanzees to absorb this work. The solution is the (innate? culturally learned? some combination of both?) idea of punishment. Punishment relies on a weird metaphysical essence of moral desert; people who do bad things deserve to suffer. The balance of the Universe is somehow off when a crime goes unavenged. Take this too far and you get the Erinyes and the idea that justice is the most important thing. There are references from ancient China to Hamlet that if you have something important you need to avenge, you need to do that now or you’re a bad person. None of this follows from the game theory, but it’s a really good way to enforce the game-theoretically correct action.

EXPLICIT MODEL: Trying to figure out how to best deter antisocial behavior and optimize society EMOTIONAL EXPERIENCE: Feeling angry when someone wrongs you REIFIED ESSENCE: Justice: the world is out of balance when crimes go unavenged ENDORSED VALUE: Wrongdoers must suffer whether or not that prevents future crimes

Value Differences as Differently Crystallized Metaphysical Heuristics 43:41 Share

Astral Codex Ten Podcast

Shownotes Transcript

Value Differences as Differently Crystallized Metaphysical Heuristics