Issues with steelmanning

People’s views don’t make sense. People do not have nice coherent philosophies. People are not consistent, people don’t think through the implications of beliefs, people are irrational. Of course people believe what they believe for a reason, in that things have explanations. (If you drop a pen you are holding up, it falls. Why? Because you dropped it. While under the effects of gravity.) However, those reasons are not “they are perfectly rational beings who sat down and thought about it, free of all bias, and came up with their politics/philosophy/etc.” They aren’t even “they are mostly reasonable with regards to politics and came up with their political views after some deliberation”. The reasons have to do with a complex jumble of the motivators that drive humans. Humans are complicated. Humans are not anywhere near reasonable when thinking about things like politics and religion. Tribal and territorial wars sometimes bear superficial resemblance to civil discussion but only as another memetic weapon (this is a lie. Sometimes they resemble civil discussion because the participants think that civil discussion is virtuous. Humans are complicated.) Steelmanning implies that the reason behind people believing something is that it makes sense. This is, well, factually false.

Making things make sense is a fun thing to do. A lot of what we do is for fun and I tentatively support the claim that blogging is usually morally abhorrent by some theoretical version of virtue ethics which I don’t endorse but view as a good thing to point to when somebody asks what a morality is. However, assuming we have some goal beyond having fun drowning ourselves in politics, well, there is a world to save. Or information to acquire or predictions to make or whatever it is you value. Being epistemically charitable in the sense of considering the possibility that others’ ideologies are as sensible is your own is reasonable. Acting as if others’ ideologies are actually sensible isn’t. (If what you want is to gain status, charity probably works pretty well in certain communities.) I’m not sure what the advantages of a better version of steelmanning which produces more true statements are, but if we’re valuing having lots of true statements anyway we might as well try to produce more.

I’ve noticed myself being more charitable to the right. (I am not sure “the right” is the correct gerrymandering here.) The process by which this happens is something like this: when somebody expresses an opinion far enough to the left of mine, I perceive it as misguided and annoying. When somebody expresses an opinion far enough to the right of mine, I perceive it as evil and my brain yells “nobody thinks of themselves as evil, their position must make sense to them, think more.” There are confounders here: when I see a person/group accusing another person/group of being inherently evil for politics, I automatically defend the accused party, and most of such accusations are from the left (I don’t know if this is because of characteristics specific to my social circle or a wider phenomenon and I’ll need to look into it more before I can comment on causes). (As I was writing this, I checked Twitter and saw somebody calling an article about how abortion is okay “pure evil. It’s not wrongheaded, it’s not foolish, it’s not mistaken – this is pure evil.” My reaction to that was the same as my reaction is when pro-choice people talk about how evil pro-life people are—”they have different beliefs about what constitutes babies, they don’t think that [pro-life talking point]/[pro-choice talking point] is okay”) Also, nearly everyone I know to the right of me is very reasonable and ingroup-y, while a lot of the opinions to the left of me which I see are something like people I follow on tumblr for their usual non-politics content but don’t really know. When people express opinions in the second category, they frequently turn out to be some sort of mainstream tumblr leftists, for whom I feel no sympathy (I believe the positive term for “politics hipster” is “contrarian”? I usually don’t feel sympathy for ideologies with too many followers) but don’t alieve that they’re evil. This almost never happens with people to the right of me because, well, they talk about politics all the time. Boring-left people are everywhere, boring-right people are dark matter universe (if an actual Republican reads this, please let me know).

Steelmanning has a depth-first problem. To some extent this may be peculiar to me, but it an existing effect if perhaps not as big one as I think. Much like nitpicking, it is mostly applied to that which is close by, not that which is important. Pathological steelmanning, much like pathological criticism, leads to being eaten by internet politics at no gain, though perhaps in a more dignified way. Charity and civility are good in that they make everything nicer and probably more correct (although also just closer to the mean of the group in question), but being charitable to whatever’s in your face in a way which implies a time investment leads to some unfortunate time allocation. All time-eating automatic reactions to politics which is nearby have this problem. (I do also oppose spending lots and lots of time thinking about the ideal system of government and writing blog posts about it. Um. Theoretically oppose. Not oppose in the sense of not doing it. Internet politics is great at making the people involved in it think that it matters. It’s lying. It doesn’t.) Just because we’re being nice while being eaten by politics doesn’t mean we aren’t being eaten by it.


On Harm-Minimizing

I have been talking about a Weird Left-Wing Low-Status-Is-High-Status Thing for a while. I doubt that “left-wing” is the right classification here, although Haidt makes me update in that direction. Nevertheless, harm-minimizing cultures. In something like mainstream social justice, suffering is high-status. Being oppressed is high-status. (The literal ideology might not say this. This is what happens. If you write a program and think it’ll do one thing and it does another thing, you can’t claim that that wasn’t really the program. It was. It has bugs.) The meme “oppression olympics” actually describes the reaction to this pretty well.

Scott Alexander, One place where they’re [India] “ahead” of the US is affirmative action. America only has a little simple affirmative action for a couple of minority races. India has so much affirmative action it makes my head spin. First there are hundreds of different tribes. Then there are the four castes, dozens of sub-castes, and out-castes. Then there are the two major religions, Hinduism and Islam, plus lots of minor religions like Sikhs and Jains and Buddhists who still have tens of millions of people. If I am a moderately wealthy Hindu Brahmin from Delhi, and you are a poor Muslim Rajput from Jodhpur, we each have our own position on the affirmative action ladder that whoever is trying to hire us has to figure out so they can make their quotas for all the billion odd different minorities.

Sometimes it gets violent. The Gujjars, a tribe from Gujarat, are according to the government kind of disadvantaged but not TOO disadvantaged, and their place on the ladder reflects that. But they feel like they’re really disadvantaged, and need a much lower place on the affirmative action ladder. Solution: three weeks of violent rioting, ending yesterday when the government finally told them they’d demote them to a more highly discriminated against class. But it’s by no means over. Now the Meenas, a group that lives near the Gujjars and has always been highly discriminated against, are worried that the Gujjars will get their jobs, and wants to be moved to an even more highly discriminated against rung of the ladder to compensate.

If some tribe or village or minority is really angry at the government, for example everyone all the time, they can hold a “bandh”. A group called the “Gorkhas” from “Gorkhaland” is doing that right now. They want an independent state, and the government’s not giving it to them. So they go on strike and block all of the roads and other transportation links they can find. Economic life completely shuts down and no one can get in or out of their territory. There are some pretty angry tourists in Gorkhaland right now, but they’re nowhere near as angry as the farmers who have just harvested their crops and are watching them rot in the fields because they can’t transport them to silos. If you’re a fan of Darjeeling Tea, don’t expect to get a lot of it this year.

There is a different right-wing or perhaps gryffindor, which is about glorifying doing things despite pain. (This also has failure modes. When I rock climb, I deliberately keep climbing after my hand skin tears, because doing hard things makes me cool. It also makes it hurt to type. Hi.) I am not talking about this. The SJ way of glorifying pain is about pain which you can’t endure. In SJ, if you are curled up on the ground crying, you win the game and the status. In the other thing, this makes you weak, but being subjected to it and continuing on makes you strong. In SJ, everything which makes you more disadvantaged makes your voice count for more. If you play it right, you can make not being oppressed into an axis of oppression and then fascinating meta stuff happens.

Robin Morgan: My white skin disgusts me. My passport disgusts me. They are the marks of an insufferable privilege bought at the price of others’ agony. If I could peel myself inside out I would be glad. If I could become part of the oppressed I would be free.

She gained status for saying this. She suffers over not being a bearer of suffering, but suffering is high status (as is humility about those things where you are privileged and pride in traits which make you be oppressed of course), so she gains status for saying that she wants to peel herself inside out.  Saying this is nominally about “checking her privilege”, but it also puts her at a lower place on the privilege ladder, a win.

Scott Alexander, The impression I’m getting is that yes, nerds think they have problems, but actually they’re really privileged. So their problems aren’t structural oppression in the same sense that women’s problems are. So. Quick hypothetical.

I’ve postulated before that “privilege” is a classic motte-and-bailey term. The motte, the uncontroversial and attractive definition, is “some people have built-in advantages over other people, and it might be hard for them to realize these advantages even exist”. Under this definition, it’s easy to agree that, let’s say, Aaronson has the privilege of not having to deal with slut-shaming, and Penny has the privilege of not having to deal with the kind of creep-shaming that focuses on male nerds.

The bailey, the sneaky definition used to push a political point once people have agreed to the motte, is that privilege is a one-dimensional axis such that for any two people, one has privilege over the other, and that first person has it better in every single way, and that second person has it worse in every single way.

This is of course the thing everyone swears they don’t mean when they use the word privilege, which is of course how the motte-and-bailey fallacy works. But as soon as they are not being explicitly challenged about the definition, this is the way they revert back to using the word.

Go back to the original Amanda Marcotte article. Check the title. “MIT Professor Explains The Real Oppression Is Having To Talk To Women”.

That phrasing, “the real oppression is…”, carries a pretty loaded assumption. I’d say “hides a pretty loaded assumption”, but it doesn’t seem to be doing much work to hide it.

If you look through Marcotte’s work, you find this same phrasing quite often. “Some antifeminist guy is ranting at me about how men are the ones who are really oppressed because of the draft” (source). And she’s not the only one. If you Google the term “are the ones who are really oppressed”, you can find an nice collection of people using this exact phraseology, including a few examples from a charming site called “Nerds Fucking Suck”.

But Aaronson is admitting about a hundred times that he recognizes the importance of the ways women are oppressed. He’s not saying his suffering is worse than women’s in every way, just that it’s really bad and maybe this is not the place where “male privilege” should be invoked. The “is really oppressed” isn’t taken from him, it’s assumed by Marcotte. Her obvious worldview is – since privilege and oppression are a completely one dimensional axis, for Aaronson to claim that there is anything whatsoever that has ever been bad for men must be interpreted as a claim that they are the ones who are really oppressed and therefore women are not the ones who are really oppressed and therefore nothing whatsoever has ever been bad for women. By Insane Moon Logic, it sort of makes sense.

As a result, Marcotte is incapable of acknowledging that Aaronson feels pain or has feelings more complicated than “all women exist solely to be my slaves”. She has to be a jerk to him, otherwise it would be a tacit admission that he has problems, which means only he has problems, which means no woman has ever had problems, which means all women are oppressors. Or whatever.

Marcotte is angry that Aaronson doesn’t cite any feminist writer besides Andrea Dworkin, so let’s go with Julia Serano here:

What if you’re trying to hold the same weird one-dimensional system in a way consistent with basic human decency? That is, you don’t want to do the Vogon thing and say Scott Aaronson’s misery is totally hilarious, but you also don’t want to acknowledge that it counts – because if it counted you’d have to admit that men have it bad in some ways, which means that the One Group That Can Ever Have Things Bad spot is taken by men, which means women don’t have it bad?

As best I can tell, the way with the fewest epicycles is to say “Yes, your pain technically exists, but it’s not structural oppression“, where structural oppression is the type of pain that fits neatly onto the one-dimensional line.

Laurie Penny is an extremely decent person, but like a shaman warding off misfortune with a ritual, she must dub Aaronson’s pain “not structural oppression” or else risk her own pain not counting, being somehow diminished.

I mean, I don’t think she thinks that’s what she’s doing. But I’m not sure why else it’s necessary to get so competitive about it.

Absent the one-dimensional view, it would be perfectly reasonable to say something like “You feel pain? I have felt pain before too. I’m sorry about your pain. It would be incredibly crass to try to quantify exactly how your pain compares to my pain and lord it over you if mine was worse. Instead I will try to help you with your pain, just as I hope that you will help me with mine.”

Given the one-dimensional view, any admission that other people suffer is a threat to the legitimacy of one’s own suffering. Horrible people will deny and actively mock the pain of others, but even decent people will only be able to accept the pain if they also mention in an aside that it doesn’t count as the correct sort of pain to matter in the moral calculus and certainly isn’t even in the same ballpark as their own.

But the one-dimensional view sucks. It is the culmination and perfection of the phenomenon I described in my post on social justice terminology, the abandonment of discourse about the world in favor of endless debate about who qualifies for certain highly loaded terms like “structural oppression”. And those terms end up as a sort of Orwellian Newspeak that makes it possible to dismiss entire categories of experience and decree by fiat who does and doesn’t matter.

Some failure modes of awarding status for pain also apply to simply trying to minimize harm caused. If your culture offends nobody, you incentivize offense. It also incentivizes framing all requests as “not doing this hurts me”. People do this in tumblr rationality a lot, because it ensures that requests will be taken more seriously (there is a different and better example from the rationalist world, but I will not name the person in question since he is an individual person).

Vladamir M, Yvain:

The offender, for eir part, should stop offending as soon as ey realizes that the amount of pain eir actions cause is greater than the amount of annoyance it would take to avoid the offending action, even if ey can’t understand why it would cause any pain at all.

In a world where people make decisions according to this principle, one has the incentive to self-modify into a utility monster who feels enormous suffering at any actions of other people one dislikes for whatever reason. And indeed, we can see this happening to some extent: when people take unreasonable offense and create drama to gain concessions, their feelings are usually quite sincere.

You say, “pretending to be offended for personal gain is… less common in reality than it is in people’s imaginations.” That is indeed true, but only because people have the ability to whip themselves into a very sincere feeling of offense given the incentive to do so. Although sincere, these feelings will usually subside if they realize that nothing’s to be gained. also applies, but it’s too long to include. I don’t endorse all of it, but do read it. is great and tangentially relevant.

More specific: utilitarian cultures incentivize becoming a utility monster. Utilitarian cultures breed people who feel as much pain as they can over something to make other people pay attention to it. This is not a theoretical concern, this is what I see happening here. When I posted about this on tumblr, I got replies like “but wouldn’t a utility-optimizing agent see that this is happening and stop it?” Well, yes, but people are not utility-optimizing agent. People are not smart. People do not believe their ideologies. Ideologies nudge people in their direction (I’ve said this before even publicly, but it bears repeating). The direction in which utilitarianism nudges people results in the creation of agents who feel genuine pain over every divergence from their wishes, because that’s how you win among utilitarians. (A utilitarian morality czar might want something about a virtue of altruism (measured by effect) and a duty to be happy, perhaps. Utilitarian metaethics do not produce utilitarian ethics.) However, the specific case is not the important one. Utilitarians are a minor group, and most are not, technically speaking, harm-minimizers. The failure mode applies to all harm-minimizers.

The Weird Left-Wing Low-Status-Is-High-Status Thing assumes that its status system is not the mainstream one, and causes people close to it to try to lower their nominal mainstream status, where the status system in question decides what the nominal definition of the mainstream status system is. Okay, for the duration of this paragraph I’m going to call the Weird Left-Wing Low-Status-Is-High-Status Thing status system “Status System A”, the actual mainstream status system “Status System B”, and the status system which the Weird Left-Wing Low-Status-Is-High-Status Thing/Social Justice claims is the mainstream one “Status System C”. By the way, if some good at names people could name all these things (modes of pain glorification too) that would be great. The best thing suggested for the Weird Left-Wing Low-Status-Is-High-Status Thing is “hufflepuffism”, but that has the issue of lack of clarity. So uh, the things should be named and then these names popularized. Let’s try again. Status System A assumes that it differs from Status System B, and causes people close to it to try to lower their status in Status System C, where Status System A decides how Status System C is defined, with some relation to reality. This is of course exacerbated by the meme about how no harm done to someone higher on the monodimensional privilege ladder counts. Not every kind of pain contributes to lowering one’s position in Status System C, only the approved kinds (which have to do with relationships to society, usually), so talking about pain glorification is somewhat imprecise in this context.

Harm-minimizing is not something obviously good. Harm-minimizing is, like every simple ethical rule I’ve seen, riddled with horrible pits of doom and perverse incentives. Awarding status for pain is similar but with the problems exaggerated.

Brief Angry Atheist Rant About Angry Atheist Rant About Angry Atheist Rants

Someday, I will give a post a title and it won’t be a sentence long.

Ozy wrote an angry atheist rant, about half of which was meta. I am not responding to the meta part. This is a brief angry atheist rant. I am responding to this:

My radical position is that you should not believe things that are factually incorrect. Even if believing them makes you feel nice. Even if it makes you a better person. Even if it connects you to your community and your ancestors. (You can go to the rituals without believing in God, anyway.) Even if some other people over there believe factually incorrect things sometimes too.

I feel like having to justify why believing things that aren’t true is bad is like having to justify why hitting people who don’t want to get hit is bad. It… is? Duh? But the number of otherwise intelligent people who say “they’re nice and they aren’t hurting anyone, what do you care?” suggests otherwise.

The doublethink necessary to believe things that aren’t true hurts your ability to figure out what’s true and what’s not. Not necessarily; humans are very good at compartmentalizing and often, say, take “I feel it in my heart” as adequate on matters of theology but not on matters of medicine. But in aggregate turning off your critical thinking and rationality sometimes hurts your ability to be rational.

If you believe things that aren’t true, you’re going to make decisions based on the false things you believe, and decisions made based on inaccurate information are usually bad decisions. If you believe apricot pits cure cancer, you might skip chemo. If you believe there aren’t any cars on the street when there are, you might cross and get hit by an SUV. If you believe all-nighters improve your grades, you might flunk an exam. If your map says that Disney World is in Michigan, you will never get to meet the Mouse. For that matter, look at basically any atrocity in history, nearly all of which were caused by people believing untrue things (usually “this group of people is inherently evil,” “God said so,” or “our ludicrous political system totally works”).

Our shared culture, that of LessWrong, values truth a lot. We have the Litany of Gendlin and the Litany of Tarski and all those other ideological claims disguised as obvious truths in order to better induct people into the cult (it’s possible that they aren’t particularly well-disguised, I was inducted into this cult when I was 12). We tend to hold it as self-evident that the true thing is the correct thing to believe. The post quoted above just made a bunch of unsubstantiated claims based on ideology and limited personal experience. “Believing false things is bad because, well… duh” is not the sort of claim you can assume is true because it’s obvious. “In aggregate turning off your critical thinking and rationality sometimes hurts your ability to be rational” is a claim for which you’d need evidence. It’s obvious because you read Overcoming Bias/LessWrong, or perhaps because you read Bertrand Russel, or perhaps because your parents told you so when you were a child and you see it as too obvious to question.

You can value truth as a terminal goal, but that is a strong claim. It is not something you just say because duh. You don’t know that religion is harmful in general. As far as I can tell, becoming very good at valuing truth has harmed me. “Value truth” as a meme is a good one, but like most such memes, it is not meant to actually be implemented. Religious people don’t follow their religions, and if they did it would be a horrible mess incompatible with civilization as we know it (possible exception: Islamists). (See it’s okay for me to make claims based on deductive reasoning because my ideology doesn’t say you can’t do that. My ideology says that the world is too complicated and the state of social sciences is too, well, unscientific to do anything else.) It might make sense to go around saying that we what can be destroyed by truth should be. It does not make sense to actually destroy everything that can be destroyed by truth (there is also my finite-time argument but I am not going into that now because brief).

Humans are really, really complicated. You can’t expect that truth-optimizing will actually be optimal for them. There are the obvious points where truth is useful. If you deny physics, your planes won’t fly. If you find a new useful truth, you can use it to build cool things and make a ton of money. Religion is not such an obvious point. Religions have been under selective pressure for hundreds or thousands of years. For most things, the best version of it or solution to it hasn’t been tried. It’s possible that a future super-religion will incorporate not being built on lies. It’s possible that atheism in its current form is in fact the best existing religion—although of course it lacks a lot of elements of community and ritual and enforcing morality/cooperation and making people have lots of kids (which I am not sure is a good thing because I can’t really wrap my head around valuing human lives, but presumably) and pointing to atheist income levels or IQ or something similar is heavily confounded by the people who can become atheists in a religious world. However, neither of these are self-evident. Religion does things, and assuming that you can just throw all of them out because it’s not factually true is naive; note that current atheists are still surrounded by culture, which is overwhelmingly religious and full of religiously based memes without context. What is “don’t brag about charity” for? It’s for proving that you’re really doing it for God and not to gain status. I’m not a utilitarian, but you are, Ozy. If religion is useful, people should be religious, even though it isn’t true.

EDIT: Alyssa Vance ( points out that while believing what everyone else believes will lead to having similar outcomes as them, sometimes doing unusual things is necessary, and in that case truth is very important because you don’t know what you don’t know.


All of these various problems mostly come down to defining words, let’s talk about this one in particular!

A while ago, there was an article by R. S. Bakker which people were arguing about on tumblr [content warning for linked article: made some people very upset, mostly disability rights left-wing crowd (I am not sure what to actually warn for so instead I’m just describing group which might want warning)]. At the time, I purposefully scrolled past every argument (the tumblr interface is terrible/amazing and leads to seeing any given argument lots of times and everyone involved always being angry) and didn’t read the article, but I put it in my list of things to read and I have finally gotten to it. Short and hopefully charitable summary of the article: when we start understanding how things work or what causes them, we decide that they’re “not their fault” and accommodate them instead of subjecting them to the normal incentive structures. As we start to understand how people work better, more behaviors will be understood and in the limit of this we have no more incentives and just try to accommodate everything. Therefore, our education system (his examples focused on education) relies on not understanding neuroscience. This sounds pretty reasonable, to be honest, and I am not sure what exactly the claim that he hates disabled people and wants them to die is based on.

The problem underlying this is basically the weird free-will-based definition of what to excuse and what to condemn. You accommodate people because They Can’t Help It. In this paradigm, understanding how something works implies that you can’t punish it. It makes more sense to define whether you should accommodate things in terms of whether negative incentives affect it. For example, if you keep giving a paraplegic detention for not running a mile, this will not make them run, so if the goal is teaching everyone to run it doesn’t make sense to do that. This is obviously susceptible to not knowing things, but there is a coherent rule which makes learning more neuroscience not cause the breakdown of society and all that we hold dear in the apocalyptic flames of slowly growing censure, which is nice.

Also, more explicit goal statements. I know I always say that, but I’m always right about it. What is your educational institution trying to accomplish? Education? Well, not really, actually—okay I guess some things would be broken by having a more explicit goal statement. The uncanny valley of explicitness of goal statements. My naive first approximation model is that the purpose of the Department of Education is to increase the budget and powers of the Department of Education. I’m assuming it also wants to provide its prisoners students with some combination of education, daycare, indoctrination, and ability and class indicators for signaling, although people are way better at following their incentives than I would expect. It’s possible that the majority even of people who will see this blog post expect people to be more rational than they actually are and I should never tell anyone that people are more rational than I expect them to be lest they update in that direction and end up being even more deluded about the standard human level of rationality.

Rules like “don’t flap your hands” generally don’t serve much purpose; large-scale hand-flapping would not bring the collapse of civilization. Rules like “sit still for 50 minutes and do the task at hand, then switch to the next task we have assigned” aren’t really generally useful for education. Rules like “turn in the homework on time” are, well, that’s probably a horribly wrong way to do education in general but within the domain of the Prussian education system—and there’s an issue with criticizing the education system where you inevitably reach “actually, we should burn everything we have to the ground and build anew”—they do serve a purpose, so lifting them would require some indication that their presence/punishing based on these rules does not cause the person in question to do their homework anyway. That creates an obvious way to exploit the system which would be relevant if this was an actual policy proposal, but as it is not doesn’t really matter. Accommodations like “printing the test in larger font”, on the other hand, should (where “should” is like “it is philosophically sound for it to be so”) be available to everyone on request, without need for diagnosis. There may be things in the category of accommodations which technically ought to be available to everyone which are financially/temporally infeasible.

What Bakker is getting at is basically a philosophical problem, and not one which is hard to solve.


I am nihilsupernum from tumblr and twitter (audiences thereof should not mingle any more than they already have, the set of things I can say is larger when they are separate). The blog title is an anagram of ‘nihilsupernum’. I have a burning urge to have everything I believe written out coherently in one place. This isn’t particularly feasible, but I am nonetheless harnessing that urge into making a real blog. My current mode is just telling people things I think of in several-hundred-word rants and then never returning to them, which is probably actually fine but feels deeply wrong. Adjusting from this will probably cause earlier posts to be short and with a lot of assumed knowledge. I can’t really predict what this will be about; I am interested in some cloud of things around sociology/politics/philosophy/cognitive science/anthropology/history, which are near each other in memespace for some reason. This post was unnecessary but I must carry on the tradition of starting with a post exactly one meta level up though I know not whence it cometh. Onwards!