15 Oct 2013 mjg59   » (Master)

Hacker News is a social echo chamber

Hacker News is a fairly influential link aggregation site, with stories submitted and voted on by users. As explained in the FAQ, the ranking of stories is roughly determined by the number of votes divided by a function of the time since submission. It's not a huge traffic driver (my personal experience of stories on the front page is on the order of 30,000 hits), but it's notable because the demographic tends to include a more technically literate and influential set of readers than most other sites. The discussion that ensues from technical posts often includes meaningful feedback from domain experts. Stories that appear there are likely to be noted by technology workers, especially in the Silicon Valley startup field[1].

That rather specific demographic appears to correlate with other traits. There's a rather more techno-libertarian bias on Hacker News than on most general discussion forums, which is consistent with the startup-oriented culture that it springs from - the desire to provide disruptive solutions to real world problems tends to collide with existing regulatory frameworks, so it's unsurprising that a belief in individual rights and small government would overlap with US startup culture. There's a leaning towards the use of web technologies rather than traditional client applications, which matches what people are doing in the rest of the world. And there's more enthusiasm for liberal open-source licenses over Copyleft licenses, which makes sense in a web-focused environment (as I wrote about here).

Now, personally I'm a big-government, client-app, Copyleft kind of person, but for the most part I don't think the above is actively dangerous. It's inevitable that political views will vary, we'll probably continue to cycle between thick and thin clients for generations and nobody's ever going to demonstrably prove that one licensing model deserves to win over another. But what is important is that the ongoing debates between these opinions be driven by facts, and that it remain obvious that these disagreements exist. As far as technical (and even political) discussion goes, Hacker News doesn't seem to
have a problem with that. Disagreeing with the orthodoxy is tolerated.


This seems less true when it comes to social issues. When a posting discussing the myth of the natural born programmer[2] hit the front page, the top rated comment is Paul Graham[3] off-handedly discounting the conclusions drawn. The original story linked to a review of peer-reviewed scientific research. Graham simply discounts it on the basis of his preconceptions. Shortly afterwards, the story plummeted off the front page, now surrounded by stories posted around the same time but with much lower scores.

How does this happen? There's two publicised methods which can result in stories dropping down the order. Users with high karma scores (either via submitting popular stories or writing popular comments) are able to flag submissions, and if enough do so then a negative weighting is applied to the story. There's also a flamewar detector, a heuristic that attempts to detect contentious subjects and pushes them off the front page.

The effect of both is to enforce the status quo of social beliefs. Stories that appear to challenge the narrative that good programmers are just naturally talented tend to vanish. Stories that discuss the difficulties faced by minorities in our field are summarily disappeared. There are no social problems in the technology industry. We have always been at war with Eastasia.

This isn't healthy. We don't improve the state of the software industry by hiding stories that expose conflicts. Flamewars don't solve problems, but without them we'd be entirely unaware of how much of a victim blaming mentality exists amongst our peers. It's true that conflict may reinforce preconceptions, causing people to dig in as they defend their beliefs. However, the absence of conflict does nothing to counteract that. If you're never exposed to opinions you disagree with, you'll never question your existing beliefs.

Hacker News is a privately run site and nobody's under any obligation to change how they choose to run it. But the focus on avoiding conflict to such an extent that controversial stories receive less exposure than ones that fit people's existing beliefs doesn't enhance our community. If we want to be able to use technology as an instrument of beneficial change to society as a whole, we benefit from building a diverse and welcoming community and questioning our preconceptions. Building a social echo chamber risks marginalising us from the rest of society, gradually becoming ignored and irrelevant as our self-reinforcing opinions drift ever further away from the mainstream. It doesn't help anybody.

[1] During the batch of interviews I did last year, two separate interviewers both mentioned a story they'd read on Hacker News that turned out to have been written by me. I'm not saying that that's what determined a hire/don't hire decision, but it seems likely that it helped.
[2] The article in question discusses the pervasive idea that some people are inherently good programmers. It turns out that perpetuating the idea that some people are just born good at a particular skill actually discourages others from even trying to learn it, even those who are just as capable.
[3] One of the co-founders of Y-Combinator and creator of Hacker News.

(Edited to fix a footnote reference)

comment count unavailable comments

Syndicated 2013-10-15 14:11:21 from Matthew Garrett

Latest blog entries     Older blog entries

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!