11 Jul 2001 deven   » (Journeyer)

Why is it that you can claim to be God, Satan or Jesus and get certified as a Master without an uproar, but being honest and forthright gets you reamed? This sort of animosity is supposed to help grow the free software community?

hacker, I never requested anyone to certify me as a Master, or as anything else for that matter. Yet ncm singled me out as an example of so-called "abuse" of the system, claiming that I "requested" Master certification. I did nothing of the sort!

I've never asked anyone to certify me -- I have complained that the certification system seems to be broken when I used to be certified and now I'm not, despite having more than a few listed certifications. That complaint was because I was worried that the system might be malfunctioning unnoticed -- I wasn't trolling for additional certifications. However, when more people certified me, I thanked them (and you) for it. But did I request those certifications? Never.

I have previously listed sixteen orthogonal concepts that people seem to want this single certification metric to convey, and most of these can be derived or inferred from Raph's description. Since it obviously can't squeeze sixteen dimensions into one, I chose (somewhat arbitrarily) to pick just one dimension (level of programming skill) to use for my certifications. I may not have the same background and experience as another given individual in terms of what code and tools I've worked with, but when it comes to innate programming ability, I'll go up against most people any day.

Give me all sixteen dimensions individually and I'll be glad to pick the best choice on each dimension. Maybe I should have done them all and averaged it, I don't know. I didn't think it would matter, because I didn't think anyone would care what I put for myself. If people really trusted the system to work correctly, nobody would have made an issue of it in the first place.

People talk about Advogato as if it has a "web of trust", but that is extremely misleading. PGP has a web of trust, but using that terminology here misleads people, because that's not what Advogato has. Instead, this truly is a computerized "good ol' boys" network, even if unintentionally. I believe raph has stated (though I can't remember where) that Advogato was arbitrarily focused on free software because he liked free software and wanted to support it, but that the true purpose of creating the site was as a testbed for his research, to see if the certification system would effectively weed out the spammers and trolls while allowing legitimate users to discuss free software without all the noise (from spammers and trolls).

While it's been pretty effective at that goal, it's not clear that it's been as good about letting legitimate participants in, especially recently. I suspect the chosen algorithm may not scale as well in practice was it was intended to, because the more people and more certifications there are, the more those certifications seem to be diluted. Raph states: "The trust metric used to evaluate the certificates is most robust when the certificate graph is dense." In practice, it seems to be excellent at keeping the "good ol' boys" in, and most others out. Since most of Raph's writings here seem to focus on effectiveness in keeping out the bad people, it's not clear whether he ever paid close attention to the flip side of the coin, letting in the good people.

I just downloaded a copy of the people page, and ran a few counts across it. The majority of the accounts are uncertified! Out of 4,760 total accounts, 3316 (69.7%) are uncertified, greyed out as "Observer". Only 322 (6.8%) are certified as "Apprentice". Another 798 (16.8%) are certified as "Journeyer". Finally, 324 (6.8%) are certified as "Master". More people are certified as Master than Apprentice! Last year, that page was mostly green with a little grey, today it's mostly grey with a little green.

Does this make sense? Are there really over 3,000 spammers and trolls here? The barbarians are at the gates? I find this very difficult to believe. It seems much more likely that the certification system is broken, since it's keeping out more than twice as many people as it's letting in. Maybe the implementation is flawless and the algorithm itself isn't scaling well. If the goal was primarily to keep out spammers and trolls (as it seems to be), but it's keeping out the majority of potential participants, isn't that cause for concern? Shouldn't this be investigated, especially if the algorithm isn't scaling as expected?

I complained about the certification system to try to draw attention to this; it doesn't seem to be behaving as intended. Of course, the "good ol' boys" who are solidly entrenched in the system are happy to say that it's working just fine, thank you. On the other hand, Raph says that "the metric is being pickier than usual, almost certainly too picky. Apologies to those who should be cert'ed but aren't."

Raph designed and implemented this certification system, and he isn't convinced it's working right. Isn't it ironic that I get vilified for daring to question it? If everyone removed their certifications from me, and I was left uncertified because of that, I would have been disappointed, but I wouldn't have complained that the system was broken. However, the certifications that used to be quite sufficient were no longer, and that seemed amiss, so (after giving it a month), I tried to draw some attention to it by complaining more vocally. I've been unfairly attacked in return, but at least Raph has noticed that something is amiss with the certifications. Hopefully he'll pay more attention to this issue now, if he can find the time...

Latest blog entries     Older blog entries

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!