Advogato: Trust me on this ...

Posted 4 Oct 2002 at 14:52 UTC by garym

Trust and trustworthiness online is a subject of some concern to those of us who live the life of telework and dream of internetworking ad-hoc business-webs of entrepreneurial free-agent nanocorps. The highest trust metric comes from complete identity, but that's only feasible in tribal groups, and it's not obvious how this is going to work in the online global village.

Hokkaido University's Toshio Yamagishi has thought about this too: His latest study, "Improving the Lemons Market" (pdf), suggests positive reputation systems, while slower to build than fault metrics, remain more stable and can build to rival trust qualities of complete identity.

Of course, here at Advogato, we already knew this, but as an exercise in a practical application of Yamagishi's results, I'd like to propose an enhancement to our local trust metric...

The Advogato Observer/Master reputation scale, while useful, is missing an important element: We trust most those sources we trust. Instead of the first degree-of-freedom of X-rates-Y, what if we used a compounding metric? I'm more likely to value the rating of 'Master' bestowed on someone if I know that rating was given by someone with 'Master' status, and I'm most likely to trust the rating if given by someone I rated as 'Master'.

The latter is probably impractical within the engines of Advogato, but what about the simpler scale? Would it be useful to post a trust metric that is weighted by the trust metric of the member bestowing the honour?

Which ever way it goes, I'd also like to propose an event. In these times of tight job markets, all of us could benefit from having some source we can point to as a professional reference. I propose we engage in a Rate Your Peer Day where every member of Advogato takes just 20 minutes to comb through our membership and rate at least 5 other members; if you have an account at Sourceforge, repeat the exercise there. Not only would it form an interesting application of the Yamagishi results, but it also never hurts to know a bit more about your fellow members.

Re-reading How the Trust Metric Works, I see where what I've described and what exists is really closer than I'd thought, only that it misses the personalization where the trust I place on someone may be different than the trust you place on the same person. Advogato's algorithm seems more designed to prevent flooding attacks.

So now my proposal is a lot fuzzier ;) ... what I want to see is "if I trust X, then how much should I trust Y?" and from the description, it does seem like that metric is in there, it's just not tapped. Comments?

Your idea of an Advogato site-wide certi-fest strikes me as a cool community-building exercise. Advogato would just be another blogging service without the trust metric system, which gives the site a closely integrated community: you notify others that you are watching their efforts, they do likewise. We all see our ranks in splendorous pastel colors. It's a lovely experiment.

It seems like we're guided by short individual behaviors in our activities on Advogato. If we see bad behavior and inane discussion on recentlog, we rate down and pluck off the certifications. If an exquisite article is posted, we applaud for the Apprentice and select Journeyour this time. Certainly these are good means to build or loose trust. The best form of trust is when I see fellow developers who know and work with each other closely and are able to certify based on real experience.

How cool would it be to see more group activity to inspire real experience with other developers? To bring in other dynamics to help newcomers find a means of building trust. Maybe it's already there, but I haven't participated enough. I suppose if you put effort into meeting others on Advogato by visiting their projects, then you could shack up to code with them, show your abilities, pontificate on the universe with them and at the end of the day mark each other purple or blue.

is not easy at all. I learned this the hard way. While we all have our opinions, it doesn't mean others will agree or even be able to accept our opinions. That aside, I don't rate as often as I should. Why? I just never think of it. Nothing personal. The "just another blogging" comment really struck a chord with me, because I often wonder what the point of posting here is (my comments included). Too often I see diary entries where Nerd A posts "I wrote a quick and dirty bla-bla-bla". Anybody can write quick-and-dirty programs. Not just anybody can write well designed programs. Hell, I can't do that either, but I think I just don't have the mathematical thought processes (background) to allow me to do write well organized and effective code. I'm off the beaten path again :( Anyway, I am most impressed when Nerd B has a site upon which he or she has written and posted programs for download, etc. Those are the folks I am most likely to rate upwards. The ones who just use fancy techno speak won't earn positive votes from me.

In the "e-business network" site Ecademy.com, they show a display where you can list the people you have rated (in their language, "added to contact list" but it means the same) and from there you can see the list of people they have rated. This lets you drill down one degree of freedom at a time. There's a lot I don't like about Ecademy, but this may be one feature they have right and one were we can steal ... er ... pay homage to the idea.

I sat down today to do as I suggested and rate 5 people, and I discovered it was not such an easy thing: You don't have enough information to just go randomly investigating blogs and diaries and rate based on that tiny bit of knowledge. So instead, since I've been around a few years, I went looking for people I knew, people I'd either worked with at companies or in open source projects. I started with the projects page (which could do with a better presentation IMHO ;) and then with the people page (which is worse but like who's complaining?) and from there I found my 5 people.

Given a cascade display, where I could hook in on Dan and see who he knew to see if I knew any of them, that might simplify the exercise. I'd still be left rating only those people who I know to be of certain skill, which is really how it should be ... within 6 degrees of freedom, any good opensource developer probably connects to every other good opensource developer anyway!

slow down, posted 7 Oct 2002 at 13:58 UTC by Malx »

Have you think that all trust-algorithms are slowing progress?

Internet could thank friendly-style of software and protocols for fast development. Just think about lot's of "access denied" you could encounter to write any new http server or web browser if it was not so.

If you think this is nonsense - you are wrong. I know HTTP server wich works only for certain types of web browsers and block access for all others (especially telnet accessd). That means if you make some error in HTTP while accessing it thins you are intruder and blocks you.

Ok it may seem to be distant from trust-metrics, but just think that software certification is basicly the same as person certifications. And you could set "I trust this software is GOOD".

And .. It whould be really hard to get into some community if it is trust-enabled :) Same as with science now.

I think that most of what is wrong with Advogato's trust metrics is lack of communication about certifications. If the certifications were moved to a separate page, new certifications asked for a couple of lines explaining the certification, and each user had a publically available certification log with this information, we would see a big improvement in the quality of advogato trust metrics.

For garym's purposes, I think what would be helpful would be having some mechanism whereby different certifying communities could exchange this information, allowing a sort of trust-metric network of trust-metric networks.

I think what some people are feeling around for are 'multi-dimensional trust metrics', or even better - trust metrics built on a quantale which is a little more interesting than the positive real numbers.

An immediate advantage of multi-dimensional trust metrics is that you can rate different qualities simultaneously. For example, you can rate someone's theoretical skills, or their skills in Perl, or Python. Then, if you like, you can project these skills onto a single scale in any way that you like - for example, you may be interested in people who are good at Perl and Python so you can take PERL+PYTHON or PERL*PYTHON, or some other increasing function of the individual variables as your measure of how good this person is on the scale.

Why do we need to think about more than just multiple dimensions (which could just as easily be some with several separate metrics)? Here's an example:

In the case of a rating system (like Advogato)- the fact that I trust someone's programming skills doesn't necessarily mean that I trust them to be a good judge of the qualities/trustworthiness of others. This is relevant because the degree to which I trust the people that they trust is actually determined by how much I trust their ability to rate people. Thus the transitivity of trust in this system is not simply the 'product' of the transitivity of two (uninteracting) metrics.