Lifespan of Social Metrics
Unlike metrics of simple systems, the metrics of complex adaptive systems have a life span. They grow in value over time. And then they decay in value as well.
Take twitter. At first it was follower count – and that still matters some. But over time, people found ways to gain followers that didn’t coincide to the value that they provided, so it started to lose usefulness as an indicator of value from the tweeter.
Jane McGonigal, @avantgame, just posted a series of tweets about how Amazon rating system is being gamed.
Interesting, my book seems to have been targeted last year by some conservative group, individuals encouraged to post negative reviews
a cluster of extremely negative reviews with a conservative POV posted at the same time with weird (untrue) criticism of my biography…
here’s a recent NYT article on partisan groups attempting to one-star bomb books on Amazonnytimes.com/2013/01/21/bus… …
“Here’s what I do: I go to Amazon.com + search for ‘liberal book’. I give 1 star, 1 star, 1 star”youtube.com/watch?v=tGB8Uu…
“Then I search ‘conservative book’ and give 5-star, 5-star, 5-star.” From a tea-party Internet training meetup youtube.com/watch?v=tGB8Uu…
apparently one trick is to purchase book so your review appears “verified”, then cancel order before books ships
Note: I took out a few tweets in the series which were just about her work, rather than about the gaming the stars method she points to.
Knowing that people are giving a count of stars based on their ideology rather than the quality of the work, I am now less likely to put stock in the star count on a given project, especially when it is politically polemic material. Thus the usefulness of the stars decrease for me (and for others) and then for the whole system.
With twitter, we first looked at “follower” count, and then that was gamed. Then the metrics had to start considering other factors like RT count to demonstrate influence. Now we rely on twittergrader, kred, klout, etc to take a great number of factors into weighted consideration to produce an “influence” or reputation score. This amalgam of factors evolves over time.
You might think this is just the way of social media, since it is a fast feedback loop. But actually, I think that a lot of measures of complex adaptive systems work this way.
What comes to mind is Scott Nelson’s story from Blockage in the Thrivability Sketch.
…It was during the crisis of 1857 that the previously ignored insights of a long-haired mathematician, abolitionist, and utopian socialist named Elizur Wright were finally recognized as critically valuable for economic stability.
In the 1840s and 1850s, Wright had tried to convince the state of Massachusetts that life insurance needed reform. As a mathematician, he had been asked calculate the present value of any given policy based on the premiums paid in, a calculation that British mathematicians had called impossible. He created a rule-of-thumb called “net present value” (NPV) to determine the value of a flow of resources in a single instant (present value) and then to subtract operating costs (net).
But the more Elizur calculated, the more troubled he became. Many companies by his calculations spent so much on advertising that they could never pay off their policies. Others profited by canceling policies for those who missed a single payment. The effect was often to end a policy a year before death, leaving families with nothing. Wright fumed, but in vain. In the go-go 1840s and early 1850s, no one would listen to his criticisms and only a few would accept his principle of valuation. But through the 1850s he returned to the Massachusetts legislature with a blueprint for reform. When the Panic of 1857 hit with the failure of a bank called Ohio Life Insurance and Trust Company, Elizur was prepared. This blockage of trade and transport, Wright declared, was a result of distrust. Insurance companies needed reliable accounting practices that would allow Massachusetts to calculate net present value, and internal rate of return. When trust returns, Wright assured them, the blockage will be over.
Unconvinced but without options, Massachusetts adopted Wright’s blueprint, preventing any company from selling insurance in Massachusetts that did not provide complete financial information. NPV offers transparency of obligations. The panic was short-lived, and Elizur Wright’s accounting principles became the basis of what we now call Generally Accepted Accounting Principles, adopted by millions of companies, states, and non-governmental organizations throughout the world. MBAs take credit for it, but a long-haired radical gave us cost accrual accounting.
Wright took advantage of blockage to identify its root cause – a distrust of opacity. Increased financial transparency was the solution; trust collapses without it. Blockage can let us make institutions open up and make them thrivable.
If the metrics we use in our economy are also being created (even at a very slow pace) they may also be declining in usefulness. Elizur’s methods didn’t anticipate the complex financial instruments to come over 100 years later that obscured the “trustworthiness” of the things our measures aim to reveal.
Consider how the measures you use can be gamed, where they may be in their lifespan of adoption and decay, and what other indicators might be emerging to reveal what matters – the territory and not the map.
