Wednesday, June 27, 2007

The (Bayesian) Advantage of Youth

File this under What Sustains Capitalism, along with that story several years ago about how something like 70% of Americans think they will someday be in the top 10% of income earners (thus the reluctance to tax the rich, whom Americans see as their future selves). It seems to me that Capitalism is best suited for young risk takers who do not study their history. Because it is suited to the young, offering them hope, it is self-sustaining. Because it is suited to risk takers, it is pre-eminently a male hazard (if you accept conventional evolutionary psychology). And because it is suited to those that don't study history, it is pre-eminently Anti-Intellectual.

May 19, 2007
The (Bayesian) Advantage of Youth
Posted by Clay Shirky
A couple of weeks ago, Fred Wilson wrote, in The Mid Life Entrepreneur Crisis “…prime time entrepreneurship is 30s. And its possibly getting younger as web technology meets youth culture.” After some followup from Valleywag, he addressed the question at greater length in The Age Question (continued), saying “I don’t totally buy that age matters. I think, as I said in my original post, that age is a mind set.”
This is a relief for people like me — you’re as young as you feel, and all that — or rather it would be a relief but for one little problem: Fred was right before, and he’s wrong now. Young entrepreneurs have an advantage over older ones (and by older I mean over 30), and contra Fred’s second post, age isn’t in fact a mindset. Young people have an advantage that older people don’t have and can’t fake, and it isn’t about vigor or hunger — it’s a mental advantage. The principal asset a young tech entrepreneur has is that they don’t know a lot of things.
In almost every other circumstance, this would be a disadvantage, but not here, and not now. The reason this is so (and the reason smart old people can’t fake their way into this asset) has everything to do with our innate ability to cement past experience into knowledge.


Probability and the Crisis of Novelty


The classic illustration for learning outcomes based on probability uses a bag of colored balls. Imagine that you can take out one ball, record its color, put it back, and draw again. How long does it take you to form an opinion about the contents of the bag, and how correct is that opinion?
Imagine a bag of black and white balls, with a slight majority of white. Drawing out a single ball would provide little information beyond “There is at least one white (or black) ball in this bag.” If you drew out ten balls in a row, you might guess that there are a similar number of black and white balls. A hundred would make you relatively certain of that, and might give you an inkling that white slightly outnumbers black. By a thousand draws, you could put a rough percentage on that imbalance, and by ten thousand draws, you could say something like “53% white to 47% black” with some confidence.
This is the world most of us live in, most of the time; the people with the most experience know the most.
But what would happen if the contents of the bag changed overnight? What if the bag suddenly started yielding balls of all colors and patterns — black and white but also green and blue, striped and spotted? The next day, when the expert draws a striped ball, he might well regard it as a mere anomaly. After all, his considerable experience has revealed a predictable and stable distribution over tens of thousands of draws, so no need to throw out the old theory because of just one anomaly. (To put it in Bayesian terms, the prior beliefs of the expert are valuable precisely because they have been strengthened through repetition, which repetition makes the expert confident in them even in the face of a small number of challenging cases.)
But the expert keeps drawing odd colors, and so after a while, he is forced to throw out the ‘this is an anomaly, and the bag is otherwise as it was’ theory, and start on a new one, which is that some novel variability has indeed entered the system. Now, the expert thinks, we have a world of mostly black and white, but with some new colors as well.
But the expert is still wrong. The bag changed overnight, and the new degree of variation is huge compared to the older black-and-white world. Critically, any attempt to rescue the older theory will cause the expert to misunderstand the world, and the more carefully the expert relies on the very knowledge that constitutes his expertise, the worse his misunderstanding will be.
Meanwhile, on the morning after the contents of the bag turn technicolor, someone who just showed up five minutes ago would say “Hey, this bag has lots of colors and patterns in it.” While the expert is still trying to explain away or minimize the change as a fluke, or as a slight adjustment to an otherwise stable situation, the novice, who has no prior theory to throw out, understands exactly what’s going on.
What our expert should have done, the minute he saw the first odd ball, is to say “I must abandon everything I have ever thought about how this bag works, and start from scratch.” He should, in other words, start behaving like a novice.
Which is exactly the thing he — we — cannot do. We are wired to learn from experience. This is, in almost all cases, absolutely the right strategy, because most things in life benefit from mental continuity. Again, today, gravity pulls things downwards. Again, today, I get hungry and need to eat something in the middle of the day. Again, today, my wife will be happier if I put my socks in the hamper than on the floor. We don’t need to re-learn things like this; once we get the pattern, we can internalize it and move on.
A Lot of Knowledge Is A Dangerous Thing
This is where Fred’s earlier argument comes in. In 999,999 cases, learning from experience is a good idea, but what entrepreneurs do is look for the one in a million shot. When the world really has changed overnight, when wild new things are possible if you don’t have any sense of how things used to be, then it is the people who got here five minutes ago who understand that new possibility, and they understand it precisely because, to them, it isn’t new.
These cases, let it be said, are rare. The mistakes novices make come from a lack of experience. They overestimate mere fads, seeing revolution everywhere, and they make this kind of mistake a thousand times before they learn better. But the experts make the opposite mistake, so that when a real once-in-a-lifetime change comes along, they are at risk of regarding it as a fad. As a result of this asymmetry, the novice makes their one good call during an actual revolution, at exactly the same time the expert makes their one big mistake, but at that moment, that’s all that is needed to give the newcomer a considerable edge.
Here’s a tech history question: Which went mainstream first, the PC or the VCR?
People over 35 have a hard time even understanding why you’d even ask — VCRs obviously pre-date PCs for general adoption.
Here’s another: Which went mainstream first, the radio or the telephone?
The same people often have to think about this question, even though the practical demonstration of radio came almost two decades after the practical demonstration of the telephone. We have to think about that second question because, to us, radio and the telephone arrived at the same time, which is to say the day we were born. And for college students today, that is true of the VCR and the PC.
People who think of the VCR as old and stable, and the PC as a newer invention, are not the kind of people who think up Tivo. It’s people who are presented with two storage choices, tape or disk, without historical bias making tape seem more normal and disk more provisional, who do that kind of work, and those people are, overwhelmingly, young.
This is sad for a lot of us, but its also true, and Fred’s kind lies about age being a mind set won’t reverse that.
The Uses of Experience
I’m old enough to know a lot of things, just from life experience. I know that music comes from stores. I know that you have to try on pants before you buy them. I know that newspapers are where you get your political news and how you look for a job. I know that if you want to have a conversation with someone, you call them on the phone. I know that the library is the most important building on a college campus. I know that if you need to take a trip, you visit a travel agent.
In the last 15 years or so, I’ve had to unlearn every one of those things and a million others. This makes me a not-bad analyst, because I have to explain new technology to myself first — I’m too old to understand it natively. But it makes me a lousy entrepreneur.
Ten years ago, I was the CTO of a web company we built and sold in what seemed like an eon but what was in retrospect an eyeblink. Looking back, I’m embarrassed at how little I knew, but I was a better entrepreneur because of it.
I can take some comfort in the fact that people much more successful than I succumb to the same fate. IBM learned, from decades of experience, that competitive advantage lay in the hardware; Bill Gates had never had those experiences, and didn’t have to unlearn them. Jerry and David at Yahoo learned, after a few short years, that search was a commodity. Sergey and Larry never knew that. Mark Cuban learned that the infrastructure required for online video made the economics of web video look a lot like TV. That memo was never circulated at YouTube.
So what can you do when you get kicked out of the club? My answer has been to do the things older and wiser people do. I teach, I write, I consult, and when I work with startups, it’s as an advisor, not as a founder.
And the hardest discipline, whether talking to my students or the companies I work with, is to hold back from offering too much advice, too definitively. When I see students or startups thinking up something crazy, and I want to explain why that won’t work, couldn’t possibly work, why this recapitulates the very argument that led to RFC 939 back in the day, I have to remind myself to shut up for a minute and just watch, because it may be me who will be surprised when I see what color comes out of the bag next.
Src: http://many.corante.com/archives/2007/05/19/the_bayesian_advantage_of_youth.php

No comments: