Survivor Bias

Sometimes when I’m presenting to a group on personal safety, there will be an individual who will claim that the advice I’m giving doesn’t match up with their experience(s) e.g., if I advise that you should always lock your front door, they will state that they always leave it unlocked and in 30 years nobody has ever broken into their house etc. In one sense it is hard to argue with someone’s experience, even if it only constitutes a sample size of one. This is one of the issues with “Survivor Bias”, we use our experiences of winning/success to blind us to the reality of our situations e.g., each time we leave our house unlocked, and it isn’t broken into we count it as a win, and this “win” blinds us to the fact that in other parts of town offenders are taking advantage of other situations where homeowners have forgotten to lock their doors etc. The first recorded instance of the idea/concept of survivor bias comes from a conversation between the Greek philosopher, Diagoras (5^th Century BC), and a friend. Diagoras was an atheist who was challenged concerning his lack of belief in the gods by a friend who pointed to all the paintings in a temple showing sailors in a violent storm praying to the gods to save them. Diagoras’s friend made the argument that because these sailors survived it was proof that not only did the gods exist but that they had answered the sailors’ prayers. Diagoras responded by asking where all the paintings of the sailors were who were shipwrecked and/or were drowned i.e., didn’t survive. When we only look at the survivors or focus on our successes, we are only getting half the story and seeing half the picture. Whenever someone demonstrates a survivor bias concerning crime and violence, it is worth coming back to Routine Activities Theory (RAT) - that for an offense to take place there must be a motivated offender, a suitable victim, and the lack of a capable guardian. If a motivated offender is lacking then an offense can’t take place e.g., if you leave your door unlocked and nobody is motivated to exploit this vulnerability then you won’t suffer a break-in, however it would be incorrect to assume that somewhere else this isn’t happening i.e., you just got lucky.

Certain types of entertainment rely on the existence of a survivor bias. I remember as a kid watching the illusionist/psychic Uri Geller on a panel show bend spoons and perform telekinetic “feats” and being completely taken in. Just before a commercial break he asked people at home with broken watches to hold them up to their TV screens, and he would attempt to get them working again; anyone with a success should phone the TV station and tell them. Geller was playing a simple game of chance – if enough people/viewers start moving a stopped clockwork/mechanical device around a very small percentage are likely to start working again, even if this is just temporary. Geller only needed this to happen for a few viewers who were motivated enough to phone in – and if they had engaged in the action of holding their broken watch up to the TV set, they had probably already bought in. After the commercial break, Geller claimed a success; some broken watches had started, and some people had phoned in. As an eight-year-old who wanted to believe that ESP (extra-sensory-perception) existed he’d sold me. However, like with Diagoras, there was no record of the unsuccessful. As a practitioner of a reality-based self-defense system, I am always encouraged when I hear an account of someone, military or civilian, getting their Krav Maga to work in a real-life violent encounter. However, I am more interested, from a technical perspective when something doesn’t work, or doesn’t seem to be effective i.e., I don’t want to develop a survivor bias around what I practice and teach, only listening to the success stories and the “wins”. As much as I believe in the Krav Maga approach and accept some of its apparent shortcomings in some very specific areas are a necessary “cost” in order for the overall approach to be successful/work, I try not to simply accept the relevance of something based on a few success stories; over-promote the wins without acknowledging the losses. Understanding when and why things go wrong is a necessary first step in evolving and developing the method, to ignore this is to allow the system to stagnate – and eventually become irrelevant. As an instructor it is extremely gratifying to see students make something work (have a success) when under pressure, however if there were some common failings amongst students to get there, then it is an opportunity to review the teaching/education process and find ways to expedite the learning process i.e., it is easy to get rewarded by the successes, and not examine the failures that took place to get there.

There are few areas of our lives not affected by the survivor bias. Academic research which presents itself as being “honest” often suffers because of it e.g., a piece of research that failed to prove something is less likely to be submitted and be published than one that does. However, this doesn’t make the research irrelevant because it might prove something else. For my criminology master’s degree thesis, my basic hypothesis was that burglars travel further to commit their offenses than muggers i.e., those who engage in street robberies. The data didn’t support this, and my “idea” was proved wrong e.g., there were those who committed burglaries in their own apartment blocks, and there were muggers who travelled quite far to specific locations, such as certain shopping malls, to commit their crimes. However, there were so many other “ideas” that sprouted out of the hypothesis that couldn’t be proven that it would be incorrect to say the research project produced no “findings” or other questions to investigate e.g., were there certain types of offenders, both burglars and muggers, who commuted to commit their offenses, and others who were purely opportunistic (professionals versus amateurs)? Sometimes by focusing purely on the successes and failing to look and learn from the losses – falling foul of the survivor bias – we stop questioning, and this can eventually lead to a failure to evolve and develop.