Our reader Jonathan Campbell has an excellent post on a bizarre example of legal “reasoning”. A defendant fires 95 shots into the woods; his friend fires 5. A man in the woods is hit by a bullet and dies. The defendant is 95% certain to be the killer (in other words, he is the killer beyond a reasonable doubt), but he is acquitted because the nature of the evidence is “statistical” and therefore cannot be the basis of a conviction.
This tends to confirm my suspicion that a legal “education” consists primarily of having your capacity for logic beaten entirely out of you. As Jonathan points out, all evidence is statistical in exactly the same sense that this evidence is. I put a bullet through your heart and you take five minutes to die. Did the bullet cause your death? Well, not certainly — there’s some small chance that you died of a heart attack that would have killed you anyway, thirty seconds before the bullet was able to finish its job. Is there a high probability I caused your death? Yes. Did I do so beyond a reasonable doubt? Yes. Is the remaining doubt of a statistical nature? Yes. So if there were any consistency in the law, of course I’d be acquitted. And of course I would not be. Which means, as has been noted in this space before, that the law is an ass. So, perhaps, are a substantial fraction of its practitioners.
It’s off the main topic, but let me take this opportunity to present a related conundrum: Two identical twins each kill a random stranger. The two murders take place simultaneously, in different locations, but each in full view of 50 witnesses. There is no further evidence regarding which twin was likely to be in which location. There is therefore no reasonable doubt that each twin committed a murder, but only a 50% chance that any given twin committed either particular murder. Question 1: Can we convict them of anything under current law? My guess is no. (I don’t blame the law for getting this one wrong; I’m sure it must be impossible to write a law that works well under any imaginable scenario.) Question 2: How could the law be rewritten so that we can convict them, but so that the rewrite does not undercut the usual reasonable doubt standard?
What ever happened to joint and several liability?!
As a law student I am both mildly offended and somewhat agree with you.
There are some decent reasons for legal professionals to not rely much on statistical evidence.
1) The vast majority of time they’re dealing with stuff that simply is unquantifiable. Ie, it’s very unusual for a legal case to have the kind of reliable premises that your examples presumes. Usually someone claims they drew 2 red balls, 10 white from a selection of an unknown number of urns. And quite possibly he’s lying about where he got those balls.
2) The legal system by far predates statistical modelling. Only changing that system gradually has a lot of benefits.
It might be ugly, but we have a pretty good grasp of how it works. Changing it to a more consistently logical system might have some very large downsides in terms of unpredictable effects.
3) There usually is possible to bring in experts to talk about stastical evidence.
4) Learning the law takes enough time that it might be wasteful to cram a scientific education in there as well. Additionally, having a bunch of lawyers with the equivivalent of stats 101 might not be a good thing.
It is called beyond *reasonable* doubt. Of course, reasonable is not defined, but if your define it to be less than 5 percent, the illogical reasoning is no more.
There’s a relatively famous case where a man drugged a woman and tried to date rape her. The drug killed her, and so he was having sex with a corpse. The judge struggled with whether he had the requisite “actus reus” and the “mens rea”. The “actus reus” is the action, the “mens rea” is the intent to do the action.
The problem was that he didn’t intend to commit necrophilia (didn’t have the mens rea) but that’s what he did (actus reus). He did mean to rape someone (he had the mens rea for that crime) but he didn’t actually rape a person (on the basis that one can’t rape a corpse).
I think he was found guilty in the end, but I forget the details.
1. This is a trolly-bus problem with the attendant issues of “rejecting the hypo[thesis]”.
In this case, what about ballistics evidence? Testimony as to direction fired in? Culpability of victim in not **lying down** when he heard the bullets going by?
2. As you pointed out before, most LEGAL scholars put “beyond reasonable doubt” at north or 90%, and as you observed, many north of 95%. Personally I like 95%. I suspect this result looks odd to you mostly because you like a lower figure. If it was 97 bullets to 3 I would be happy to convict.
On the twin problem, again, reject the hypo. There is no way in hell that they both killed someone at the exact same time without colluding. So they would both be convicted of BOTH murders.
#1) if a jury of peers is responsible for the acquittal/conviction then i believe the answer is yes.
#2)if the answer to #1 is yes, then the legislation against murder need not be rewritten at all.
#2a)if the answer to #1 is no, we should just flip a fair coin. if heads acquit, if tails convict. i figure this strategy may let a few criminals slip past, but it should most certainly catch them on a second pass.
I think that you are misstating the legal argument. There is no percentage cutoff for reasonable doubt, and there is no prohibition of statistical proof. Prosecutors give statistical arguments all the time. The problem in this case is that there is an insufficient causal connection between the shooter and the death.
For your second conundrum, if the prosecutor thinks that the twins conspired, they could be prosecuted together, against the prosecutor’s best guess of what happened. The jury is unlikely to acquit just because an equally bad murder scenario is possible.
Interesting post, Prof. Landsburg.
Regarding the bullet in the woods, I think most people reject the idea that someone “probably” killed someone else. That’s not strong evidence, no matter what the probability is. My opinion is that the man was tried for the wrong crime (murder) when he should have been tried for a series of other crimes (reckless endangerment, manslaughter, criminal negligence, etc. etc.). The sum total of punishment for those lesser crimes would be more than enough to bring the man (and his friends) to justice. So lawyers are an ass, but in this case it’s the prosecution that’s the real problem, not the defense.
Regarding the case of the twins, we don’t need to re-write any laws. Simply try them both at the same hearing for both murders – problem solved.
Here in Denmark, the identical twins would be sentenced. (According to a professor of law at Århus university.) The reason being that the law is flexible like that. Common sense trumps the word of the law.
There is a large company here in Denmark that administers a test to people they consider to hire. The test is more or less equivalent to an IQ test, and they only take people who score high. Anecdotically, they have problems finding lawyers, because none of them scored high enough.
In “Ill-Gotten Gains,” by Leo Katz, pp. 67-68, the author presents a similar “Twin Paradox,” where one twin is committing murder while another commits armed robbery. He argues that you can’t convict either twin of anything, even though you are sure beyond a reasonable doubt that each is guilty of “at least” armed robbery.
I guess you could allow “John Doe” victims and allow a merger of the two cases. Then you could prove that each twin murdered exactly one victim, though we don’t know which killed which, and therefore convict both. If the cases are separate or victims must be identified, they would be acquitted (in theory).
As a practicing lawyer, may I stand up for my profession here? You see, there is actually a perfectly good reason for why lawyers argue and judges rule that way. And it is …, it is…, no, sorry, I cannot do it!
The fact is that most lawyers and judges are like most othere wordy type of thinkers almost completely innumerate, feel threatened by any rigorous application of mathematics, statistics or logic, and will fall back on nonsense like “I’m not a machine!” or “That’s just numbers/circumstantial evidence!” when faced with situations and arguments like your hypos. The only honest explanation is that they fear encroachment of types of reasoning in which they are not well-versed (and often not well-suited for) into their field. In short, their conduct and reasoning are indefensible by any standard of justice.
(As an aside, in my experience, the converse does not hold true of the numbery sort of thinkers–they tend to be quite adept with words and unafraid of them too. Cf. C.P. Snow, The Two Cultures).
Fortunately, hypos like yours are relatively rare.
Much more common is the problem of the multi-element claim. Most civil legal claims contain multiple elements (often four to six, though more are not unheard of), each of which must be proven to exist and the law insists that the standard of proof for each element AND for the claim overall is “preponderance of the evidence,” that is “more likely than not,” that is “p>0.5.” That this is logically impossible is realized by only a minority of lawyers and usually hand-waved away when brought to attention of a court.
@Ryan “I think most people reject the idea that someone “probably” killed someone else. That’s not strong evidence, no matter what the probability is.”
That may be true. And that is why most people are unfit to judge criminal cases. There is no statement of fact which is not ultimately probabilistic. There is *always* a probability of error, though often quite small. If you cannot comprehend that your judgment should not be allowed to affect the administration of justice (or any other area of non-trivial decision making).
A sticking point with the hypothetical is that we all know that there is more evidence available. We all know about forensics. So it *feels* like saying, ‘yes convict him’ is unfair because we feel we could examine more evidence and not just rely on this 95%. If you can craft a purer example I bet you will get less emotional resistance to conviction.
That isn’t the only thing about this example which seems calculated to raise qualms. Murder? Isn’t reckless endangerment a better charge and won’t you convict both?
In short this may be less an example of how lawyers don’t get that 95% (or 98% or whatever) or that statistics are good evidence so much as it shows the power of subtle emotional cues. Here’s a hypothetical to show what I mean:
I bet if you gave this to a large class of first years but in one story you killed a beautiful young girl and in the other you killed her fleeing rapist you’d get very different numbers.
In the identical twins scenario, I’d bet that certain courts of justice are well-equipped to handle this case and hand down convictions. E.g. If there was a Nazi concentration camp at which many people were murdered, and if the camp had 2 Nazis in charge of the killings, but if it was impossible to discern which Nazi killed which people, I doubt the Nuremberg court would have withheld punishment on these grounds.
Aeternitatis – why is that last thing you mentioned logically impossible? I might misunderstand that premise.
I need a lawyer’s thoughts on this. Both were acting together with
reckless disregard for human life. One of the two killed someone
during this. Doesn’t the felony murder rule make them both guilty
of murder/manslaughter? If so, it doesn’t matter which one fired
the bullet, as long as it’s certain the one of the two did the deed.
@Jonathan Campbell
Let me explicate. What is logically impossible is to consistently judge multi-element claims by BOTH the standards:
A. whether overall all elements exists by a preponderance of the evidence (i.e., joint probability of element 1, 2 .., n > 0.5)
B. whether each individual element exists by a preponderance of the evidence (i.e., for each element n, probability of element n > 0.5)
Naturally in some cases, both standards will lead to the same outcome.
But in many quite plausible cases, the two standards will demand different outcomes: Imagine a claim with six elements (each statistically independent from all the others), each of which is proven with a probability of 0.51. Under standard B, that would mean that the claim is valid. Under standard A, the joint probability that all elements exists is only 0.51^6 or about 1.8%, so the claim is not valid.
In fact, it is fair to say that each element of the claim has been proven by a preponderance of the evidence, but that it is beyond a reasonable doubt that at least one element does not exist.
So claiming to judge a civil claim by both standards A and B is logically impossible. Yet, most judges and cases make that claim.
[The same argument applies in criminal cases if you substitute “offense” for “claim” and “0.5” by a higher “beyond a reasonable doubt” figure like 0.9 or 0.95.]
In fairness to my profession, it has to be said that often, even in multi-element claims, certain facts are not in dispute and the only question, if any, is whether they fit within the law, which is a logical exercise with an outcome of p either equal to 0 or 1. For that reason, the conflict between standards A and B practically occurs less often than one might imagine.
I am surprised that no one seems to get the objection at issue here. A man can be convicted from a causal chain, such as the defendant causes A, A causes B, and B causes death. Statistical reasoning can be used in a link, such as “the witness testified that A caused B, and there is only a 5% chance that he is wrong.” But if the link is purely a mathematical probability without any other reason to believe that A is related to B, then the defendant can argue that he was just unlucky.
If 20 people are convicted for crimes using the above 95% reasoning, then someone will likely be sitting in prison for a crime he did not commit. While that happens all the time because of mistakes, sloppiness, and perjury, this defendant will be in prison purely because of bad luck, and without any direct evidence linking him to the crime.
In the above case, you might say that the man has some culpability anyway because he was firing a gun recklessly. But it is easy to construct scenarios when the man is entirely innocent. Eg, suppose 2 unusual similar crimes are committed at different places, so that it is very likely that they were committed by the same person. Suppose also that one man can be shown to be on both scenes. Then he probably committed both crimes, but we do not want him rightfully saying that he is being sent to prison just because he was at the wrong place at the wrong time. There has to be more causal linkage.
@Roger
But *all* causal linkage–in fact, all judgment about facts–is by absolute necessity probabilistic and therefore the possibility of error *always* exists in *all* legal judgments. For good and sound reasons (which I’m happy to elaborate on and others have done beyond my skill), in criminal cases, we place a much heavier weight on the harm of a conviction of an innocent than on letting the guilty go free. Therefore we demand a very high probability of guilt before criminally convicting.
However, we cannot remove all possibility of error in any system of human factual judgment. Therefore we either abolish criminal punishment entirely or we accept that on some, rare occasions a genuinely innocent person will be convicted and punished. These are your only two options.
You cannot devise a system of criminal law and punishment which can guarantee that no innocent person will ever be punished just because they are unlucky. More than that–in a nation the size of the U.S. with a prisoner population of over 1 million, it practically guaranteed that at least thousands of innocents will be incarcerated.
But the only alternative is to abolish criminal law. Unless you are willing to embrace that, you *must* accept the rare, but real, criminal punishment of innocents. Wishing for an alternative system which avoids this drawback does not make it logically, practically, or legally feasible.
Aeternitatis – Ok, I see what you are saying. I thought that the requirement was simply that both standards A and B must be met for the claim to be considered valid, but it seems you are saying that judges require either both to be met, or neither to be met, for a coherent outcome. Obviously standard A is more stringent in all cases than standard B, so standard B, it seems to me, should just be dispensed with (I’m assuming that in practice both standards need to be met for the claim to be considered valid).
Ken B – in the example the charge is supposed to be manslaughter (which I believe is what you get with reckless endangerment leading to death).
Let us suppose that human being evolved to have an eternal life unless killed by a silver bullet. The killer, if guilty, is condemned to death by silver bullet.
Take the average honest and decent man, he does not want to kill anybody. But we live in a statistical world. The probability that he kills someone today is given by $p \in (0,1)$. It is very small, but using the large numbers law, we get that, with probability one, someday he will kill someone.
In this futuristic world, the police department is pro active and has full enforcement to put potential killer in jail. What should they do?
P.S.:A republican congressman from Texas is proposing an alteration in the current law. In order to decrease the expenses he proposes to use silver bullets on the potential killers. What should we do?
Professor,
As you like physics and mathematics, I am sure you are going to give us a kind of a “The killer of Schrödinger” here, will you?
@Æternitatis: Yes, of course the possibility of convicting an innocent man is unavoidable. I said so in my message. You are not addressing the argument.
Let me try to illiustrate what I think is Roger’s point about causal connection. Apologies to Roger if this is off base but I think it’s relevant anyway. I will not change any of the information just add some. See if your certainty on the right verdict fluctuates.
1. The 95 shots I fired were all last week.
2. The five shots my friend fired were straight into the ground.
3. The dead man was sleeping with my friend’s wife and raped his daughter.
4. I fired a .22 gun and my friend fired a .38 but the cops lost the bullet, and I’m a cop.
5. Landsburg was with us; he fired 1200 shots.
The point is that not all statistical evidence is equal. “Error bars” matter.
@Jonathan: You may be right about manslaughter as a matter of law, but that still requires accepting that 95% means his gun fired the fatal bullet. I was looking for a charge that did not depend on that finding. I’m not saying that manslaughter is wrong or that I would not convict, just highlighting a difference to emphasize the hidden assumptions at work in the example.
@Roger
I thought I did address your argument. What point did I miss? Do you think that there is a valid distinction between “direct” evidence of causal connection and merely “probabilistic” or “statistical” or “mathematical” evidence? If that was the crux of your argument, it still would not be valid.
There is no relevant difference between “statistical” and “direct” evidence. All inferences from all evidence, be it eyewitness testimony or number of bullets fired, is by necessity statistical and subject to some level of error. That in the case of the fired bullets we can estimate our risk of error numerically, while in other cases–is the eyewitness perhaps mis-remembering or lying?–it is harder to put a figure to our risk of error should make us trust what you term merely “statistical” evidence more, rather than less.
@Æternitatis: Yes, you missed the point. You keep arguing that there is always a chance of error. Everyone agrees to that. It is beside the point.
Unlikely as the twins scenario is, real life examples occur frequently when children are killed by their carers. It is often not possible to say with certainty which carer was responsible. If both stick to the story that the other did it, it becomes impossible to convict for murder. In the UK, there is a relatively new crime of “causing or allowing the death”, or similar, which allows for conviction of both parties for the lesser crime. The non murderer still bears some criminal responsibility for not preventing the crime. An example is the “baby P” case http://en.wikipedia.org/wiki/Death_of_Baby_P
The carers were cleared of murder because of lack of evidence, but convicted of causing or allowing the death.
As for shooting in the woods, presenting the stark statistics reveals the true situation. When we have a “95% certain” situation based on a judgement of the evidence, we can ignore the 5%, and we sort of shift the “beyond reasonable doubt” to a certainty. When it is “statistical” we cannot do this, and we are forced to acknowledge the chance we are wrong. Therefore, in our minds it seems (wrongly) that the “statistical” evidence is of a different type, and not admissable. As others have mentioned, it is not realy the type of evidence. If one person had shot 5000 times, and the other once, would that still be inadmissable? Or 5 million times?
“…Do you think they came today?” he said, “I do. There’s mud on the floor, cigarettes and whisky on the table, fish on a plate for you and a memory of them in my mind. Hardly conclusive evidence I know, but then all evidence is circumstantial…”
~~ Douglas Adams
Oh, and in the shooting-in-the-woods case, you could also convict both for common-cause. The fact that they shot into the woods together if it was an action undertaken in common between them, would render them both liable even if it could be established whose bullet killed the man.
My child murder case is different from the twin case, because in the twin case we know that both twins did actually commit a murder. But it does have some similarities.
In the twin case, if it could be argued that the twins planned the murders to take place simultaneously, then they could be convicted of conspiracy. If it could be proved that the two murders took place at the same time by chance, then the twins (in my opinion) could not be convicted. However, the chances of the twins acting without conspiring is so small that the conspiracy conviction should stick beyond reasonable doubt.
@Roger “[Y]ou missed the point. You keep arguing that there is always a chance of error. Everyone agrees to that. It is beside the point.”
Well, yes, I did think that you were arguing that convicting purely on statistical evidence would be one or more of unjust/unfair/illegal because it will sometimes result in innocent defendants ending up in jail because of error.
In all fairness, you did give that impression by stating that a defendant convicted purely on the basis of statistical evidence “can argue that he was just unlucky.” And that if one uses, e.g., a 95% probability threshold “then someone will likely be sitting in prison for a crime he did not commit” and “will be in prison purely because of bad luck.”
So, I hope you will concede that mine was an understandable misinterpretation of the basis of your argument. Or perhaps I am just slow and I require your argument in simpler sentences and shorter words.
In either case, could you please explain to me what the actual basis of your objection to convicting somebody purely on the basis that statistics shows that it is at least 95% likely that he committed the crime?
@Æternitatis, I have explained it a couple of times. If you don’t like my explanation, you can try following the links back to the original arguments. I think that reasonable people can disagree about whether the shooter should be found guilty, but your disagreement should be based on the actual arguments being made.
Steve will have an aneurysm — put your coffe down now Steve, and swallow that mouthful — but I suggest this example shows the inadequacy of MATHEMATICAL reasoning. or more precisely with the idea that the question is just a simple mathematical question.
Steve says, “anyone who can divide can see its 95% and convict. Anything else is foolish” But several responses show that in a real world situation there are other issues. Consider just some of the points made in posts above:
– what if data is missing. Say that a third party was present and fired 1200 shots?
– say the reason we only have such limited information is that some of it was suppressed by an interested party
– maybe murder is the wrong charge so conviction for murder would be wrong, but the law allows for other charges
– the twins could be charged with conspiracy and convicted
– similar cases to the twins hypothetical — more realistic ones like nazi camps — HAVE been successfully dealt with.
I am always suspicious of arguments along the lines of “my story exposes a deep flaw/truth/oddity about how people reason” because there are always lots of related issues and the researcher arbitrarily decides to ignore some of them, which he subjects might not have ignored. This is related to the difference of expressed and revealed prefernces, and why the former are so unreliable.
Roger said “A man can be convicted from a causal chain, such as the defendant causes A, A causes B, and B causes death. Statistical reasoning can be used in a link, such as “the witness testified that A caused B, and there is only a 5% chance that he is wrong.” But if the link is purely a mathematical probability without any other reason to believe that A is related to B, then the defendant can argue that he was just unlucky.”
I am not sure, but Roger may be getting at the idea of prior probability. Witness testifying that there is only a 5% chance that A caused B is meaningless unless we know the probability that the defendant caused A. This is a vey important point, but not the one under discussion here. Other than this I can’t quite see the point.
I had a problem with this in earlier posts. If we define the system such that 95% of convictions are correct, then by definition 5% are wrong. However, I thought along the lines of convicting if the juror was “95% sure” that the person was guilty. I don’t think these are the same thing. The former has a specific meaning. The latter is a vague definition of a feeling of certianty. People are reluctant to face up the fact that a 95% correct conviction means 1 in 20 convicts must be innocent. In the case of the shooting into the forest, the two definitions coincide, as there is no room for vague feelings – we are forced to realise what being “95 certian” actually means, and we are uncomfortable with it. This leads some rejecting evidence of this type.
Just a further note on the difference between the two “95%” criteria I mentioned above. If we have 95% convictions correct, we must have 5% wrong. We know nothing about the number of false aquittals. If we use the juror being “95% certain” we actually know nothing about the level of correct convictions. If the evidence gathering and presentational capabilities of the prosecution are greater than the defence, we could have a feeling of 95% certainty resulting in a 50% level of correct convictions. Equally, if the jury is reluctant to convict for some reasons, then a “95% certain” may correctly convict in only 0.5% of cases.
Harold,
The “95% certain” standard leads to a population of convicts of whom fewer than 95% are guilty, then the jurors should use that information to recalibrate until they achieve the standard. Such a situation would simply suggest that people are not able to judge probabilities as well as we’d hoped. However, it still seems as though the ideal system is one where we tell jurors to convict when they are > X% certain that the conviction is the correct decision. All of the factors that Ken B mentions should be inputs to that decision. To the extent that people are bad at judging probabilities, this system will not be perfect, but any other system will be worse.
@Harold: Re your second last post, about Roger’s comments. Steve wants to argue that 95 shots out of 100 is the ONLY relevant datum here. When you look at his argument that really is what it boils down to. That is the only way you can decide 95% is the right number. But that is really a “word problem” kind of answer. “Ignoring all other factors we calculate as follows …” But he is applying that to achieve a real world conclusion where you cannot ignore all other factors. That’s a slight of hand.
@Harold
Actually, even if juries and judges adopted a standard that they will only convict when they are convinced with a p of at least 95% that the person is guilty, it would not follow that 5% of all convictions are of innocents.
The key words are “at least.” That means that juries will also convict if they are judge the likelihood of guilt to be 96% or 99% or 99.9% as they rationally might in some hypothetical cases. That level of rational conviction is not all that rare in real cases either.
The exact fraction of the convicted who are innocent is a question of the distribution of guilt probabilities in cases, but it never follows that 5% of all convicts will be innocent under a standard of “beyond a reasonable doubt” equals p>=0.95.
The actual percentage of innocents convicted is likely to be substantially smaller (but not zero under any even theoretically feasible system of criminal justice).
@Ken B “Steve wants to argue that 95 shots out of 100 is the ONLY relevant datum here.”
I do not think that our gracious host would be so unreasonable. If some other factor (e.g., the caliber or marks on the fatal bullets were different, or the 95-bullet firer was in another part of the area, or is an cautious expert shot who would never accidentally hit someone), the ultimate outcome may be different. Perhaps the other shooter could be found guilty beyond a reasonable doubt; perhaps neither could.
But, all else being roughly equal, and one shooter having fired 20 times as many shots should be enough to get you to find that shooter guilty of having fired the fatal bullet beyond a reasonable doubt.
@Aeternatatis: Unreasonable or not that’s what his argument requires. He wants to argue that “a legal “education” consists primarily of having your capacity for logic beaten entirely out of you”. He does this by giving a hypothetical and asserting that a refusl to convict in this case demonstrates it. Why? because a refusal to convict means you don’t get that it’s 95%. Which is only case if the ratio presented is the only relevant datum.
@Roger: I sympathize.
Let me jump in late with a case for which I was on the jury. The defendant charged with robbery in county K clearly had been involved in the planning. He claimed that he had not engaged in the robbery itself and for his defense argued to have been involved in a different robbery at virtually the same time for which he had been indicted and was soon to go to trial in county S. The prosecution made little attempt to refute this claim. Our jury was convinced (p = 100) that he was an armed robber, but not convinced beyond reasonable doubt (say, p = .65) that he had been a party to “our robbery.” We acquitted, but several of us empathized with the county S jury who might parallel our reasoning letting a certain felon go free. Rather than finding some lesser joint guilt across two reckless shooters, we and the other jury needed some form of guilt across geography. Oh yes, a charge of conspiracy was taken off our plate before we began deliberations.
Ken B,
the hypothetical under consideration was originally presented by Leo Katz (in the book that Phil Birnbaum references above), who was defending the notion that the courts should not convict based on the evidence. The idea that Steve is artificially restricting the evidence we should consider is not true. Katz himself presents the case exactly as Steve does, indicating that he has in fact provided all relevant information, and asserts that the courts would not convict based on the evidence.
Ken B. I think the key phrase is “the nature of the evidence is “statistical” and therefore cannot be the basis of a conviction”. This suggests two “types” of evidence, and “statistical” evience is insufficient on its own. Whether or not you would actually convict in this case, it is this distinction that is questioned in the post, and I think it survives the transfer from hypothetical to “real world”.
Jonathan Campbell: I don’t see how the “recalibration” could occur, since the actual guilt or otherwise is never known.
Aeternitatis: Thanks for pointing out my simple error. It would only be 5% innocent people convicted if people were put on trial at random, I think, or they tried the first person they found who reached the 95% threshold. Hopefully the prosecutors do a better job than this. In the shooter case the suspect does in fact just pass this threshold, so has a 1 in 20 chance of being falsly convicted (of firing the fatal bullet).
Perhaps this “just past the threshold” is also causing difficulties in our minds, and we try to justify our doubts about the conviction by introducing false dichotomies between “statistical” and other evidence. If we say that 95% is the right threshold for “reasonable doubt”, and change the story a bit. What if one man had been firing into the wood for hours, and shot off 999 bullets, then along comes his friend and takes one pop? The body is not found until later, and the time of death is not known. Would you convict “beyond reasonable doubt”? I can see no justification for not convicting in the 95 case, but convicting in the 999 case *based on different types of evidence*.
@Harold: And not all statistical evidence is equal. I gave several examples where an ADDITIONAL bit of evidence would sway a reasonable person’s certainty of guilt. Katz is wrong to dismiss statistical evidence holus bolus obviously. But it is also wrong to just look at a stat and pronounce it dispostive.
An inflammatory example. Young black men are statistically far more likely to commit murder than young asian women. Let’s posit ad arguendo that the ratio is higher than 20:1. The hypothetical is a man was murdered in the woods. A young black man and an asian woman where the only ones near and they accuse each other. Convict?
Perhaps the point Katz is trying to make is that some statistical evidence is not evidence of causation but a reflection only of our state of knowledge: not about what really happened but about our possibly very incomplete knowledge of what happened. And in that case — which I think is Roger’s point too — it behoves us to be a little bit circumspect.
Ken B. Very interesting example. Assume we can say with certainty that one of the two committed the murder, and there is no forensic evidence and no possible way to obtain further information. My gut feeling is that it would be totally wrong to convict on the statistical evidence of black men alone. If there were no statistical evidence, the probability is 1/2 for each. With the statistics, it becomes much more likely that it was the black man, yet still it feels wrong to convict on this basis. So is there a difference between the stats in this example and the stats in the shooter example? I think there is. In the shooter case we know (or we assume to know) that the man was killed by one of the two men. So far so good, and the same as the black man and asian woman case. The statistic we have in the shooter case is derived from the individuals – one man shot 95 and one shot 5. This is incontravertable and applies to the two suspects directly. We know that one is 20 times more likely to be the killer. The individuals are the statistical population. In the other case, we have no information about the individuals, only about the populations from which they derive. We do not know how representative the individuals are as a sample, or the pread of the poulation. This introduces a further uncertainty in our knowledge of the individuals, and we do not have enough information to asses the size of this uncertainty. It is unreasonable to convict in this case.
@Harold: I take it then you agree there ARE different kinds of statistical information which can reasonably be evaluated differently. I think this is sufficient to show that Steve’s conclusion about the absurdity of legal reasoning is unjustified.
Once again though I think you are missing a point. The 95 shots is just one fact you know. It is NOT like Steve’s example of shooting a man who dies. We know quite a lot about the effects of gunshots, and so on. There is a strong causal inference; the statistics speak to what is actually going on in the world. The 95 shots doesn’t really in quite the same way. Consider again any of the additional factoids I mentioned. None change the fact that 95/100 is 95% but all — by changing the rather limited amount we know — affect the reasonableness of convicting.
Further the law does NOT as Steve seems to assume just throw up its hands here. The shooters can be charged with reckless endangerment of which they are both clearly guilty. This is not the sign of having the capability to reason beat out of you; it is a sign of being willing to modulate your conclusions based on the certainty and cogency of your evidence.
Here’s what I understand Landsburg to say: When confronted with the scenario described above, a judge should say to a jury, “I withhold from you the power to rule on the question of the defendant’s guilt. I, the all-knowing and statistically-educated judge will single-handedly make this decision. And I do so to demonstrate that all my legal education – education about process, and humility, and the perils of assuming too much power into the least democratically-accountable branch of government – has not beaten logic out of me. And the fact that 95% is the appropriate threshold of risk is not a matter of judgment to be deferred to a jury or a legislature; it is a matter of inexorable logic! Logic! LOGIC!”
Landsburg is merely making one more argument of the species “The world should be run by a philosopher king who happened to share my views.” And as far as I am aware, we are talking about philosopher kings here. Landsburg cites a discussion by Jonathan Campbell who is citing a discussion by Leo Katz, a law professor. Katz is best known for his books on apparent anomalies in criminal law. I like Katz’s work; I have a copy of Bad Acts and Guilty Minds at my desk. But it’s unclear that Katz ever practiced criminal law. He was a law clerk and then worked for five years in a big corporate firm before becoming a law professor. As far as I can tell, everything Katz writes is from the perspective of the philosopher, not the practitioner.
According to Katz, “courts would refuse to interpret the [statistical] evidence” that a man who fired 95 times into the woods had committed manslaughter, provided someone else had also fired 5 times. I’d like to know more about the foundation for Katz’s claim. Here’s the scenario I foresee: Prosecutor brings charges against defendant based on this statistical evidence. Defendant moves for dismissal/summary judgment on the theory that there’s insufficient evidence. Katz seems to imply that a judge would grant the motion. This strikes me as an empirical question: When confronted with such scenarios, do judges actually dismiss such cases?
My guess (and only a guess): No. I suspect Katz was speaking about legal theory, not legal practice. But in practice many trial judges are elected, and generally the voting public doesn’t seem that thrilled with leniency for those accused of homicide. Rather, I’d guess that a judge would concluded that the defendant had raised an argument that addressed the persuasiveness of the evidence, not its adequacy to establish a prima facie case, and the judge would defer to a jury to conclude whether the evidence was sufficient to convict. Think about it: if you were a judge, wouldn’t you defer as much responsibility to a jury as you could?
And here’s the larger point: What do people perceive to be the biggest problems facing criminal law? Recidivism. Cost. The conflict between instrumentalism (producing good social outcomes) and the public desire for retribution. In contrast, I never hear philosophical anomalies identified as a big problem by anyone other than would-be philosopher kings.
And I surmise that this is because, as Oliver Wendel Holmes observed,
To put it another way, the law exists to produce real, practical solutions to real, practical problems. It’s a way to produce an outcome that on average will be better, more efficient, and cheaper than the alternative: a shoot-out at the OK Corral.
And in part, a trial is a play performed to garner public support for the outcome, whatever that outcome might be. Admittedly, technical experts (“philosopher kings”) might be able to render better decisions in this or that case. (And there are many types of specialized tribunals in which they more or less do.) Or they might not. But unless a technical expert is able to persuade the public of the fairness of her decision, it wouldn’t matter; we’d still end up with people resorting to a shoot-out.
Landsburg says that “all evidence is statistical.” That’s one point of view, the point of view of the philosopher king; I humbly suggest that the public may have a different point of view. Wisely or not, the public has greater confidence in some kinds of evidence than others. Statistical evidence contains within it a candid acknowledgement of error. Most other forms of evidence do not. And to the extent that one purpose of a trial is to build public support for the outcome, whatever the outcome may be, then statistical evidence has a different quality than other kinds of evidence.
The public may be an ass to fail to acknowledge the probabilistic nature of all knowledge. But the law is not an ass for acknowledging the public’s limitations; rather, the philosopher king who fails to recognize the need to be accountable to the public is the ass.
This is a long way of saying: First, I think the role of a jurist should be, to the greatest extent practicable, to implement the law, not to make it. Second, if I think that courts should declare anything proved to a probability of 95% to be proved beyond a reasonable doubt, then I should seek my redress from the legislature and ultimately the voters, not from a lawyer or judge. The fact that I have thus far failed to persuade the people to this perspective should not be blamed on the judiciary.
Finally, the fact that educated people seem to regularly draw the contrary conclusion about the appropriate roles of the people, the legislature and the courts leads me to despair for the republic.
nobody.really – you say “This is a long way of saying: First, I think the role of a jurist should be, to the greatest extent practicable, to implement the law, not to make it. Second, if I think that courts should declare anything proved to a probability of 95% to be proved beyond a reasonable doubt, then I should seek my redress from the legislature and ultimately the voters, not from a lawyer or judge. The fact that I have thus far failed to persuade the people to this perspective should not be blamed on the judiciary. ”
My guess would be that if Leo Katz is correct about this particular point (and I agree with you that it is not 100% certain that he is), it was in fact judges who established the practice of excluding “statistical evidence” rather than the legislature (I highly doubt a bill was ever passed in a legislature suggesting that “statistical evidence” is not valid). If that is true, the ire should be directed at them. If not, not. Also, to the extent that judges recognize that there are certain clear flaws in their system, even if these flaws are not the judges’ fault, they should, as responsible citizens, raise the public consciousness about these flaws, rather than simply deferring to the public’s misinformed views.
Late to the party, and not directly on point, but California solved the virtual simultaneous shooting problem and by extension the twin problem in civil cases (preponderance of the evidence standard of proof) almost 63 years ago in a unanimous decision. Summers v. Tice (1948) 33 Cal. 2d 80
http://law.justia.com/cases/california/cal2d/33/80.html
Two hunters firing at quail shot toward a third hunter, striking him twice with bird shot. The injured hunter sued for damanges. The California Supreme Court affirmed judgment for plaintiff, holding that where defendants are negligent, actining in concert or independently, and cause injury and the plaintiff is unable to show which defendant in fact caused the injury, the burden of proof is on the defendants to show that the other defendant(s) caused the injury.
(By the way, here ia a fuller version of the quote:
“If the law supposes that,” said Mr. Bumble . . . the law is a ass- a idiot.” C. Dickens, Oliver Twist)
This is tricky. True, in the English/US legal systems much of the law was not adopted by statute, but rather emerged from the accretion of legal decisions that we call “common law.” For example, most laws governing autonomy and property rights arose in this manner. Should we therefore conclude that every trial judge is free to change this law at will, simply by saying, “Hey – nobody cited a statute that says I can’t”?
The doctrine of stare decisis says no: generally courts should adhere to precedent until instructed to do otherwise. The legislature is presumed to be aware of public decisions, and can take the initiative to change the law if the legislature deems it appropriate to do so.
Admittedly, many celebrated decisions have flouted this doctrine – Brown v. Bd. of Education among them. So I know of support for each point of view.
This is REALLY tricky. Yes, by tradition the chief justice of the highest court (federal and in every state) advocates for legal changes on matters of the administration of justice. They often advocate for the courts’ budget, for example, and for the need to fund public defenders. And these high courts often have rulemaking powers regarding judicial procedures. But otherwise judges often shy away from this role in deference to the separation of powers. For example, members of the US Supreme Ct attend the president’s State of the Union address but take pains not to express approval or disapproval of the public policies discussed. (Admittedly, Scalia takes a different view on this.)
And when speaking to “the public” in the jury box, judges tend to be even more circumspect. In the English/US “adversarial” legal systems, we let the adversaries educate the jurors; judges try to stay above the fray to maintain the appearance of impartiality. But judges make procedural rulings that can have substantive consequences.
For example, a judge can choose to admit or exclude an “expert” who would testify about the probabilistic nature of evidence, about how often “eye witness” testimony is wrong, etc. The judge can choose to exclude or admit witnesses offered to rebut this testimony. And the judge can rule with greater or lesser leniency regarding objections to the testimony.
More covertly, a judge can choose to admit or exclude voir dire questions to a potential juror about the juror’s knowledge of statistics and probability; these questions not only help in jury selection, they can also plant ideas in the minds of the potential jurors about the kinds of issues to look for in a case.
But if one party makes a doubtful argument and the other party doesn’t know enough to oppose it, the judge is mostly constrained. I can’t recall a judge saying, “Wouldn’t you like to challenge the foundation of that assertion?” to opposing counsel, for example.
Two small points:
@nobody.really
I’d hesitate to quote the vastly overrated Holmes. Really, he is to the Supreme Court what JFK was to the presidency; in the popular mind he gets far more respect than he deserves on the basis of what ought to be extrinsic considerations: Youth, handsomeness, assassination, good speech writers for JFK; a facility with language for Holmes; ideological alignment with the then-as-now dominant progressive intelligentsia in both cases).
@Prof. Landsburg
Despite your frequent well-founded claims of irrationality in the legal system, please accept my assurance that law school does not ” consists primarily of having your capacity for logic beaten entirely out of you.” I was a practicing theoretical physicist–perhaps even as facile and well-trained as you are–before I went to law school. The capacity of disentangling and thinking *logically* about the *exact* meaning of complex Constitutional, statutory, regulatory, or precedential language I had learned as a scientist (and computer programmer) turned out to be a decided advantage in studying particularly some of the areas of law outside the public eye (like, e.g., tax law or much regulatory law).
I think ‘reasonable doubt’ is an interesting formulation precisely because it is defeasible and refers to the background episteme. Reason itself changes over time- we don’t believe it reasonable to convict for witch-craft because our, modern, Reason has an antipathy for action at a distance and anything other than efficient causes.
Reasoning in Economics, as in many other fields, has changed greatly- consider the example of ‘Anti- Trust’ legislation- this is now viewed very differently than it was back at the turn of the last Century- or the highly sophisticated, ‘Corporate’, view of Crime represented by anti-Racketeering legislation.
If ‘reason’ is defeasible, doubt- by its very essence- is even more so.
The problem with Legal thinkers is that few- despite this appearing (to me, at any rate) the trajectory of the great H.L.A Hart’s thinking- embrace radical defeasibility as the essence of Law- particularly in America. The notion that some ‘Judge Hercules’ can always reinterpret the Law so as to preserve its fabric without tear or wrinkle is a sort of only half-hearted admission that legal reasoning is deontic and defeasible.
Prof. Landsburg point- which I don’t understand- may be that the Law as currently practised is not ‘reasonable’ in the sense of reasoning along the lines of the correct Bayesian theory.
As such, it may be co-opted to the pantheon over which Dworkin’s false God ‘Judge Hercules’ continues to preside.
My own impression is that the logical culmination of the Ronald Coase approach in ‘Law & Econ’ might truly transform legal reasoning.
Sorry hit the wrong button- I meant to add, if Legal Reasoning admits that it is defeasible, deontic, and temporally discontinuous rather than seamless and substantive- then that would be one way to say okay not an oxymoron but a fact about the world.
Another, European, method is to say ‘Legal reasoning’ is a rule bound procedural rationality with no normative content- i.e. the Law does not punish the guilty it punishes those whom the correct application of its rules label ‘guilty’. Again no oxymoron necessarily arises as Jorgensen type dilemmas are off the table.
However, the peculiar ancestor worship of the Americans- Coke and then their Founding Fathers and so on- and their determination to have their deontic cake and alethically eat it to, is perhaps what motivates Prof. Landsburg’s comment.
Sadly, I will not live to see this issue properly resolved because South Park has only six episodes left before it goes off the air permanently. Considering how their ‘Imaginationland’ 3 parter solved the problem of Meinongian objects, the demise of the series is the greatest set-back to Philosophy’s service to the Human Spirit since Zeno gave Parmenides a hand-job to get his mind off the problem of the One and the Many.
Ken B: I do not acknowledge 2 types of statistical information – there is one type which consists in these examples of a probablilty. In the shooter case we have a statistic with miniscule uncertainty – there were 95 shots fired by one man and 5 by another. In the black man case we have a statistic with high level of uncertainty – black men in general are 20 times as likely to murder as asian women, but we don’t know how this applies to our particular individuals. So we have the same “mean” but a different “standard deviation”. In each case we can say that our best guess is 1 in 20, but in the second, we cannot say how accurate this is likely to be. In the first case, this translate into a probability of 95%. In the second case it does not, and is less than this. I think this is a valid point, but I am not an expert.
Your factoids are merely ways of introducing extra information so we cannot asssign the 95% probability. If the probability falls below this, then logically we should not convict. However, IF we can assign 95% probability AND we use this as the basis for reasonable doubt THEN we should convict.
More generally, the post seems to me be pointing out a kind of logical fallacy. Of course there would not be a trial based only on this information. The post is saying that to not convict *for these reasons* would be a fallacy, not that not to convict at all would be. As nobody.really said, we are assuming Katz is correct in his assesment of how the law would operate in these cases. However, IF the law operates in this way THEN it is making a logical fallacy. I don’t think it can ever be wrong to point this out. Maybe judges in the future will be more inclined to rule in a clearer way.
I think that Ken B’s example of the black man and asian women is a good example on why statistics shouldn’t be the basis of convicting someone.
Also unlike mathematician who can solve questions about an infinite amount of talking ants moving in random paths. Lawyers need to deal with the real world and in the real world any aspect of a case such as whether ants actually talk or whether anyone can keep track of an infinite amount of ants matters.
In the real world, we have a case where one person fires off 95 shots while one person fires of 5.
In this case, the prosecutor should not base his argument on statistical evidence (95/100), but on intent to injure/kill. You don’t need to fire 95 shots to kill an animal you can’t see, so he must have known he was shooting at a human. Also the fact that he shot 95 times, shows that he wanted to make sure that he finished the job.
@ Æternitatis:
“The exact fraction of the convicted who are innocent is a question of the distribution of guilt probabilities in cases”
Is it your assertion that in the cases where DNA shows someone to be innocent that the judges, juries, and prosecutors was lying when they found that the evidence indicated that the person had a high probability of being guilty?
It seems to me the exact fraction of the convicted who are innocent and the innocent who aren’t convicted is also (but not only) a question of the distribution of incomes among the defendants.
Will A. How many shots does it take to shoot an animal you can’t see?
I had attempted to point out that basing the conviction on statistics is good, but these statistics must be correctly applied. I think in the black man case the statistics DO NOT give a probability of guilt of 95%, therefore correct application of statistics would lead to an aquital. Hopefully someone with a better knowledge of statistics will elucidate.
Lawyers need to deal with the real world, but surely they can deal with it better if they understand it. The key to the application of statistics is to use it properly. That means not overestimating its power, nor underestimating it.
Lets see if I can make my point by “cancelling out” all the extraneous information we don’t have.
Case 1. There is one hunter, he recklessly fires bullets in a direction where he heard a rustle even though he has good grounds for fearing that the rustle might have been caused by a person rather than an animal. A man was killed by one of the bullets. I think there would be little doubt about convicting for the killing, on whatever charge is appropriate – say manslaughter.
Case 2. There are 2 hunters, and one had a gun, the other a bow. The man was struck by both an arrow and a bullet. The exact cause of death was not possible to determine. However, it was assesed by the medical expert “beyond reasonable doubt” that the bullet was the direct cause of death. I don’t see how you justify not convicting again, if you would convict in case 1.
Case 3. There is one hunter. He hears a noise and believes there is a deer in a wood, but has good grounds for fearing that the rustle might have been caused by a person rather than an animal. He spends most of the day shooting into the wood, and fires off 999 shots. His friend comes up and takes one shot into the wood. Later it is discovered that a man has been killed by one of the bullets, but we cannot say which gun fired it. Do you convict? I would say yes. The chances of the first man being the killer is beyond reasonable doubt.
Case 4. There are 2 hunters who hear a noise in the woods. They recklessly fire bullets in a direction where they heard a rustle even though they have good grounds for fearing that the rustle might have been caused by a person rather than an animal. Defendant One fires 95 shots; Defendant Two fires 5. In the end, it cannot be determined whose bullet killed the victim. Defendant One is prosecuted for manslaughter. Do you convict?
Katz claims that the court would not consider the evidence in case 4 because it is “statistical”, and therefore “would not be connected with the actual facts in the right kind of way to permit one to say that the court actually ‘knew’… that Defendant One killed the victim.”
But how is it different from the evidence in cases 2 and 3? In case 3, the evidence is of the same type, but now we have a 1 in 1000 chance instead of a 1 in 20. Can the court not consider this? Can we say we do not “know” that defendant 1 shot the fatal shot because the nature of the evidence is statistical, even though the chances of error are 1 in 1000?
In case 2, the chances are “beyond reasonable doubt”. What if the medical expert had said he was “95% sure” instead of “beyond reasonable doubt”? Does this then become “statistical” evidence, which the court can no longer consider? Of course not.
The point here really seems to be that knowing only the numbers of shots fired in the case above may not allow us to asssign a probability of guilt of 95%, not that the type of evidence is inadmissable. There is nothing wrong with the type of evidence, just that there are possible other factors which introduce some extra uncertainty. So this comes down to the correct application of statistics. At the trial, the prosecution would introduce the 95% chance of guilt based on the number of bullets. This is the probability without any further information. This must then be multiplied by any other factors the defence may introduce. If the victim was the sworn enemy of the second man and unknown to the first, we may consider with some probability that it was all a set up by the second man, say a 50% chance. We must then multiply our 95% by 50% to arrive at the chances of guilt of the first man, and so we aquit. We do not simply ignore the statistical evidence of the number of bullets fired.
Will A: “Is it your assertion that in the cases where DNA shows someone to be innocent that the judges, juries, and prosecutors was lying when they found that the evidence indicated that the person had a high probability of being guilty?”
This is a good point. Juries frequently find the evidence convincing where the accused is clearly innocent. I believe it is exactly in these sort of cases where a proper application of statistics would help. I suspect that juries and laywers fail to apply good Baysian multiplication of the probabilities in each of the steps of their reasoning. Thus, if 3 factors are ALL needed to convict and each one achieves 95% probability, then perhaps juries will convict, believeing that each one has been proved “beyond reasonable doubt”. Of course the probabilities should be multiplied, resulting in only 86% confidence overall. Please correct me here if I am wrong.
I also believe that the term “reasonable doubt” will vary immensly. If you percieve the accused as a low down scumbag, you may find quite a large amount of doubt “beyond reasonable”. If the accused is very much like you, and seems to be a fine, upstanding member of society, it may take a very large amount of doubt to seem “beyond reasonable”. There seems no other explanation when you look at some famous miscariage cases, where convictions have been obtained without any real evidence at all.
@ Harold
Katz claims that the court would not consider the evidence in case 4 because it is “statistical”,
The way I read it is that the court wouldn’t consider the evidence as statistical, not that it wouldn’t consider the evidence. I believe that the evidence could be considered to show intent/state of mind.
As it relates to using statistics to “objectively” determine guilt, You are correct that .95^3 = .857375. I only see Baysian multiplication applying though where probabilities are well known.
Let’s say that I’m a juror. When I was younger, I was arrested for something I didn’t do. How do I come up with the possibility that a police officer is lying?
Do I use my own experience with police officers which says that the probability of the officer lying is 100%? Should I ask the court to question everyone who knows the police officer (mother, father, ex-wife) and ask if the police officer has every lied to them?
Do I commission a poll that asks the general public how often they have known police officers to lie?
Can you let me know the steps that a juror should take to come up with the correct/true probability that an officer is lying? This approach should of course be objective and so the jurors shouldn’t be allowed to use their own person experience.
Will A. The quote from Katz’ book is ” Again, courts would refuse to so interpret the evidence. But why?”. I am not quite sur4e how to take it.
For th elying policeman, you make the best estimate, and you qualify it with a degree of certainty. In the case of the number of shots fired in the example, the best estimate is 95, and we know this with a high degree of certainty. In the case of the black man – if that is all we know about him, then the best estimate is that he is 20 times more likely to murder than the asian woman (using the made up statistics from earlier), but we have a high degree of uncertainty. With the lying policeman, you have to make your best guess. You may think it is 95% certain that he is not lying, but there is a very high degree of uncertainty. As a juror, all you can do is estimate. However, statistics that have a clear error associated with them are going to be more reliable than those we have just made a guess at.
The expanded quote is:
“The argument is made that the mere fact that he fired 95 of the 100 bullets that rained down in the vicinity of the victim proves by a preponderance of the evidence that he is the killer. Again, the court would refuse to so interpret the evidence.”
The court would refuse to interpret the evidence under a statistical argument. It doesn’t mean that the evidence wouldn’t be valid under a different argument.
E.g. Let’s say a killer is caught red handed with a red gun. The prosecution moves to submit the gun as evidence. The defense objects. The prosecution argues that red is the color of violence therefore the gun should be submitted as evidence. Here the court would in all likelihood refuse to so interpret. However, it would probably allowed to the gun to be submitted under a different argument.
As it relates to the lying police officer, my point was to Æternitatis who seemed to be implying that you could come up with a formula for determining the percentage of innocent convicts that by looking at the percentages of guilt associated with each piece of evidence.
You seem to be making my point. You use the term that each person makes their best estimate. Or is it your contention that using Baysian multiplication on probabilities based on peoples guesses arrives at a better/truer answer than just a guess?
@ Harold:
Lastly, I can see how corporations would like it if jurors were required to use the math you propose on pieces of evidence. Chevron, Exxon, B of A, etc. could submit 1000 pieces of evidence each showing that they are 99.9% guilty and be acquitted.
Harold: “So is there a difference between the stats in this example and the stats in the shooter example? I think there is.”
Harold’s evil twin: ” I do not acknowledge 2 types of statistical information – there is one type which consists in these examples of a probablilty.”
Harold, you really should speak to that evil twin of yours.
Since this thread is still going on, let me throw anothert pig in (as the British say). When considering convicting consider how much extra information you would need to change your mind. I mean information is a somewhat technical Shannon sense. In the 95/5 shots case none of the little factoids I mentioned is an unlikely event. Landsburg was out shooting that day too? Since it’s not an unlikely possibility telling you provides you with only a small amount of information. Not a huge number of extra bits. In the case where a man died after landsburg shot him telling you he died moments later of a rare tropical disease and his condition was not worsened by the shot) is a very unlikely event. How often do people die of that in a random 5 minute period? So that conveys a lot of information — a lot of bits.
This suggests as I and other have remarked earlier that not all 95% calculations are the same. Some are more certain, some have wider “error bars”. the width of the error bar matters.
Consider a thought experiment based on this example. We know 100 shots were fired. You are allowed to ask “who fired shot N” and you can ask the question 10 times. You get 9 for suspect 1 and 1 for suspect 2. I am allowed to ask 50 times, and I get 45/5. Whose estimate is better? Same estimated value but different levels of certainty. That matters.
Ken B, Will A. I do not acknowledge 2 types of natural numbers. Is there a difference between the natural numbers 4 and 5? I think there is. Whilst my phrasing may not be perfect, I (or my evil twin) was attempting to point out a difference akin to the difference between 4 and 5. The difference between the statistics is that they are both 95%, but one has no uncertainty and the other has an unknown uncertainty (at the risk of sounding a bit Rumsfeldy). They are both of the same type: 95% plus-or-minus X, it is just that X is different, which gives us a different answer.
I think we are not differing too much, and when you say “not all 95% calculations are the same”, this is exactly what I have been struggling to say.
I agree in the strict, literal sense that perhaps a “real world” court would be correct in the following: “Again, the court would refuse to so interpret the evidence.” i.e. that the first shooter is guilty beyond reasonable doubt (defined as 95% certain). However, the reasons given are surely the wrong reasons. The reasons given are that the information is “of the wrong type”. But surely, in this case it is that the information is “insufficient”.
I suggest a criterion for conviction: the threshold of 95% probability of guilt has been reached with 95% confidence. Lets say there is no possibility of error in the number of shots fired. This statistic we know to be 95 shots and 5 shots, with no error. Lets further say that we “know” absolutely that there was no chance of another person being involved, there was no plotting, each shot was equally likely to have been the fatal one and there was no possibility of any other information relevant to the shooting coming to light. This may be unrealistic, but it is necessary to make a point. Then the number of shots would transfer directly to a 95% chance of guilt with no margin of error. In this case do you think we should convict? From my criterion, we shouldm as 95% has been reached with 100% confidence. From my interpretation of Katz, he thinks the court would not convict.
However, in the real world, there would be some chance of other information affecting our assesment of guilt. Therefore, the statistic of 95 shots to 5 shots is not quite enough to give us a 95% confidence of guilt because there is some margin of error.
Assume we had uncertainty in counting bullets (and no other uncertainty). (This is where I may go off beam). Lets say we knew the standard deviation of bullet counting was about 1%, or 1 in 95. Can we say with 95% confidence that the conviction threshold has been reached? Well, there is a 1 in 20 chance of being 2 SD away from the mean, i.e. that he fired 93 bullets, and a 66% chance of being 1 sd away, that he fired 94 bullets. In fact, as the best estimate is exactly on the threshold, there is a 50% chance that the threshold has not been crossed. Therefore there is less than 95% confidence that the “threshold” has been passed. We must aquit. This is even though our best estimate of the number of shots fired is 95. I think this is more or less what you are saying, but that the uncertanty could arise from a number of sources.
If he had shot 99 bullets (to the other’s one), then 2 SD’s gets us to 97 bullets. The 95 “threshold” has been passed at greater than 95% confidence, and we should convict. I am not sure if you agree with this. If so, then it is nothing to do with the “type” of evidence, merely the magnitude if the numbers.
In the case of the black man, assume that the statistics for black men was they are 20 times more likey to murder than Asian women. Our best estimate (given no other information) is 95% chance of guilt, but there is a big plus-or-minus (analagous to SD.) Therefore we should aquit, as there is a much greater than 1 in 20 chance that the threshold has not been passed (again, actually 50% since the best estimate is exactly on the threshold). If the statistics were that black men were 100 times as likely to murder, my reasoning indicates that we should only convict (on this evidence only) if the “standard deviation” among the populations was less than about 2.5%. Of course, we cannot readily obtain the “standard deviation” in such a case, but knowledge of the statistical basis would allow us to make better judgemnents of guilt, and lead us to convict on this type of statistical evidence alone only in rare situations.
I am pretty sure that the basic thrust of my argument is correct, but much less so that I have the right details. I therefore reject katz’s argument that the evidence “would not be connected with the actual facts in the right kind of way to permit one to say that the court actually ‘knew’… that Defendant One killed the victim.”
I think my numbers are off by 50% since we are only looking at the proportion of results less than 2 SD away, not within plus-or-minus 2 SD. This does not alter the argument.
Interestingly enough I found a similar case discussed in Blackstone, a trial from 1647.
Herewith the transcript.
SHERRIFF: M’lud, Bodger was killed in the wood, an arrow pierced his heart through.
Doger and Podger were about the wood and had upon their persons quivers most fine.
Dodger’s quiver was but small, capacious enough for methinks 5 arrows, but Podger’s
was marvellous grand, and could hold nigh on an hundred.
By my reckoning Podger slew poor Bodger.
JUDGE: Did thee inquire of witnesses?
S: Nay m’lud; 95 parts of 100 suffice for me.
J: Did thee inquire if Bodger were known to these men, if he had lain with Dodger’s wife
or stolen Podeger’s cow?
S: Nay m’lud; 95 parts of 100 suffice for me.
J: Did thee seek other huntsmen that day?
S: Nay m’lud; 95 parts of 100 suffice for me.
J: Did thee bring the fatal bolt that we might wit from it’s flashings its maker?
S: Nay m’lud; 95 parts of 100 suffice for me.
J: Where were’t thou schooled sirrah?
S: Coke’s legal academy m’lud, where I had all my bad logic beat out of me.
Yes, very nice transcript! Nevertheless, the Dodger and Bodger case is different from the shooters in the wood.
One last kick at this entertaining can.
If you sat down to play poker with Phil Helmuth to give you hints, and he said “Don’t try to win this pot” would you heed his advice?
Or would you cry “Have you had the logic beaten out of you man? The only goal of poker is to win pots!”
If so, please join my game.
The goal of poker is not only to win pots; it is to win money.
This is a more complex goal.
The goal of a trial is not only to convict the guilty and acquit the innocent.
It must also ensure that rights are protected — for the benefit not just of the accused but of the rest of us.
This can involve many subtle considerations.
Look again at my inflammatory racial example. By hypothesis the chances are higher than in the 95 shots case.
this is purely “statistical” evidence. It is the only evidence you have and it is probative.
I would bet the jusge would even admit that statistic into evidence.
Does everyone agree that might might reasonably serve a more complex goal than just locking up the 95% guilty?
I still await an explanation of why noting these concerns is evidence of impaired reasoning faculties compared to not noticing them.
And I’ll bet Phil Helmuth attends to more than just combinatorics at the poker table.
I think that you are misstating the legal argument. There is no percentage cutoff for reasonable doubt, and there is no prohibition of statistical proof.
That is a point, I think, not in the legal system’s favor either.
“I still await an explanation of why noting these concerns is evidence of impaired reasoning faculties compared to not noticing them.”
Noting these concerns is not necessarily evidence of impaired reasoning, just irrelevant to the main point.
How about the yellow cab case? A woman is knocked down by a taxi, and that is all she knows. The town has 600 cabs from the yellow cab co. and 400 from the green cab co. She sues the yellow cab co. because on preponderance of evidence (>50%) it was a yellow cab co. cab. The court will dismiss the case (according to Katz). Are they right, and if so why?
Katz argues that they are right to dismiss the case, because the statistical evidence is “the wrong type”, and “it “would not be connected with the actual facts in the right kind of way to permit one to say that the court actually ‘knew’… that {it was a yellow cab co. cab].”
Maybe Ken B. would agree to dismiss the case, because the fact that there are 600 of one and 400 of the other does not fix the chances of it being a yellow cab co. at 60% (as maybe the woman was mistaken, or maybe a green cab driver wanted to kill the woman, or maybe a cab from out of town was in town, or…).
I think Ken B’s reasons are different from Katz’s, and therefore Ken B has avoided dealing with Katz’s reasons, and therefore avoided the point of the post.
In this case, we have no extra information about motives, but lets assume that there could be no out-of-town cabs or any mistake about a cab being responsible.
If you judge correctly, then there is a correct compensation awarded. If you rule against the wrong cab co., then you do them out of the compensation. If you rule against the victim, you do them out of their compensation. The only way to minimise the chance of doing the wrong thing is to award against the yellow cab co. Why should the stats not be admissable? There are 3 choices: Dismiss the case – 100% chance of error. Award against yellow: 60% chance of error, award against green: 60% chance of error.
The Yellow Cab story nicely illustrates how the law can treat some statistical arguments differently from others. Reasonable people can disagree about whether the Yellow Cab company ought to pay, but it is wrong to say that only the raw probability matters. Some probabilistic arguments are more legally persuasive than others, even if they give the same percentage.
I’m with Æternitatis up to a point when s/he wrote…
“But *all* causal linkage–in fact, all judgment about facts–is by absolute necessity probabilistic and therefore the possibility of error *always* exists in *all* legal judgments.”…
Establishing the truth of unique single events (a crime) must allow for “improbable” possibilities. P > 0.05 etc are useful in a predictive sense: If A fires 95 times into the woods and B fires 5 times, is it most likely that A or B will hit some given target?
But of course it’s possible for the 5-shot person to be guilty.
I suspect the original example is too simplistic to be relevant.
The yellow cab case gives rise to both an action in tort and one in criminal law- the tort, in most jurisdictions, would permit statistical arguments and in any case the Insurance Companies would have an incentive to work together to find an actuarial formula for this sort of thing.
The criminal action is a different matter. Essentially, part of our Social Contract with the Govt. is that we not be convicted of crimes we didn’t commit because- unlike torts- conviction for a crime has a huge extra psychic cost and strikes, so to speak, at the very foundation of our identity. In other words there is a non-quantifiable, highly subjective, element in the cost of criminal as opposed to tortious liability.
Elementary Econ theory tells us that probabilities are meaningful only in relation to the costs or benefits associated with them.
Unjust conviction for a crime imposes a very high cost which can’t (unlike tortious liability) be hedged.
The Social Contract with the victim, on the other hand, is of a different nature. It is not that the actual perp. will be brought to justice but that the suspect will be arraigned and prosecuted in accordance with the law even if this involves a larger expense than the victim might settle for in compensation.
To summarise, the arguments made so far on this page, are unworthy of Economists for they focus on probabilities rather than Expected Values.
We actually had a case here in Berlin, Germany, that comes quite close to the described twin scenario. Two twins were suspected to have broken into a shopping mall stealing diamonds and jewellery. The case rested on DNA found at site that could be traced to the twins but not one specific individual, so in the end none of the twins could not be prosecuted. (Yes, even identical twins do not have exactly the same DNA, but german law only allows DNA-tests in 11 specific areas that did not allow for a differentiation between the two).
This kind of case seems to involve, at least, how we would use the word “know” in connection with frank statistical evidence. Do you know which hunter fired the fatal shot? No, you don’t, the defense attorney would say. You have a well supported belief, but it leaves room for doubt here. When do we claim to know something in cases where we have statistical support? This may involve, as well, something called the lottery paradox and such claims to know; there is a good book on the topic by J. Hawthorne (Oxford U Press), titled something like Knowledge and the Lottery Paradox.
The Yellow Cab case is very nice indeed, because the preponderance standard feels more reasonable. I agree with Rogers here too. I do dispute this “Dismiss the case – 100% chance of error.” But we don’t know the cab is at fault even it did hit her. Establishing that would require establishing more facts about the incident. It is hard to sustain the premise that she or we know enough about the incident to establish what happened with establish more facts that might be relevant to yellow/green.
Now a question to discomfit the comfortable. We have had three examples: the arrows, the yellow cabs, and the young black man/old asian woman. In which example OUTSIDE OF A COURT would you have the greatest confidence in the inference? And remember it was part of the hypothesis that the odds in my example were greater than 20-1.
See why courts might treat different data differently?
@Harold: “Noting these concerns is not necessarily evidence of impaired reasoning, just irrelevant to the main point.” No, they go directly to the main point. The main point is, is it irrational for a court to treat as insufficient certain kinds of “statistical” evidence. And it is not, partly because of the concerns I mentioned. Courts do not just decide factual questions, they operate under other constraints as well.
Ken B. I thought the main point was not that the statistical evidence was insufficient, but that it was “of the wrong type”. The phrase ” although the belief that Defendant One killed the victim may be justified, it “would not be connected with the actual facts in the right kind of way to permit one to say that the court actually ‘knew’… that Defendant One killed the victim.””
Lets have another think. Two hunters. One fires a shot at a noise in a bush etc. No connection could be found between either of the men and the victim, and each blames the other. It appears to be a result of reckless behaviour. If a forensic scientist said he was 95% confidant that shooter A’s gun fired the fatal bullet, this would (probably) be enough to secure a conviction. The evidence would be weighed by the court, and the 95% chance of the bullet coming from the gun would be enough to say that we “knew” shooter A fired the fatal shot. However, there is still a 5% chance that shooter B fired the fatal bullet.
Now if shooter A fired 95 bullets and shooter B only 5, Katz suggests we would get a different outcome, because we could not say we “knew” shooter A fired the fatal shot in the same way.
Now have I mangled Katz’ position or the stats? Is there a good reason why we could convict in the first and not the second case?
You could say we could not convict in either case. If so, re-work the examples with 1% chance of the forensic scientist being wrong, and 99 vs 1 bullets.
@Harold: To me the main point is this. A ruling that that the evidence presented is insufficient to sustain a conviction without anything more is taken as proof that ” a legal “education” consists primarily of having your capacity for logic beaten entirely out of you.” As so often when Steve ventures into law this is wrong. That is the issue I’ve been arguing all along.
In your latest example you have snuck in more data. The men heard a noise and fired at the bush. This goes to intent and simultaneity where the earlier example does not. It rules out different caliber bullets (ie the chance that we are only getting cherry picked facts), when the earlier example did not etc. You state that the charge acknowledges recklessness, Steve’s does not. The use of a forensic exam rules out my lazy sheriff from my transcript who saw 95/100 and stopped looking (Just as Steve saw 95/100 and stopped looking for *other concerns*.) The evidence in your example is more probative for all those reasons. I don’t know if it is or is not enough to convict but it comes closer than Steve’s does.
Incidentally Harold et al, this has been a very good discussion, pretty much free of the straw men we see too often here.
Plus WillA got it right this time! :>