Since my blog post yesterday about the MPs expenses scandal and the correlation that I seem to have discovered between the likelihood of an MP being caught up in this scandal and the safeness of their parliamentary seat, I have gone through the data again. This is because I discovered there was an anomaly in that Margaret Hodge had appeared in my source data twice (not sure why this was). Also, the Telegraph list here was updated at about 3:00pm yesterday so there are a few more people added to the list.
The new result is a value of 262.4 with 94 MPs implicated. We would have expected if there was no correlation between safeness of seat and implication in the scandal for this figure to be 323 (646/2). So it is still a long way from the median and implies a link between safeness of seat and likelihood of being involved in this scandal.
The quartile graph now looks like this:
As you can see there is still a clear gradation in each quartile indicating that the safer the seat, the more likely the MP is to be implicated.
If I am right about this correlation then there are surely very serious questions to be asked about our electoral system. Advocates of First Past the Post always claim as one of their main arguments that the constituency link needs to be maintained (even though Single Transferable Vote, a much more proportional system with multi-member constituencies that the
Electoral Reform Society and
Make Votes Count advocate also has a constituency link). However looking at the above analysis it strikes me that FPTP does not serve its constituents well at all when it comes to this scandal.
When the dust of this scandal finally settles, I think we need a full "drains up" job in Westminster where everything is looked at in close detail. Proper analysis should be performed on the question I have posed, whether there is a correlation here and action taken to address this. I personally think an electoral system where there are no safe seats and the electorate can give their verdict without needing to vote tactically and where every vote counts towards the final result is needed.
UPDATE2: WARNING - From here on down it gets very technical!
Andy Hinton
who blogs here has done some statistical analysis on this data now. Andy is not a statistician himself but has some experience of running stats tests from his degree (and he certainly knows more than me!). I have all the details he sent me but here is a synopsis of the two tests that he has run:
Firstly, he performed a "t-test for unequal sample size/variance" which he suggests is the best sort of test for this sort of data. I will let Andy take it from here:
It gives us ample grounds to think there's something in your hypothesis. The effect is significant at p<0.001.>
However, disclaimer: The t-test is only strictly valid for normally distributed populations. It is fairly robust for large samples like the one we're looking at, but... The problem is that it is impossible to have a majority of less than 0, so there is a cut-off point which skews the distribution of majorities somewhat, especially for the innocent MPs (whose average majority is closer to that cut-off). I don't think this completely invalidates the test, but if someone who was more of a stats geek than myself wanted to have a look, they might have some ideas about getting round it.
So it would appear that there is a 1 in 1000 chance that the effect we have seen would happen by chance, albeit caveated with what Andy says towards the end.
Andy then ran another test. Again, over to him:
I have added a sheet where I do a Wilcoxon Rank Sum test (also known as a Mann-Whitney test), which looks at the rankings rather than the absolute values. That should make the test more valid, and help us get around the skew problem. This time, I get p = 0.0003! ie. A 0.03% chance of getting that result randomly.
I think we're on pretty safe ground to say that this is a statistically significant effect.
So with this test there is a less than 1 in 3000 chance that the effect would have been by chance. Andy's overall view on the two tests when I asked him which one was best is:
As I understand it, as long as the assumptions are not violated, the first test is the more sensitive and preferable, since it is the one looking at the actual data rather than simply the ranks, so it is taking account of more of the information we have.
However, if the skew of the data is sufficient to invalidate that test, the second is, in technical terms, non-parametric, and therefore it's the more widely applicable. As long as I have performed it correctly, there shouldn't any problems about the second test. BUT, the second test is the one I have the less experience in performing, so I would be less surprised if I had made a mistake in doing that one.
So.. no simple answer, really. I think, probably, I'd go with the second one, but would be much happier saying so after some peer-review!
My feeling is that I have now had someone who is fairly knowledgeable about statistics run two different tests and whilst there are caveats, it is looking like the effect I have discovered is statistically significant.
Any statisticians reading this who want to offer their opinion or get hold of the data and run your own test(s) and/or peer review Andy's work please let me know.
UPDATE3: I have been invited onto Radio 4's "More or Less" to talk about my findings regarding this correlation this week. It's broadcast at 1:30pm this Friday afternoon (22nd May) and repeated this Sunday at 8:00pm. See this post for more details.
UPDATE4: Government minister Ben Bradshaw has now referenced this research on national television.
UPDATE5: I have done an updated version of the analysis here.
Posts
59 comments:
Wow!
There must be a high correlation between being a strong party creature and being allowed to stand in a safe seat. So are they people who have a massive sense of entitlement and are happily ripping us off? Or is it a simpler correlation between believing "the rules are everything", always voting on the party line, and claiming the maximum allowances without thinking to consult their consciences?
Hmm, not a statistician myself Mark and once the dust has settled on this (will it ever) then definitely worth doing again.
I think one of the main issues I have with the analysis that we don't know what the DT's methods were in prioritising which MP should be looked at first.
One could assume that the DT ordered a list based on how well known the MP was.
Clearly it will sell more papers if it involves a household name, so I would have expected that their list was based on that fact.
By definition I suppose, an MP in a safe seat will likely to have been in Parliament for longer and therefore will be more "well known".
What follows from the above argument is that there is a lot more to come from the "less famous" MP's.
Of course the above is all based on assumptions etc, but definitely an exercise worth repeating over time to see if that correlation that you have identified changes.
Nice and thought provoking on a Monday morning!
Have to agree with UB41. You're relying on the DT sampling representatively. Hopefully Polly will read the Grauniad's "Bad Science" column, before making too much of this.
You are 80% less likely to die from a meteor landing on your head if you wear a bicycle helmet all day. Apparently.
Anon:
Of course we are relying on the Telegraph's data as I freely admitted. I never claimed this was definitively scientific but it is an interesting indication nonetheless and until all the data is released into the public domain it is all we have to go on.
I am a big fan of Ben Goldacre and I would hope he understands what we are doing here.
BTW I have turned comment moderation on here temporarily.
I don't mind strongly felt opinions being expressed, indeed I feel strongly about what has been happening myself but please try and keep the language moderate.
Mark, the reliance on a selected dataset is a major shortcoming. Although you have currently found an effect, the reasons for it are not defensibly to do with the relationship you surmise. You either need all MPs data or a random sample. Even then, there may be other covariates that cause the relationship, rather than it being explicitly due to calculation on the MPs part.
By the way, I am a statistican for Matrix Knowledge Group, and so believe I am correct in being a little more cautious. Regards, Will.
Will - like I keep saying I do not see this as job done. I am reporting what I have found so far and have caveated it accordingly. My conclusion is that you are more likely to have been involved in this scandal the safer your seat. That is irrefutable.
As you say, the question is what that tells us. My interpretation is that it tells us something about the FPTP electoral system. I am of the view that this system needs to change.
I am as keen as anyone else to see if my conclusions stand up when all the data is released. In the meantime I welcome the debate that my work so far has provoked.
Will - just seen your second comment.
Would you willing to help me come up with something more robust once all the data is released?
A 0.03% chance is a 3 in 10,000 chance, not, as stated in the post, a 1 in 3,000 chance.
Julie - I did say less than 1 in 3000 chance to be fair but your way of putting it is more accurate. Thanks.
You know... I've been thinking about this.
Has the Telegraph been factored into this correctly?
They've had limited resources and have to investigate each MP individually, and started with the big names and gradually worked their way down the list.
It's reasonable to expect that bigger names have safer seats. Labour have a high proportion of Scottish MPs which are some of the safest seats there are (or were).
So if I were to sound a note of caution, it would be against jumping to conclusions until all MPs have been examined - we've seen 100 out of 647 or so at the moment - the 100 biggest names.
Still, excellent analysis all the same.
Ah I see Will Parry made the same point. Whoops, apologies :)
You've been linked to by Guido Fawkes - congratulations, and ... brace yourself.
I think the best test in the circumstances is the Pearson Chi-Squared test. Might have a number crunch in a bit.
My head hurts now!
Until their mindset changes, their will not be proper reform.
Good post.
http://www.plenty2say.com
I love the internet. Well done.
Ok, for those interested I've now done the Chi-squared test in Excel using the quartile data you've posted. I assumed an H0 of a flat distribution (i.e. p(MP is corrupt)=94/647, E(Number of corrupt MPs per quartile)=p(MP is corrupt)*(Number of MPs per quartile)). If this flat distribution is assumed, you'd expect 23.5 corrupt MPs per quartile.
I got a probability of 0.008. That is to say that if the safety of an MP's seat has no effect on the likelihood of that MP abusing expenses then there is only an 8/1000 chance of observing this. The normal barrier for statistical significance in this kind of test would be 5/100 (0.05).
I haven't corrected for the fact that the results are discrete.
Excel has a function that does this test for you (=CHITEST), so if you divide your data into deciles in a similar way you'll get a more accurate result. The excel help file on how to use it is very helpful.
JABItheW - Thank you very much for that!
So if I am understanding this correctly, you are saying that this test also seems to confirm that the result is statistically significant (caveated with the fact that at the moment we don't have all the data)?
Mark - maybe you could call this the 'trough test'?
Mark,
Exactly. As you observe, the data might be skewed by the fact that the Telegraph has focused on the parliamentary big beasts, who tend to be in safe seats. Nevertheless, this is a statistically significant result suggesting that the safer the seat, the more corrupt the MP.
If you send me the raw data you've compiled I'd be happy to do the test again using deciles myself. Address is my blogger ID at gmail dot com.
I am in work and all the data is on my home machine but I will send it over tonight.
Mark - very very interesting.
My MP {Charles Hendry] has a majority of 15921 [137th] and has claimed full ACA most yrs too.
He is yet to be in The DT's crosshairs...
Have linked to you.
couldn't agree wit you more on making votes count.these parasites are always bemoaning low turnouts,little realisign they are it's main cause.
A system of positive voting is an absolute must.
It would be interesting to see data on how many tactical votes there are at any given GE,effectively skewing the party support data.
UB41 mentions the likely correlation between having a safe seat and length of service as an MP. ie the trend you're detecting could be to do with long service rather than/as well as safe seats. Is this something that you could test? The sample may be too small, but it could be interesting to see if the trend seems to apply to MPs in safe seats elected since, say, 2001.
Hi Mark, I think I will have to politely decline on undertaking any further analysis of this dataset. I am very busy at the moment, and besides, I doubt it is a particularly rich dataset and so the risk of wrongly implied causality (a social scientist's constant concern) is too great. Regards, Will.
I think concerns about left-truncation of the data are a bit nit-picky given the roughness of the data. So, don't sweat the statistics. Do worry about the implications of reform. Just because greater expense claims are associated with the safeness of the district doesn't mean changing to another electoral system will make the problem go away. In fact, most academic research on corruption shows that systems of proportional representation are associated with more, not less corruption. A quick Google Scholar search might help.
Chris, it depends which system you use.
I am a big fan of STV (Single Transferable Vote although Scottish Television is quite good too) which is not usually associated with corruption and there are no safe seats. If a constituency wants to get rid of an MP it can but can still elect another member of the same party. I think this would focus a lot of parliamentary minds in a way that we just don't get at the moment.
OT I know, but its rather surprising to see how un-Bayesian everyone involved in political analysis seems to be. I obviously move in the wrong circles.
Chi-square analysis is the appropriate test for these data. The t-test examines the difference between two means that presumably are on a Gaussian sampling distribution (yours are not), and thus makes no sense. Also, t-tests should only be used when the scale of measurement employed in assigning 'numbers' is interval or ratio scale data; yours appears to be ordinal. The Wilcoxan is for dependent measure designs, which yours is not.
Nice. Economic theory might expect the relationship to run the other way, though, with those on smaller majorities more likely to cheat expenses. The rationale would be that those less certain of long-term MPship have a shorter discount rate (i.e. time horizon) to exploit their position. I guess in a way this is democracy-affirming - this result might be explained by factoring in the risk of discovery and a calculation of the proportional loss of votes. For those with low majorities, even a small drop in votes in caught dibbling would lose them their seat, but those with larger majorities calculated (wrongly, one hopes) that they are able to withstand such a risk and still get voted back in.
As someone who worked in politics I have this explanation.
Misusing an allowance for personal gain, expecting payment and not expecting to be found out are symptomatic of arrogance.
A number of things happen to MPs who have been in the job for several years but they almost all become arrogant. They do really help some people and those people are grateful. MPs also find that public bodies cave in when approached on behalf of constituents. They are listened to respectfully by members of their local party and they receive lots of letters begging for work experience or a job.
MPs in a safe seat are able, if they want to, to stay an MP for a long time. MPs with narrow majorities are not as likely to be an MP for a long time so the arrogance factor takes a while to kick in.
You will find that some MPs in marginal seats hang on for quite a few parliaments. Their expenses will be questionable but they are not household names so don't appear in the papers on day 1 or even day 11.
I'd suggest looking at the length of time a person has been an MP, rather than their majority.
I've just been having a look at this data. I got the expenses from http://www.parliament.uk/documents/upload/HoCallowances0708.pdf and the majorities from http://www.psr.keele.ac.uk/area/uk/mps.htm.
I am looking at the relationship between majority and total expenses.
A scatter plot does not show any relationship that my eye can see.
A linear model is not valid because the data is not normally distributed.
I tried a Spearman Rank correlation test to show the relationship between majority and total expenses. This is not significant.
I did not do anything to tidy up the data such as removing odd cases or examining only a subset of MPs.
I reject a comparison of the MPs that the Telegraph did in detail against those not mentioned, as I suspect it says more about the newspaper than the MPs (as Will Parry says).
I think it is necessary to be very cautious about this. Firstly what you are showing is correlation, not causation. Now it might be that what you propose is a reasonable explanation, but I can think of several confounding variables.
The biggest weakness by far is that you have to take into account how the Telegraph have chosen their targets and what criteria they chose. They have admitted that they are swamped with documents - I think the figure of 2 million has been mentioned - and it has taken a huge amount of research. So they will have prioritised. They might easily have done this by choosing the more senior figures first, and I think there is a fair chance that these will have safer seats, if only because they are often long-standing members or because the political skills that got them to the top are also got them a safer seat.
The Telegraph might also have targeted figures on political grounds. Maybe investigating more Labour then Conservative MPs (the distribution of majorities is different among the parties due to demographic issues).
So in all, as a way of proving the first-past-the-post system encourages corruption, the best that can be said is that it doesn't contradict it. However, as a robust conclusion then it leaves rather a lot to be desired. I think the use of the probabilities is rather overstating things given these confounding factors.
This is not to say that having a safe seat and being a long time MP doesn't encourage complacency or even corruption, but if the aim is to increase accountability, then make sure your proportional voting alternative isn't worse. In some party list systems, the electorate defines the proportion of representatives, yet the party defines what order their candidates are elected in. This is a truly dreadful system as it makes the candidates far more accountable to their parties than the electorate. In effect, being at the top of the list is the direct equivalent of having a "safe" seat as long as you can avoid being down listed by your party the next time around. It is exactly this sort of appalling system that the UK has for the EU elections. If yo dislike a candidate, then you can only choose to vote for another party.
I should also add that proportional representation systems are often dreadful for true independents, although not so bad for minority parties.
Edit above:
If anything there is a negative relationship.
MIssJ.3U09qd9F_0sYfInyTeifmhA--#7dce0:
What an interesting name!
I think going for the data-set that ranks the MPs by the amount of expenses they claim could be very misleading in itself. There are legitimate reasons why MPs may need to claim more than the average, usually related to how far away they live (e.g. an MP in North Scotland would need to claim more than an MP in Kent for travel). So to use this as one of the axes may not tell us very much. I thought that going with the DT's implicated MPs so far was the best measure we currently have of who is worth comparing against. I know there are arguments against using this in this way but I thought it was the best way to do it at the moment.
Steve - I guess the same applies to the start of your comment. As for you point about electoral systems, yes I agree if we used a list system but I am not advocating that. I am advocting Single Transferable Vote in multi member constituencies which is roughly proportional and means that there are no safe seats.
I had thought about looking at correlating the amount of expenditure claimed against the seat majority as a more objective indicator. Now it is, of course, true, as you suggest, that some MPs have valid reasons for higher expenditures than others, with the distance from Westminster being the most obvious factor.
However, that only invalidates this as a test if there is, in turn, a correlation between the distance of the constituency (as a proxy for expected valid expenses claims) and majority sizes. For example, if seats in Scotland tend to have larger majorities than those in London then this would clearly distort that relationship. However, it is possible to test that hypothesis and introduce correcting factors, although the relatively small size of the total sample size might water down any conclusion.
I think that not knowing the DT criteria and methodology is much the most likely cause of any distortion. That is such a large factor that any conclusions at all have to be highly tentative.
On the Party List vs STV system, be very wary. We have the party list system in the UK rather than STV for the EU elections (and both are allowed) as those making the decisions are powerful people in political parties. Frankly they prefer systems which give them the decisions over who gets elected rather than the electorate. If you are not careful then we could get something much closer to the position in the House of Lords where it is effectively political appointees based (roughly) on the proportion of MPs in the House of Commons. Tweaking that to party nominees based on share of vote isn't so different.
STV might work - the machinations can get a bit complex - but I think you'll find that parties will fight tooth and nail to avoid it. Ending up with a worse, less representative system, than first past the post is a real danger. Too often the talk is of "proportional" systems rather than what is really needed, and accountable one. The former does not necessarily mean the latter a nicety which power-brokers will gloss over.
The skewed sample is a problem although I suspect that the DT have used a combination of fame and bad behaviour to select. Multiple regression would be of some help here although safe set would be co-variate with length of service. Of more help is that I know Will's boss (Jacqui) and I could give her a ring and ask her to tell him to do it!
A Chi-Square test is a slightly odd choice, surely, since the MPs don't fit into discrete categories for majority other than by artificially putting them in groups (quartiles, deciles, etc)? Why do that, when you can use a test for data on a continuum?
lXJb.MIssJ.3U09qd9F_0sYfInyTeifmhA--:
Total expenses won't tell you much, since that includes all sorts of perfectly legitimate costs associated with eg. running an office to deal with your constituents' letters etc. Have you looked at just Additional Costs Allowance?
Statistically_insignificant:
The t-test examines the difference between two means that presumably are on a Gaussian sampling distribution (yours are not), and thus makes no sense.As I noted at the time (Guassian = Normal). I did, at the time, do some histograms to look what the distribution of majorities was like, and it's not actually a million miles away from a normal distribution, albeit with the no-majorities-less-than-zero problem I mentioned.
Also, t-tests should only be used when the scale of measurement employed in assigning 'numbers' is interval or ratio scale data; yours appears to be ordinal.um.. the t-test was applied to the majority numbers themselves, not the rankings derived from them.
The Wilcoxan is for dependent measure designs, which yours is not.I did a Wilcoxon rank-sum test, not a Wilcoxon signed-rank test. If that doesn't answer your point, then I'm not sure I follow your argument.
correlation and causality are two different things.
I'd be interested if you can find the correlation between MPs who had something to hide, and those voting Aye in the following division, where they needed 100 members to break the LibDem filibuster against the 2007 Bill to exempt MPs from the FOI Act:
http://www.publicwhip.org.uk/division.php?date=2007-05-18&number=122&display=allvotes&dmp=1047
This is where the conflict of interest between personal affairs and the power to shape the law of the land is at its clearest.
Has anybody got a comment on the correlation between the length of service and the amount of troughing? It's just that before the expenses sytem was expanded before about 2000 I think, a lot of MPs had to make do with what they had. It would be nice to see whether they still are being frugal (Chris Mullin with his black and white TV comes to mind).
(former MPs researcher)
I've just tried the analysis that Julian Todd suggested above.
Divide the MPs into 2 groups according to their vote at
http://www.publicwhip.org.uk/division.php?date=2007-05-18&number=122&display=allvotes&dmp=1047
The MPs who voted "aye" are the ones who are against public scrutiny of expenses.
I did a one sided Wilcoxon rank sum test on each class of expenses (plus the total) grouped by vote. The one sided test was for "aye" voters having greater expenses. Allowing for the multiple comparisions, none of the differences between the groups of voters were significant (although, informally, the differences were generally in the direction expected).
I have a theory. I think cold weather causes MPs to spend more on travel. I obtained statistics on mean annual temperatures in each constituency, and individual members’ travel expenses, and calculated my correlation coefficient. I performed my statistical significance tests, and it’s good news: p<0.000001. Even my Newey-Wilcox-Markov-Trouserpress test for serial statistical misunderstanding comes up trumps. So my theory is correct: cold weather causes travel profligacy, and there is only a 1 in a million chance this effect could have happened by chance. Or does spending lots on travel cause MPs’ constituencies to be cold? Hmm…
It should be obvious what’s wrong here. Travel expenses are correlated with distance from constituency, which in turn is correlated with colder weather. By not separately allowing for the fact that temperature and distance from London co-move (i.e. move together), my cold weather variable has subsumed the influence of the true explanatory variable (distance), leading me to false inference.
If that example seems far fetched, consider the fact that the safety of an MPs’ seat is well-correlated with their length of service. In addition to seat majority, you will therefore find a good correlation between length of service and ‘corrupt behaviour’ (I dread to think what your dependent variable is for this). So what is the true explanation? Could be safety of seat, could be tenure, could be age, could be all three, but until you include variables that co-move with both corruption and seat majority in the framework of a multivariate regression, your model explains precisely nothing, and no fancy statistical tests can alter that fact.
Your theory might be more intuitive than my cold weather one, but the statistical inference you draw is equally invalid.
Inference from correlation is a dangerous thing. The same technique can be used to show black people are stupid (things that really explain poor educational outcomes – poverty; low self-esteem; parents’ education level – are correlated with being black). The p-value on a correlation between being black and educational achievement is very low, but my ‘black’ variable is picking up the influence of a host of other factors which, when accounted for, make ‘blackness’ statistically insignificant in determining educational achievement. Equally, in your model, the seat majority variable is picking up the influence of other explanatory factors; until they’re included, you can’t say anything about a causal relationship between majority and profligacy.
As for tests of the significance of any causal relationship (i.e. the p-value), these must fulfil a set of requirements to be valid, which I won’t go into.
I am sorry if this sounds rather blunt and I’m sure this was never your intention, but your misunderstanding of statistics seems has misled people. Increasingly, numbers are used to apply a veneer of ‘scientific’ credibility to arguments, perhaps because people take them at face value more readily than words. In that sense, a bad quantitative ‘argument’ can more pernicious than a bad qualitative one.
What you have is a theory, and, separately, two variables that move together.
I am not criticising the theory per se; my point is that the statistical analysis done thus far lends absolutely no support to it, even leaving aside the fact that the DT’s sample is not representative. That is a matter of fact, not opinion.
GT - Thank you for your very detailed analysis.
Well as I said I am not a statistican and I am keen for others to take this further to see if it stands up.
Would you be willing to perform a more rigorous analysis that takes into account these other variables?
As someone who earned his living for some years as a statistician, I distinguish 4 classes of statistics:
- 'damned lies'. Ben Goldacre is a current champion at nailing those.
- 'lies'. Darrell Huff's master work is the best guide.
- 'statistics'. The great mass of data where you have to pick your way very carefully, and sometimes very sophisticatedly, to avoid false conclusions.
- 'stub your toe numbers'. These are the statistical facts which look blatant once you see them. The way to treat them is not as certainties, but as something that you can assume has meaning unless proved otherwise.
Mark's figures look to me like stub your toe numbers - too blatant to ignore. Whether they mean that big majorities, long service or political prominence are most associated with likelihood of proving corrupt remains to be investigated. (Other factors may also be found to correlate with majorities) That investigation will be better done with a fuller data set; but that does not mean not doing what we can with this one.
Crucially, we are not looking for factors which may cause corruption, we are looking at factors which may tend to permit it to flourish. On Mark's initial figures, our electoral system is a factor tending to permit corruption until proven otherwise.
(In passing, I would say that closed party list systems of proportional representation may well be demonstrably even more likely to permit corruption)
As a proper Statistician I can confirm that the Mann-Whitney test is the correct approach here. The calculation is based on rank ordering in exactly the same way as what you did in your original analysis. The T-test isn't quite valid because your data is binary as Andy H points out and some would argue the 'large' sample assumption is tenuous.
@dheigham
The strongest I would put it is that it's worth following up, but it certainly isn't worth the "guilty until proven innocent line" that you've taken. That smacks very much of having made up your mind first now cherry pick evidence to fit it. Apart from the very well covered point that there is plenty of scope for recruitment bias and that the DT's criteria are unknown and also subject to bias, then the most obvious thing is that the correlation is with "safe seat". Now do we have evidence that alternative systems don't produce safe seats also? There are a myriad other ways whereby political machinations can lead to corruption. Second, is there evidence that political systems that use fptp are, on the whole, more corrupt than those that use the others.
I find the rush to conclusions concerning. Even Ben, when he strays outside his specialist areas and into that of social policies shows clear signs of selecting the stories which suit his agenda - the micro-blog has several of them. Social and political policy is a vastly more complex topic than even medicine. It's an endless supply of misleading stats to support opposing positions.
This story is an interesting observation which ought to spark some more robust investigation. But it's nothing more. Judged by any of the standards of controlled experiments in Ben's book it falls short on many measures. That it plays into a lot of people's pre-decided views is just one of them.
For the record, I would prefer a more accountable system than the current one. (And there are many much worse then fptp). However, I'm not convinced that this is as important as a transparent system - the MEPs are voted in by proportional systems (of various sorts), and that has an even worse expenses system with its own corruption scandals.
GT makes makes very good point re: inferred causation and the correlation being via something else such as length of tenure for example. If you could get this info there are some fairly simple methods where you can 'partial out' or allow for other potential effects. This might give some more weight to your findings if you can be arsed.
Regarding the p values given in the original post:
I am by no means a statistitian, but isn't something amiss if a non-parametric test gives a smaller p value than a parametric test of the same data, assuming they are both applied correctly?
Can I congratulate you on actually doing some empirical analysis to back up what otherwise is just speculation. However, I think your analysis is flawed, and has as much chance of being wrong as right. As such, I don't think it can bear the conclusions you place on it. There are at least two key problems, from what I can see.
1) The sample, as a number of people have pointed out, is not random and is probably skewed.
As important, your sample only includes data on those MPs that the Telegraph has looked at AND judged to be dodgy. You haven't got (or considered) information on those MPs the Telegraph has looked at and have cleared (you mentioned Norman Baker has being cleared, but there is probably many others who the Telegraph haven't named; also you removed Baker from your sample which was, I think, I mistake).
The group of reviewed and cleared MPs has just be clumped with the MPs who have not been reviewed. So your probabilities are a compound of two, possibly linked, events: being chosen by the Telegraph for review and being found to have exploited expenses rules. The only way to separate the probabilities of these events is to get the list of MPs reviewed and cleared. Even then there is probably some other problems, but without this data, you're going nowhere.
2) There are also likely to be a number of factors that effect the chances of an MP exploiting expenses. Safeness of seat seems plausible, but so does the length of time an MP has been an MP (old lags being more likely to get up to no good compared to bright new things). As these two factors may be correlated (MPs in safe seats stay around a long time), the significance of one may be overestimated if you do not control for the other. You can get round this with a multivariate regression, as someone in another comment suggested, using something like a probit model (see http://bit.ly/CISt3). But again, this will only work if you get the data problems sorted out.
So overall, I applaud your efforts and you have an interesting hypothesis that corruption and safeness is linked, but your analysis doesn't really take it forward. I have further doubts whether there is a logical link from this to systems of PR are less corrupt; you really haven't presented any evidence. But I'll leave that you another time.
Jody: Yes, quite possibly, which is why I'd say the non-parametric test is the more valid.
I have removed an earlier comment from William Lee at his own request.
This site seems to get a good amount of visitors. Congratulation for the great post.
www.n8fan.net
Post a Comment