Opinion polls are an ever present part of our society. All too often, in the absence of real democracy, they seem to take on the role of the “people’s voice” in public debates.

I was fascinated by the recent headline from the Verge declaring More than half of Americans think Apple should comply with FBI, finds Pew survey and from the NY Times declaring In Poll on Apple, Public Sides with F.B.I..

Clearly the people had spoken. Or had they? Firstly Apple is a global company with 2/3 of their sale from outside the US. So it isn’t clear that the opinion of its US customers should come before those of other countries.

But more importantly polls are just samplings with their own biases. What did it mean for the NY Times to declare that a majority (51%) of the respondents sided with the FBI when by Pew’s own admission they had 24% of respondents admitted to knowing nothing about the case.

I decided to run my own survey of the US internet users as well as adult internet users in Canada, Australia and the U.K. In an attempt to replicate Pew’s results I made the question exactly the same. Unsurprisingly the answer to my survey did not produce a 51% majority and is much more ambiguous with ~32% holding no opinion:

We see even more ambiguous results in the UK with ~47% expressing no opinion

Similar results from Canada

And from Australia

In all four countries, the percentage of people that thought Apple should unlock the phone was in the mid 30’s and very close given the margin of error noted in the survey (in parenthesis for each result). But in no country was their a clear majority favoring Apple unlocking the phone.

And given the high percentage of folks in all of the countries that had no opinion including the US, it seems premature for the NY Times to declare that More than half of Americans think Apple should comply with FBI. 

Why the differences? Polls are not science. A lot depends on the context and presentation. Even the questions create bias. I, in emulating Pew, asked simply about unlocking the phone. The actual request is much more complicated than that and in fact Apple needs to create a custom version of their OS. Perhaps if I had asked that the answer would have been more definite one way or the other.

As much as possible pollsters attempt to remove their own biases but even things like the way a question was asked or the order that it was asked can impact outcome. For example in the case of the Pew survey,  there were 6 questions asked prior to asking the principal question and we have no idea what five of them were. The sixth question immediately preceding the main question could also have set the stage for an inflated answer:

All surveys have a margin of error built in based on how large and representative the sample is. In the case of Pew, they asked 1000 people which is large enough to be sure that the margin of error is reasonable but it was not stated as to how much it actually was. That being said, making definitive conclusions as they do about attitudes of sub groups within the sample becomes much less reliable and will likely have a much greater (but non stated) margin of error.

Pew for example found no significant age preferences while the Google survey found a very strong age based preference:

Finally methodology affects outcome. My surveys were done using Google Survey which admittedly only sample the US Adult Internet Population.

So has the US public decided? Has the greater global public decided (which is really Apple’s audience)? As much as Pew proclaims that they have I would assert that the court of public opinion, like our Supreme Court, is pretty evenly split and nowhere near a majority.

Here are the links to my surveys so that you can examine the results for yourself:

US Survey

UK Survey

Australian Survey

Canadian Survey