Modern Australian
The Times Real Estate

.

A survey needs to involve how many people before I'm convinced?

  • Written by Adrian Barnett, Professor of Public Health, Queensland University of Technology
A survey needs to involve how many people before I'm convinced?

Research studies, opinion polls and surveys all rely on asking a number of people about something to try to extract some pattern of behaviour or predict a result.

But how many people do you need to ask for that finding to have any convincing meaning?

Before any election you’ll always hear some politician casting doubt on opinion polls, saying: “There’s only one poll that matters.” They try to make us believe that those headline-grabbing polls count for nothing compared with the real election poll of those registered to vote.

Read more: The seven deadly sins of statistical misinterpretation, and how to avoid them

But opinion polls are useful because they can give a rapid insight into people’s intentions.

Taking small samples from large populations is a valid statistical technique for getting accurate information about the wider population, for a fraction of the time and cost.

This applies wherever we have large or hard to measure populations.

Examples include quality-control checking in a factory production line, counting jaguars in Peru, and even surveying the readers of The Conversation.

So how big does a sample need to be for its results to be reliable? Well, that depends.

Margin of error

All sample estimates have a margin of error, which compensates for the imperfection of the sample compared with the population. For example, a recent Newspoll put Labor 2% ahead of the Coalition on a two-party preferred basis.

Newspoll says it surveyed 1,728 people, with a maximum sampling error of ± 2.4%. This means the largest plausible win for Labor would be 4.4% (2% plus 2.4% margin of error), but it’s also plausible it could lose by 0.4% (2% minus 2.4%).

For this tight race we might want to reduce our margin of error by increasing our sample size. But that will be costly as the gains in accuracy diminish for greater numbers. A sample of roughly 2,400 people would be needed to reduce the margin of error to ± 2%, and a massive sample of 9,600 to reduce it to ± 1%.

Quality matters as well as quantity

Survey estimates and their margins of error are only valid if the sampling has been well conducted. If the sampling is biased then larger sample sizes likely just give us high confidence around an inaccurate estimate.

Read more: Paradoxes of probability and other statistical strangeness

Survey samples are often biased because they differ from the population in important ways. With 12.6 million respondents, the 2017 same-sex marriage survey is a good example as this clearly overrepresented older people who were more likely to return their postal survey.

Fortunately, in this case the bias does not undermine the result, which was a resounding vote for marriage equality. But the estimate of 61.6% in favour of marriage equality, with a tiny margin of error of 0.03%, may not accurately reflect the opinion of the Australian population.

Unrepresentative samples also happen in clinical trials because high-risk patients are often excluded from trials for safety reasons.

One study found that 94% of people with asthma would have been excluded from the 17 major clinical trials used to write guidelines for doctors about treating the condition.

This is a serious problem, because doctors need to give advice to all of their patients, but the best evidence comes from trials that used generally healthier patients.

Similarly, imagine trying to predict how subscribers to Netflix or Stan will rate movies based on ratings from other similar subscribers. These ratings are likely to be biased, as only people who particularly like or dislike a given movie may bother to rate it.

This is an important problem to solve for online content distributors in order to provide accurate movie recommendations to customers.

How does the public judge a good sample?

There are no simple rules for judging a good sample size. Bigger is generally better, but only when the survey has been well conducted.

Some very large samples may have used cheap data collection tools, such as Facebook, and so may be highly skewed. Small surveys of just 25 people can be insightful, especially where efforts have been made to ensure a representative sample and chase people who don’t initially respond.

The Australian Press Council has guidelines on reporting opinion polls, and here are some questions you can ask yourself when reading about any survey:

  • Where were the participants found? How typical are they of the whole population of interest?

  • How many participants declined to respond? If only 10% of people responded then it is likely an atypical sample who have strong feelings about the survey’s subject. (Think about what surveys you’d likely respond to.)

  • Were the survey respondents paid? Payment will increase the response rate, but might also affect respondents’ answers.

Sadly these details are often lacking from the media releases and news reports of exciting findings from surveys, and are also often lacking from published papers.

Survey respondents can also be steered towards desirable answers. For example, a Nature survey of 1,576 researchers on the reproducibility crisis asked the question:

Which of the following statements regarding a ‘crisis of reproducibility’ within the science community do you agree with?

(i) There is a significant crisis of reproducibility

(ii) There is a slight crisis of reproducibility

(iii) There is no crisis of reproducibility

(iv) Don’t know.

A majority (52%) of people said “Yes” to a significant crisis, 7% answered “Don’t know” and just 3% “No”.

Read more: Regression to the mean, or why perfection rarely lasts

This leaves the question of what is meant by a “slight crisis”, a verdict reached by 38% of people. Did they answer slight because they are close to the “no” or “don’t know” categories, or are they close to considering it a significant crisis? We can’t tell.

The point here is that people were given two options for “yes” and only one for “no”. Yet the study was – and still is – reported as strong evidence of a crisis in science.

Overall it’s best to read the results of any survey with healthy scepticism. Our survey of the two statisticians who wrote this article showed a 100% agreement with this statement.

Authors: Adrian Barnett, Professor of Public Health, Queensland University of Technology

Read more http://theconversation.com/a-survey-needs-to-involve-how-many-people-before-im-convinced-96470

The Best Nail Art for Short Nails

Short nails often get overlooked in the world of nail art, but they’re actually the perfect canvas for stylish, sophisticated designs. Whether you...

How Much Does a Skip Bin Cost? Skip Bin Prices, Sizes & What Influences the Price

Regardless of whether you're renovating a house. You are tidying up your backyard, or running a construction site. Chances are you'll require a skip...

Panoramic Elevators: Combining Functionality with Aesthetic Appeal

In the evolving world of building design, the demand for architectural features that are both practical and visually striking continues to rise. One...

No Dig, No Drama: Sydney Pipe Relining Explained

You know that feeling when water starts pooling where it shouldn't? Or received a quote for pipe repairs that made your eyes water? Let's chat about...

The SEO Benefits of AI-Optimized Landing Pages: Smarter Metadata, Faster Load Times

Landing pages are integral to the conversion funnel in digital marketing. Yet, when SEO works well to drive traffic to the landing page, it's just a...

How to Reduce Turbo Lag with an Upgraded Mazda BT-50 Exhaust System

Are you struggling with turbo lag affecting your driving experience?  Did you know upgrading your Mazda BT-50 exhaust system could be the key to u...

How to Segment Email Lists Based on Content Engagement (Not Just Demographics)

Email segmentation generally refers to age, location, gender demographic, and conventional elements. While this could be helpful information, it doe...

6 Qualities to Look for in Local Caravan Dealers

If you're searching for reliable caravan dealers in Campbellfield, it's essential to choose a dealer that offers quality products, excellent custome...

How to Repair a Faema Coffee Machine: A Practical Guide for Coffee Lovers

Introduction If you’re the proud owner of a Faema coffee machine, you already know that it’s more than just a kitchen appliance — it’s a vi...

Smart Health Decisions in Your 30s, 40s, and 50s: The Ultimate Prevention Guide

Making informed health decisions becomes increasingly important as we navigate through different life stages. What may have seemed unimportant in our ...

Sydney’s Electricians Helping Cut Business Costs

Running a business in Sydney isn’t cheap. Between rent, wages, equipment, insurance, and endless operational costs, it’s no wonder business owne...

Can Air Conditioning Help with Allergies and Asthma?

Living with allergies or asthma can make everyday environments feel challenging—especially during certain seasons or in homes where dust, pollen, ...

A Fast Charging Power Bank and Wireless Power Bank: The Future of Power Bank Technology

As the use of mobile devices grows, so does the demand for better, more efficient, faster, and more imaginative methods to power them. Previously re...

How to Extend the Lifespan of Your Power Bank: An Overview

Power banks are vital for ensuring your devices are powered while on the go, especially when on the move, outdoors, or during emergencies. They prov...

What is a Power Bank and How To Choose One

In a world filled with gadgets running out of battery is a real issue. This is where power banks can come in useful. They're portable, they hold pow...

Portable Chargers: How They Work & Essential Tips to Extend Battery Life

Portable chargers (also known as power banks) are essential accessories that keep your devices powered up while on the go — be it outdoor activiti...

How Artificial Tulip Displays Enhance Your Home’s Aesthetic

Artificial tulip displays have grown to be a common option for improving interior decor because of their exquisite look and long-lasting quality. Th...

The Psychology of Relationships: How Therapy Can Strengthen Bonds

Relationships require effort, understanding and communication. Couples and individuals can have better relationships with the assistance of therap...