From http://www.publicagenda.org/
"There are about 209 million adults in America, of every imaginable background and circumstance. So how can a survey of only 800 or 1,000 adults reflect what the entire country is thinking? How can a thousand voices speak for us all?
Public opinion researchers liken it to making a big pot of soup — to taste-test the soup, you don't have to eat the whole pot, or even a whole bowl's worth. You only have to try a bite. The same is true of public opinion. You don't have to ask every single person in America to find out what Americans think; you only need to ask a few to get the flavor of public opinion.
This fact is reflected by a survey's margin of error, or sampling error. When public opinion researchers report the margin of error for their polls (usually expressed as something like "plus or minus 3 percentage points") they are stating their confidence in the data they have collected. The lower the margin of error, the more accurately the views of those surveyed matches those of the entire population.
You must also remember that every margin of error has a "confidence interval," usually 95 percent. That means that if you asked a question from this poll 100 times, 95 of those times the results would be within 3 percentage points of the original answer. Of course, this means that the other five times you ask the question, you may get answers that are completely off the wall.
For example, if 50 percent of a sample of 1,000 randomly selected Americans said they favor recycling laws, in 95 cases out of 100, 50 percent of the entire population in the U.S. would also have given the same response had they been asked, give or take 3 percentage points (i.e., the true proportion could be 47 percent or 53 percent).
The bigger the sample, the smaller the margin of error, but once you get past a certain point -- say, a sample size of 800 or 1,000 — the improvement is very small. The results of a survey of 300 people will likely be correct within 6 percentage points, while a survey of 1,000 will be correct within 3 percentage points, a lower margin of error. But that is where the dramatic differences end — when a sample is increased to 2,000 respondents, the margin of error drops only slightly, to 2 percentage points.
Despite this, some surveys have sample sizes much larger than 1,000 people. But why ask two or three thousand respondents when 800 will do? Well, it sounds more impressive, but that's hardly worth the cost of interviewing all those additional people. Usually when a study has a large sample, it is so certain subgroups — like parents or the elderly — can be teased out and compared to each other or to the whole. If you want to compare retired people to the general public, for instance, a sample of 1,000 might yield only one or two hundred people who are no longer working, which may not be enough to get a solid grasp on the views of that group. A sample of 2,000, however, will probably yield a larger group of retired Americans, and provide a more accurate picture of their views.
Sometimes increasing the sample size is not enough, if the subgroup you are examining is rare or particularly hard to find. Young black men, for example, make up only a small percentage of the U.S. population. In a standard random sample, you would have to interview an enormous number of people before you had a large enough subgroup of young black men. In this instance, you would take an "oversample," purposely seeking out members of the particular group you are interested in, and comparing the results to the main sample.
Of course, in both general samples and oversamples, who is asked is as important as how many are asked. Reputable survey organizations go to great lengths to make sure their interview sample is random and representative of whomever they are surveying, be it retired people, young black men, or all Americans. For more information on random sampling techniques and other important aspects of polling, see 20 Questions Journalists Should Ask About Poll Results.
The convention for survey researchers is to report sampling errors that are based on a 50 percent split, where the margin of error is largest."
The sample size is legitimate.
