1. Check to See if the Findings Already Exist

Often there are larger bodies that already have the particular statistics you are looking for or have similar statistics that could be applied to your situation or area of concern. Larger surveys like those conducted by Statistics Canada, are carefully conducted in such a way that conclusions about the population you are concerned with might be extrapolated. Sometimes, a custom tabulation may be possible for a reasonable fee. A review of likely locations of such findings should be the first step for any project requiring this type of data. Government ministries, web sites, and offices are good places to start looking.

2. Ensure That Your Sample is Random

"Random" is often confused with "haphazard". To achieve a defensible representative sampling you must pay close attention to selecting the respondents in a truly random fashion. What this means is two things: 1. That the respondents were selected without bias, and 2. That the entire population had an equal opportunity or chance to respond to your survey. What this means in practical terms is that you do nothing in your methods that favours certain segments of the population in question, nor do you put any barriers in place against the participation of o thers.

For example: a phone survey of parents in a community conducted between 9 and 5 would skew the respondent pool away from working parents. Another example would be to do client interviews within the offices of an agency that offered both "walk-in" and "in-home" programming. This would skew the results in favour of the clients using the walk-in services and exclude those who don't usually come in to the office. Other concerns can be language barriers, literacy, access to a phone or transportation, privacy concerns existing relationships between the interviewer and the respondent, timing cycles, etc.

Suffice it to say that establishing a "random" sampling is anything but a haphazard process and any unaccounted-for introduction of bias will weaken the results. Conversely, a truly random sampling is a very costly and time-consuming investment, one often beyond the capability of one student working alone.

3. Ask Only the Most Important Questions

Each question on a survey represents a large investment in time in terms of asking the question, recording the responses, tabulating the results, drawing conclusions and establishing findings. Always return to your Key Research Questions to assess whether each particular question contributes directly to the big-picture results you are after. Further more, surveys that are too long are seldom completed thoughtfully or correctly, or completed at all. An incomplete or improperly completed survey weakens your findings by reducing your sample and bringing your methods of delivery into question. A survey that takes 10 minutes or longer to answer is a long survey indeed.

4. Encourage Answers You Can Use

It is tempting to allow for the most accurate answers to come in on an individual basis. Questions like "describe your experience with our agency" asked of 300 respondents will result in 300 unique responses. Although this might be interesting reading, it is difficult to produce a finding. There can be no clear statistically representative conclusions drawn and any qualitative summaries a researcher can make will be fraught with personal bias. Questions like "which of the following best describes your experience with our agency" and providing several options can result in strong findings like "75% of respondents found our agency lacking in one-on-one counselling" Or "only 10% of respondents said they would not use our services again". If your sample is carefully constructed, these conclusions can be applied to the entire population being studied.

5. Predict the Results

This might sound wrong when you are striving for unbiased findings, but all research is done with a starting set of assumptions about the population and about the answers you will likely receive on your survey. Making predictions serves two purposes: 1. It brings your opinions out in the open and makes the efforts you have made to account for this bias clear and evident. 2. It gives you an opportunity to re-think some of the questions you are asking in terms of how they will be perceived by the respondents and how useful the findings will likely be.

6. Start Backwards

Answering the questions "who is going to use this information?", "how will they use it?" and "how will they judge its validity?" can go a long way in making decisions about the design of your research. Determine who will ultimately be reviewing the results and consult with them as to what they would like to see. If it is a funding agent, determine what criteria they judge by, if it is a board of directors; determine what decisions they will be making with your findings. Find out if you can have anonymous respondents, find out how much personal data will be required (profile of respondents, age, gender, etc.) Find out if statistical information is what you really need. If you are dealing with a small population you may be able to get in touch with everyone, or if you are dealing with an indistinct population, percentages may be meaningless anyway. If after-all, you are looking at some new ideas or just a quick bit of feedback, other mechanisms like focus groups or informational interviews might be more a more useful and efficient use of your time.

7. Record Everything

Make notes explaining why you make each decision, who you want to include in your sample, who you will exclude and why. Why you choose the methods you use, who you consulted with where did the idea of this research come from and why. Who is conducting the research, when and where. These are questions that will be asked by people who know about research and if the answers are not readily available, your findings will be suspect. Making detailed notes to justify each decision you make is always a good idea anyway, in that it will also help you make more thoughtful decisions.