Surveys are not interviews!

Contact  Malcolm at
malcolm.mcinerney@thebartonsc.sa.edu.au
The address for the Researchit blog
http://researchittsc.blogspot.com.au/


Some guidelines and hints on surveys

What is a survey questionnaire?
Survey questionnaires present a set of questions to a subject who with his/her responses will provide data to a researcher.

The design is critical
The key to obtaining good data through a survey is to develop a good survey questionnaire. It is worth spending the time designing your survey to ensure you gain the information you want and that the persons surveyed understand your questions.

Some hints and guidelines with design

* The key to developing a good survey questionnaire is to keep it short while ensuring that you capture all of the information that you need.

* Before you begin to design your survey questionnaire, you developed a set of focus questions for your research. Your survey questions should relate in some way to your focus questions. These questions should serve as a plan for the survey and a question should be rejected if it cannot be seen as helping you answer one of the focus questions in some way.

Types of Questions:

There are two different types of questions that can be used to collect information. The first is called a fixed response question and the second is called an open question. It is important to understand when and how to use these questions when designing your survey.

1. Fixed response
These are questions that offer the respondent a closed set of responses from which to choose. Such a question makes data collection and analysis much simpler and they take less time to answer. Fixed response questions are best suited in the following situations:
• when you have a thorough understanding of the responses so that you can appropriately develop the answer choices
• when you are not trying to capture new ideas or thoughts from the respondent.

Examples of a fixed response question

Do you have a driver's license?
( ) Yes
( ) No
How many hours a day do you spend doing homework?
( ) 0 to 1 hour
( ) 2 to 3 hours
( ) 4 to 5 hours
( ) more than 5 hours

* When writing the selection of responses for a fixed response question, you should make certain that the list covers all possible alternatives that the respondent might select AND that each of the answers is unique (ie they do not overlap).

* For valid data collation it is suggested to use the "Don't know" and other option sparingly.
* You should try to ensure that your respondents are capable of answering the majority of the questions on your survey questionnaire.
* It is suggested that with some fixed choice questions it is better to add a part (b) to the question requesting other choices they wish to make or the opportunity to explain their response.

2. Scaling question (also fixed response but different)

Sometimes you will be interested in obtaining a person's opinion on a topic, subject, product, event, etc.... To capture varying degrees of emotion/opinion about a subject, it is best to use a scale question. A scaling question asks respondents to explain the degree with which they feel about a certain topic, subject, event, etc... For example:

Please describe how you felt about the Harmony day.
(1)Unsatisfied (2)Somewhat (3)Satisfied (4)Satisfied (5)Extremely Satisfied


* In some cases you may wish to have a part b to the scaling question to provide the opportunity for the surveyed to explain why they circled the number they did. Few will take this opportunity but it is worth putting it in your survey.


3. Ranking question (also fixed response but diferent again)

A ranking question asks respondents to explain how they feel about something by comparing it to other items in a list. For example:

Please rank the following Harmony Day activities in order of preference (starting with 1 for your favourite activity).

___ Music
___ Displays
___ Dancing activities
___ Food

* A ranking asks respondents to list their responses in order of preference. This type of question leads you to an answer where the respondent is comparing one thing to another rather than giving you their feeling about each individual item as was the case with the scaling question.

* The disadvantage to a ranking is that if the respondent feels the same about two or more items, they are still forced to sort them into a ranking. The results of a ranking basically tell you which is the most preferred and which is the least preferred item on the list, but you do not know from a ranking if the respondent likes or dislikes any or all of the items on the list.

* It is often a good idea to have part b to a ranking question to provide the respondent with the opportunity to state any other option/s they would have liked to rank.

* Do not have a “other”: category because it makes collation invalid.

4. Open-ended questions

Open-ended questions, are questions where there is no list of answer choices from which to choose. Respondents are simply asked to write their response to a question.

Here is an example:

Example of a open-ended Question
What else would you like to see happen on Harmony Day?

* It is best to use open-ended questions when you are exploring new ideas and you don't really know what to expect from the respondents.

* Open-ended questions let you get more insight into the respondents' thoughts and ideas about a subject.

* The disadvantages to using open-ended questions is that it can be much more time consuming and difficult to analyze the data.

* In general you should try to minimize the number of open-ended questions in your survey questionnaire.

* If you find yourself designing a survey questionnaire where the majority of the questions are open-ended, then you may need to do more exploratory research to get a better foundation of knowledge for the subject you are researching.

General tips to creating a good survey questionnaire:
Here are some tips and tricks to help you ensure you are developing a good survey questionnaire:

* Clearly state your intentions with the research.
At the top of your survey, write a brief statement explaining why you are collecting the information and reassure each respondent that the information is entirely anonymous. If you need to know specifics about a person, respect their privacy by identifying them as subject1, subject2, etc...

* Include instructions with your survey questionnaire
What may seem obvious to you probably is not very obvious to someone else. To ensure that you collect valid survey results, make sure you include instructions on how to answer the survey questionnaire. There should probably be a short introductory set of instructions at the top of the survey questionnaire, and additional instructions for specific questions as needed.
Your overall instructions may be something like:
Please mark the appropriate box next to your answer choice with an "x" ( X ). Please answer all of the questions to the best of your ability.

* Don't ask for personal information unless you need it to help answer your focus questions.

* Keep the questions short and concise
The wording for survey questions should be short and concise. Each question should be clearly stated so that there is no misunderstanding about what is being asked. The best way to ensure your questions are well worded is to test them by having other people review and test your survey before you distribute it to the full sample.
Ask only one question at a time (the double barreled question)
This is a very common mistake in survey questionnaires and one that will severely impact the results of your data. When you are writing a question, you must make sure that you are only asking one question at a time.

Here is an example of a double-barreled question:

How have teachers and students at your school responded
to Harmony Day?
( ) Satisfied
( ) Unsatisfied

It should have been:
How have teachers at your school responded to Harmony Day?
( ) Satisfied
( ) Unsatisfied
How have students at your school responded to Harmony Day?
( ) Satisfied
( ) Unsatisfied

* Make sure the questions are unbiased
When developing your survey questionnaire, you want to make certain that you are asking the questions in a neutral way, ie that you are not leading them toward a particular answer.

* Here is an example of a leading question:

Do you think that the new cafeteria lunch menu offers a better variety
of healthy foods than the old one?
( ) Yes
( ) No
( ) No Opinion

Here is a non-leading question

How do you feel about the new cafeteria lunch menu compared to the old one?
( ) The new menu offers a better variety of healthy foods
( ) The old menu offers a better variety of healthy foods
( ) The selections are similar
( ) No opinion
Ask questions that can be answered by your subjects
Make sure that the questions you are asking are questions that people will be able to answer.

* Order/group questions according to subject
If you have more than six questions in your questionnaire, then you should make an effort to organize your questions so the respondents can answer them as quickly as possible. A good way to organize the questions is to group them together by subject. This way your respondents can focus their thoughts and answer a series of questions around these thoughts. Put the open questions (not part b open questions) at the end of the survey.

* Present the questions in a clean and organized layout
A clean layout will make it much simpler for people to respond to the questions and for you to collect the data. Make sure that your method for marking answers is well explained and that your answer boxes are consistent throughout the questionnaire. Look at this example of a survey.

* Your sample
Make sure that you get a sample of adequate number (at least 20) and one that meets the sample requirements of your research i.e. your survey sample may require a gender balance or gender focus, across ages or of a particular age, across ethnic groups or just one group etc. These sample requirements will depend on the focus of your research. Discuss this sample issue with your teacher.

* Test the survey questionnaire
Once you have developed your survey questionnaire, you should conduct a small test (5 -10 people) to make sure that respondents clearly understand the questions you are asking and that you are capturing the information that you need for your study.

Types of questions for your survey design.

Suggested % in brackets after question type. This is only a guide.
1. Forced choice question:
• Yes/no (15%)
• Selection of choices (15%)

2. Ranking question (20%)
A nominated number of choices (cannot have suggested others – make that part b)

3. Scaled question (20%)
Some form of scale response from 0-10

4. Open-ended response – an open question
• As a question in itself (10%)
• As a follow up open question to either the forced choice, scale or ranking question. The question could be asking for explanation of forced choice, ranking or scale decision or other options that the interviewee would have liked to have as a choice (20%)




Activity

What do you think?
Have a look at and critique this survey about the 1960 youth. Is it good or bad?

Choosing a capability

As part of your Research Project you are required to complete the Research Project Capability task.

Developing the Capabilities Statement for your Research Project.

• Now that you have a draft proposal for your Research Project you can complete the Capability Statement for your research Project. Go on to the capability task below when you have completed the draft proposal. Hand it up the draft proposal to the teacher at the end of lesson 3.

• You are required to identify one of the SACE Capabilities best related to your Research Project (click here to see the summary of SACE Capabilities).
As the SACE document says:

“In the Research Project students choose a research topic that is based on an area of interest,and a capability (communication, citizenship, personal development, or work) that is relevant to their research. The capability for learning is integral to the Research Project for all students.”

• In this lesson I ask you to visit the SACE website at http://www.sace.sa.edu.au/subjects/stage-2/cross-disciplinary/research-project and navigate to the Research Project page at http://www.sace.sa.edu.au/subjects/stage-2/cross-disciplinary/research-project

• Read the information on the Capabilities and decide which one is best suited to your project.


• When you have read about the capabilities go back to the SACE website at http://www.sace.sa.edu.au/subjects/stage-2/cross-disciplinary/research-project and view the examples (exemplars) on Bone Marrow, Football Boots and Motorbike provided by the SACE Board. In particular, view the capability statement for each exemplar.

• After selecting the capability for your project complete the student worksheet for your capability (this is to go in your folio). Go to http://www.sace.sa.edu.au/c/document_library/get_file?uuid=f06351b5-7361-40d5-ab25-5e893979e15d&groupId=10336 to download and use the appropriate student worksheet for the chosen capability. Have this ready for class on Thursday.

Putting the Folio together

Contact Malcolm at
malcolm.mcinerney@thebartonsc.sa.edu.au
The address for the Researchit blog
http://researchittsc.blogspot.com.au/


Assignment 3: Putting the Folio together

Thanks everyone for some great discussion sessions during the last week of Term 3.  They really helped to make sure everyone knows what is going on. As well as conducting your research and possibly commencing writing your report outcome, you should be well underway putting together your Folio. 

Here are the guidelines for the Folio again:

• 10 pages in length
• The 10 pages of the Folio should be something like this:

1. Pages 1 and 2: Your Proposal written as a 2 page statement of what you have done in terms of * your  topic *focus questions *why you chose the area of research * the details of the capabilities you have chosen * what primary research you have conducted (interviews/surveys) *the ethics involved in the research.
2. Page 3: Your timeline clearly showing what you have done and when.
3. Page 4-6: The key secondary sources you have used - a SnagIt 8  screen capture of the website or  article (not too big a capture image) with an explanation (annotation) of what focus questions it helped answer for you and how useful it was - include some of the most useful information from the source.  You may wish to also annotate in reference to bias/limitations of the source if you see as relevant.
4. Page 7-8: The primary research you conducted.

• If you did an interview, put in the questions asked, who you interviewed, how long, where etc, discuss how useful the interview was in addressing the focus questions.
• If you did a survey, put in the questions and what type of response was asked, how many surveys were conducted, how did you decide your sample, how did you get your surveys completed, what focus questions were answered in the survey and how useful was the survey.   
• You may insert a screen capture of your survey.

     5.  Pages 9-10: The analysis sheets for your secondary and primary sources (as per handout on the Moodle).

At this stage it would be good to look at the examples of Folios provided by the SACE Board.

* Bone Marrow Folio: example withour SACE Board comments
* Moped Folio: A grade example with with SACE Board comments
* Ned Kelly Folio: example with SACE Board comments
* Grandma Folio: A+ example
* Football boots: B- example
* Motorbike: C+ example


*** Note that these examples do not necessarily follow my suggested 10 page structure outlined above.  I have just put that together to help you organise the Folio and you are welcome to follow other examples if you wish. ***

Good luck with this work everyone.  This is assignment 3 of the Research Preparation class.

The interview

The type of interview you will be doing for the research Project is called a semi-structured interview. Such an interview is the best to get specific information and a range of views. They are the type of interview most commonly used when gaining qualitative research. The semi-structured interview invites responses to a set of guided questions but also allows for interaction and follow-up discussion. Although mostly open-ended questions, several closed factual questions may be involved. Finally, with such an interview the interaction between interviewer and interviewee is much more relaxed than with a structured interview. Here is a useful YouTube video about semi-structured interviews.


BEFORE THE INTERVIEW

* Make sure you understand the purpose of the interview so that the appropriate questions are asked.
* Understand what information is necessary to complete the investigation?
* Do detailed research about the topic so you know what questions to ask.
* Choose the person to be interviewed carefully to ensure they are a creditable source of information.
* Prepare your equipment, such as cue cards, tape-recorder with microphone, video etc.
* Call and make the appointment for the interview and tell your interviewee about the topic of your investigation.
* Ask if they would like the questions to be sent ahead and ask permission to use tape-recorders or video.

DURING THE INTERVIEW

* Choose a quiet location free from interruptions.
* Begin by explaining the purpose of the interview.
* Listen very carefully to what the person is saying(Often their responses will give you an idea for a follow-up question that you hadn't planned to ask).
* Don't be afraid to ask for more depth or greater explanation.
* Try and show your interest by your body language.
* Don't concentrate on your notes all the time.
* Towards the end of the interview you can ask the person if there is anything that you have omitted or anything they would like to add.
* Let them know the interview is drawing to a close by saying 'One last question'.

Remember the success of your interview depends on your research, choice of interviewee, your genuine interest in what they have to say, and your own skills in thinking on your feet.

Developing interview questions

Contact me at malcolm.mcinerney@thebartonsc.sa.edu.au
The address for the Researchit blog
http://researchittsc.blogspot.com.au/
Course Calendar for your time management


Preparing the questions to ask.

Finally it is time to get our interview questions together. The following ideas will help you determine the type of questions to ask and how to arrange them to find out what you require for your research project.

Before you start to design your interview questions and process, clearly articulate to yourself what need is to be addressed using the information to be gathered by the interviews. This helps you keep clear focus on the intent of each question.

Most importantly the interview questions must draw out what you need to find out to answer your research topic.

Types of questions

Mostly design open-ended questions are asked during interviews. Avoid closed questions. These are questions that ask for a limited response eg. “Is this a good computer?” A response to a closed question has only two possible responses: 'yes' or 'no'. Neither answer will help you much, because you won't know how the interviewee is deciding on his or her answer. Does a 'yes' mean that the computer is good value for money, or best for games or terrific for a boat anchor or ... Such questions start with words like “Is, did or are”

Another point in relation to open questions is that such questions do not have any restrictions. eg. “What are the advantages and disadvantages of this sort of computer”? Such questions allow an interviewee to make a complete response which expresses their opinion honestly and in detail giving you access to large amounts of information. Open questions usually start with words like “How, what, when, where, why”


There are thought to be six kinds of questions. One can ask questions about:
1. Behaviors - about what a person has done or is doing in relation to your research 2. Opinions/values - about what a person thinks about a topic
3. Feelings - note that respondents sometimes respond with "I think ..." so be careful to note that you're looking for feelings
4. Knowledge - to get facts about a topic
5. Sensory - about what people have seen, touched, heard, tasted or smelled in relation to a topic
6. Background/demographics - standard background questions, such as age, education, ethnicity etc.

Sequence of Questions

1. Get the respondents involved in the interview as soon as possible.
2. Before asking about controversial matters (such as feelings and conclusions), first ask about some facts. With this approach, respondents can more easily engage in the interview before warming up to more personal matters.
3. Intersperse fact-based questions throughout the interview to avoid long lists of fact-based questions, which tends to leave respondents disengaged.
4. Ask questions about the present before questions about the past or future. It's usually easier for them to talk about the present and then work into the past or future.
5. The last questions might be to allow respondents to provide any other information they prefer to add and their impressions of the interview.

Wording of Questions

1. Wording should be open-ended. Interviewees’ should be able to choose their own terms when answering questions.
2. Questions should be as neutral as possible. Avoid wording that might influence answers, e.g., evocative, judgmental wording.
3. Questions should be asked one at a time.
4. Questions should be worded clearly. This includes knowing any terms particular to the program or the interviewees’ culture.
5. Be careful asking "why" questions. This type of question infers a cause-effect relationship that may not truly exist. These questions may also cause respondents to feel defensive, e.g., that they have to justify their response, which may inhibit their responses to this and future questions.
6. Ask questions which allow the interviewee to do at least 70% of the talking. For the most part, avoid questions that can be answered "yes" or "no." The best questions are ones in which the interviewee has the opportunity to provided detail through elaboration i.e. goes on to explain why and what they think.
7. Phrase your questions so that the desired or "right" answer is not apparent to the applicant. Don’t ask leading questions, like, “don’t you think?”
8. Ask the easy questions first so as to make the interviewee feel comfortable.
9. Alternate between easy, non-threatening questions and more difficult, pointed ones.


Linking interview questions to research focus questions.

Finally, the interview questions developed must have a direct relation to the focus questions you developed for your research. If the question cannot be seen as linked to the focus questions, then don't ask it, unless just a "getting to know you" introduction type question.

Use the attached interview questions template to design your questions.

Primary and Secondary research

Contact me at malcolm.mcinerney@thebartonsc.sa.edu.au
The address for the Researchit blog
http://researchittsc.blogspot.com.au/


Gathering and harvesting!

Regardless of the topic, you should be doing secondary research at this stage to establish your knowledge and understanding of the topic you have chosen to research. Once you have comprehensive secondary knowledge it is then time to start deciding on the primary research you wish to conduct.

Just a reminder about the two types of research:

* Secondary research is based on the findings from other people's research. It involves the gathering of the results of other's research from books, reports or the Internet. Selections or summaries are made of the research allowing for evidence to be gathered to support your conclusions.

Secondary research may include:

* statistical analysis where information is readily available from the census studies, Australian Bureau of Statistics, local councils and other government bodies, is analysed to give a notion of the need for a particular target market for a project. This may be useful for establishing if there is a genuine need for a project.

* information research, including all forms of print, that is, texts, magazines, journals, pamphlets. It also includes electronic sources. These need to be checked for reliability and relevance. Anyone can publish on the Internet. Print sources should not be too out of date. Use your school and local librarians, they are trained to help you find information.


* Primary research is the research you generate by asking questions, conducting trials and collating results. This research can take the form of quantitative research ('countable' data collection) or qualitative research(opinion/knowledge data gathering).

The most common way of collecting primary data is through surveys/questionnaires and interviews.

* A survey is usually general and covers a wide range of issues. It is designed to find information rather than to investigate specific questions about an issue. We tend to use surveys when we don't know about something and we want to identify the most important ideas, questions and issues.

* A questionnaire usually focuses more on a particular topic or issue. We tend to use these when we know something about the topic and we have some hunches about what might be the most important issue or questions to investigate.

* Interviews can be face-to-face or over the telephone or Internet. It is crucial to have a list of questions prepared. This helps prevent being side tracked and ensuring the information you require is collected. These questions may provide insight into the development of your project, as you should endeavour to seek expert advice. After all designers often work as part of a team when brainstorming ideas and solutions to problems. Many have spent their lives building up knowledge in specific areas. The yellow pages are an easy way to get in touch with such experts.


Considering the secondary research you will do for your assignment, I suggest you use the more specific questionnaire and interviews as part of your primary data collection.

Guiding your research

Contact me at malcolm.mcinerney@thebartonsc.sa.edu.au
The address for the Researchit blog
http://researchittsc.blogspot.com.au/


Getting a focus with your research

At this stage you should have submitted the draft proposal and discussed it with me. Tuesday and Thursday's lesson next week are dedicated to this task as you commence secondary research on your project.

Regardless of the topic, you should be doing secondary research at this stage to establish your knowledge and understanding of your chosen area. Once you have comprehensive secondary knowledge it is then time to start deciding on the primary research you wish to conduct.

Before doing that I require you to develop the guiding questions (sometimes called focus questions)for your Research project. These are the questions which guide your thinking and guide your research.

Research Project question/issue/challenge

Guiding Question 1:
Guiding Question 2:
Guiding Question 3:
Guiding Question 4:
Guiding Question 5:

You should not have more than 5 guiding questions.

Here are some samples to consider before trying yourself with your Research Project.

Research Project question/issue/challenge
Is cricket still the sport at the centre of the Australian culture?

Guiding Question 1:
What role has cricket played in the history of Australia and the development of the Australian identity?

Guiding Question 2:
How many people play cricket in Australia and is there a pattern ot participation?

Guiding Question 3:
What are the statistics in relation to media coverage of cricket?

Guiding Question 4:
Does the Australian public have a good knowledge of cricket happenings?

Guiding Question 5:
How achievable is it for young people to play cricket in their local community?


or


Research Project question/issue/challenge
Has cricket become the sport of the minority in the Australian community?

Guiding Question 1:
How many people play cricket in Australia and is there a pattern of participation (who plays cricket)?

Guiding Question 2:
Is cricket still a high rating TV sport. What are the ratings?

Guiding Question 3:
What does a young person have to do to play cricket?

Guiding Question 4:
How accessible and achievable is playing cricket for all young people.

Guiding Question 5:
What are the challengers for individuals to continue to play cricket beyond the teenage years?


As for the formal proposal, here is the proposal template to complete.