Questionnaire Design Questionnaire Design

Questionnaire Design - PowerPoint Presentation

luanne-stotts . @luanne-stotts
Uploaded On 2016-06-26

Questionnaire Design - PPT Presentation

Van Bennekom Chapter 4 Identifying the questions to ask What are concerns and interests from your U pstream shareholders Eg monitor service quality to avoid future problems Different managers might have different issues ID: 378056

questionnaire questions interview scale questions questionnaire scale interview group question scales identifying step personal structured interviews focus number formats response survey page




Download Presentation from below link

Download Presentation The PPT/PDF document "Questionnaire Design" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.

Presentation Transcript


Questionnaire Design



, Chapter 4Slide2

Identifying the questions to ask

What are concerns and interests from your


pstream shareholders

E.g., monitor service quality to avoid future problems

Different managers might have different issues

Immediate recipients

The direct target, truly reflect their concerns (otherwise, low response rate or wrong answers)

Personal interview: interview representatives

Focus groups: small group interviews

Related literatures, documentations, white papers, reports, etc.

Downstream customers

Might be the indirect cause, their influence to the immediate recipients

Follow your statement of purpose with clear definition of scopes and goals.Slide3

Identifying the questions to ask

Personal Interviews

Defining a set of interview questions that helps elicit the survey purpose

Semi-structured questionnaire (creating your questions during the interview)

Questionnaire: add more questions, alter the sequences of questions, change wording, Slide4

Identifying the questions to ask

Personal Interviews

How to take the notes

Using tape recorder (please check the legal issues)

Start with the statement of the interview

The best notes are quotes from the interviewee.

How many interviewers should we interview?

1 to many

Who should conduct the interviews

Someone with the interview training

Person who will design the questionnaire, conduct the surveySlide5

Identifying the questions to ask

Personal Interviews

How should the interview be sampled

Purposive sample (most representative, hard and critical

Where should these interviews be conducted

Convenient for interviewees

Face-to-face is the best

How long should these interviews last?

30min-one hour

When should we stop

When you stop hearing new thingsSlide6

Identifying the questions to ask

Personal interview


Deep in context

Explore unexpected paths

Paints the initial picture


How structured a questionnaire?

How many interviewers

By phone or in person


Identifying the questions to ask

Focus group

To have a directed discussion with a small group of people

The dynamic interaction with groups can bring some issues or ideas that would have been missed or ignored during personal interview.

Time efficient

How many people should be in a focus group


Identifying the questions to ask

Focus group

Who should be invited

Purposive sample, attendee covers different fields

Who should not be invited

Limit the number of observers

What is the role of the moderator?

Help to bring out all sides

Encourage members to discuss/present their ideas

Control an overbearing opinion leader

What type of questionnaire should be used?

Same as personal interview: semi-structured Slide9

Identifying the questions to ask

Focus group

Where should the focus group be held and how should the discussion be captured.

A comfortable meeting room

Audio taping

How many focus groups should be conducted?

Until you did not hear new information.

How long should it last

Two hours, morning is better than afternoon, evening is also good

Serving a light snack is always good.Slide10

Identifying the questions to ask

Focus group


Deep in context

Explore unexpected paths

Interaction among participants


Planning & execution critical

Good moderator critical

Have one central theme

How many to invite

Whom to invite

Time consuming textual analysisSlide11

Documentary Data

Existing documents

White papers

Related literatureSlide12

Analyzing data

Doing content analysis of the textual notes

Boiling down the words into categories, supplemented with a few well-chosen statements

Identify patterns: group similar comments to reveal patterns

Some free-form comments can address the “why” for a question

Literal transcript of every word is not necessary, only capture key points and key focuses.

Connect words to interviewee’s background

A distillation processSlide13

Drafting the questionnaire

It is an iterative process:

Step 1: Organizing your essential findings from the interviews, group them into logic groups

Step 2:

Consider three basic types of questions (e.g., demographic, specific, overall)

Step 3:

Select question formats: scale type (limit the number of different scale types), open-end

Step 4:

Draft some questions

Step 5:

Take a break

Step 6: Repeat steps 3-5.

Step 7: seek other input (look for external reviewers to go through your questionnaire, ask domain experts

Step 8: Pilot test your draft questionnaire with a representative sample (critical stage)Slide14

Elements of a questionnaire

A questionnaire contains several components

An introduction


Initial questions

Sections and their headings


A thank youSlide15

Elements of a questionnaire


Personally addressed to the respondent

To explain the purpose of the survey

Who should complete the survey

The expected time to complete the survey

Emphasize anonymity, if appropriate

What to do with the completed questionnaire

When to do the survey (now!)

Definition of certain terminology

Example: Van


, p75Slide16

Elements of a questionnaire


How to enter answers

Explain the scales

Example: Van


, p78Slide17

Elements of a questionnaire

Initial questions

Pay attention to the initial questions (engage them, subject matter, motivation to move ahead, easy to answer)

Never open with demographic questions in the beginningSlide18

Elements of a questionnaire

Section headings

Should be short

Group 50 questions into 3 section vs. put all 50 questions in one section.Slide19

Elements of a questionnaire

Summary and Thank youSlide20

Sequencing questions

From general to specific

Initial questions should be general

Demographic questions

Do not ask questions which data you do not need

Question interaction

Rating a person, rather than a system

Connection of different questions

Rotating questions

Branching: direct respondents to the right sectionsSlide21

Question formats


Open-ended, free-form

Limited the number of open-ended questions (respondent burden)

Post them after certain structured questions about key theme, or ask them to clarify or expand on previous answersSlide22

Question formats



(see page 85)

multiple choice or categorical,

ordinal scales,

interval-rating scales,

ratio scalesSlide23

Question formats

Structured: multiple choice or categorical,

For frequency distribution (e.g., job title, company size)


Multiple choice, multiple response

Multiple choice, single response

Binary choice

Adjective checklist (page 90)

Design issues:

Range of choices in response sets (include other category)

Distinctions among choices

Number of choices (max 6-8)

Sequencing of choicesSlide24

Question formats

Structured: Ordinal scales (page 93)

Basic ordinal scale: age range, or salary range

Forced-ranking scale: ask respondent to rank some items

Paired comparison

Design issues

Number of items

Clarify of instructions on what the rank meansSlide25

Question formats

Structured: Interval-rating scales (page 95)

Such as: level of agreement

Verbal scales: scale in text

Numeric scales: scale in number


-type scale: strongly disagree-disagree-neither agree nor disagree-agree-strongly agree

Many others (page 98-100)

Design issues

Advantages of grouping by type

Which step first, developing scale or writing questions?

Importance of anchor choice

Scale direction (1 is best or 5 is best?)

Interval equality

Anchor balancing (extremely dissatisfied, vs. very satisfied)

Even (1-7) vs. odd (1-6)-numbered scales

Number of points on the scale: 10 points are too much.

Presenting the scale low to high or high to low

Use of multiple scale types creates confusion

Avoiding response ruts

Dispersion of resultsSlide26

Question formats

Structured: Ratio scales (page 108)

Fractionation scale: open-ended scale, no upper limit, has better precision.Slide27

General guidelines


Maximizing the information we receive

Minimizing response burden

Do not ask meaningless questions

Keep the instrument focused

A survey is not a test (instruction should be clear)

Be concerned about the visual and layout of your questionnaire

Keep wording and grammar simple

Be creative in question designSlide28

Pilot testing

Don’s skip it

It is not complicated

It should be conducted as one-on-one or personal interview on purposive sampled targets

The purposes are

To know whether questions are well designed, difficult to answer, or have proper response choices

Engage the respondent?, too long? Clarity of instructions, find awkward wording