/
Questionnaire Design Van Questionnaire Design Van

Questionnaire Design Van - PowerPoint Presentation

trish-goza
trish-goza . @trish-goza
Follow
433 views
Uploaded On 2018-02-22

Questionnaire Design Van - PPT Presentation

Bennekom Chapter 4 Identifying the questions to ask What are concerns and interests from your evaluation U pstream shareholders Eg monitor service quality to avoid future problems Different managers might have different issues ID: 633963

questionnaire questions interview scale questions questionnaire scale interview question scales group structured identifying step formats personal focus interviews page

Share:

Link:

Embed:

Download Presentation from below link

Download Presentation The PPT/PDF document "Questionnaire Design Van" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.


Presentation Transcript

Slide1

Questionnaire Design

Van

Bennekom

, Chapter 4Slide2

Identifying the questions to ask

What are concerns and interests from your evaluation

U

pstream shareholders

E.g., monitor service quality to avoid future problems

Different managers might have different issues

Immediate recipients

The direct target, truly reflect their concerns (otherwise, low response rate or wrong answers)

Personal interview: interview representatives

Focus groups: small group interviews

Related literatures, documentations, white papers, reports, etc.

Downstream customers

Might be the indirect cause, their influence to the immediate recipients

Follow your statement of purpose with clear definition of scopes and goals.Slide3

Identifying the questions to ask

Personal Interviews

Defining a set of interview questions that helps elicit the survey purpose

Semi-structured questionnaire (creating your questions during the interview)

Questionnaire: add more questions, alter the sequences of questions, change wording, Slide4

Identifying the questions to ask

Personal Interviews

How to take the notes

Using tape recorder (please check the legal issues)

Start with the statement of the interview

The best notes are quotes from the interviewee.

How many interviewers should we interview?

1 to many

Who should conduct the interviews

Someone with the interview training

Person who will design the questionnaire, conduct the surveySlide5

Identifying the questions to ask

Personal Interviews

How should the interview be sampled

Purposive sample (most representative, hard and critical)

Where should these interviews be conducted

Convenient for interviewees

Face-to-face is the best

How long should these interviews last?

30min-one hour

When should we stop

When you stop hearing new thingsSlide6

Identifying the questions to ask

Personal interview

Pros

Deep in context

Explore unexpected paths

Paints the initial picture

Cons

How structured a questionnaire?

How many interviewers

By phone or in person

lengthSlide7

Identifying the questions to ask

Focus group

To have a directed discussion with a small group of people

The dynamic interaction with groups can bring some issues or ideas that would have been missed or ignored during personal interview.

Time efficient

How many people should be in a focus group

7-15Slide8

Identifying the questions to ask

Focus group

Who should be invited

Purposive sample, attendee covers different fields

Who should not be invited

Limit the number of observers

What is the role of the moderator?

Help to bring out all sides

Encourage members to discuss/present their ideas

Control an overbearing opinion leader

What type of questionnaire should be used?

Same as personal interview: semi-structured Slide9

Identifying the questions to ask

Focus group

Where should the focus group be held and how should the discussion be captured.

A comfortable meeting room

Audio taping

How many focus groups should be conducted?

Until you did not hear new information.

How long should it last

Two hours, morning is better than afternoon, evening is also good

Serving a light snack is always good.Slide10

Identifying the questions to ask

Focus group

Pros

Deep in context

Explore unexpected paths

Interaction among participants

Cons

Planning & execution critical

Good moderator critical

Have one central theme

How many to invite

Whom to invite

Time consuming textual analysisSlide11

Documentary Data

Existing documents

White papers

Related literatureSlide12

Analyzing data

Doing content analysis of the textual notes

Boiling down the words into categories, supplemented with a few well-chosen statements

Identify patterns: group similar comments to reveal patterns

Some free-form comments can address the “why” for a question

Literal transcript of every word is not necessary, only capture key points and key focuses.

Connect words to interviewee’s background

A distillation processSlide13

Drafting the questionnaire

It is an iterative process:

Step 1: Organizing your essential findings from the interviews, group them into logic groups

Step 2: Consider three basic types of questions (e.g., demographic, specific, overall)

Step 3: Select question formats: scale type (limit the number of different scale types), open-end

Step 4: Draft some questions

Step 5: Take a break

Step 6: Repeat steps 3-5.

Step 7: seek other input (look for external reviewers to go through your questionnaire, ask domain experts)

Step 8: Pilot test your draft questionnaire with a representative sample (critical stage)Slide14

Elements of a questionnaire

A questionnaire contains several components

An introduction

Instructions

Initial questions

Sections and their headings

Summary

A thank youSlide15

Elements of a questionnaire

Introduction

Personally addressed to the respondent

To explain the purpose of the survey

Who should complete the survey

The expected time to complete the survey

Emphasize anonymity, if appropriate

What to do with the completed questionnaire

When to do the survey (now!)

Definition of certain terminology

Example: Van

Bennekom

, p75Slide16

Elements of a questionnaire

Instructions

How to enter answers

Explain the scales

Example: Van

Bennekom

, p78Slide17

Elements of a questionnaire

Initial questions

Pay attention to the initial questions (engage them, subject matter, motivation to move ahead, easy to answer)

Never open with demographic questions in the beginningSlide18

Elements of a questionnaire

Section headings

Should be short

Group 50 questions into 3 section vs. put all 50 questions in one section.Slide19

Elements of a questionnaire

Summary and Thank youSlide20

Sequencing questions

From general to specific

Initial questions should be general

Demographic questions

Do not ask questions which data you do not need

Question interaction

Connection of different questions

Rotating questions

Branching: direct respondents to the right sectionsSlide21

Question formats

Unstructured

Open-ended, free-form

Limited the number of open-ended questions (respondent burden)

Post them after certain structured questions about key theme, or ask them to clarify or expand on previous answersSlide22

Question formats

S

tructured (see page 85)

multiple choice or categorical,

ordinal scales,

interval-rating scales,

ratio scalesSlide23

Question formats

Structured: multiple choice or categorical,

For frequency distribution (e.g., job title, company size)

Kinds:

Multiple choice, multiple response

Multiple choice, single response

Binary choice

Adjective checklist (page 90)

Design issues:

Range of choices in response sets (include other category)

Distinctions among choices

Number of choices (max 6-8)

Sequencing of choicesSlide24

Question formats

Structured: Ordinal scales (page 93)

Basic ordinal scale: age range, or salary range

Forced-ranking scale: ask respondent to rank some items

Paired comparison

Design issues

Number of items

Clarify of instructions on what the rank meansSlide25

Question formats

Structured: Interval-rating scales (page 95)

Such as: level of agreement

Verbal scales: scale in text

Numeric scales: scale in number

Likert

-type scale: strongly disagree-disagree-neither agree nor disagree-agree-strongly agree

Many others (page 98-100)Slide26

Question formats

Structured: Interval-rating scales (page 95)

Design issues

Advantages of grouping by type

Which step first, developing scale or writing questions?

Importance of anchor choice

Scale direction (1 is best or 5 is best?)

Interval equality

Anchor balancing (extremely dissatisfied, vs. very satisfied)

Even (1-7) vs. odd (1-6)-numbered scales

Number of points on the scale: 10 points are too much.

Presenting the scale low to high or high to low

Use of multiple scale types creates confusion

Avoiding response rutsSlide27

Question formats

Structured: Ratio scales (page 108)

Fractionation scale: open-ended scale, no upper limit, has better precision.Slide28

General guidelines

Goal

Maximizing the information we receive

Minimizing response burden

Do not ask meaningless questions

Keep the questionnaire focused

A survey is not a test (instruction should be clear)

Be concerned about the visual and layout of your questionnaire

Keep wording and grammar simple

Be creative in question designSlide29

Pilot testing

Don’s skip it

It is not complicated

It should be conducted as one-on-one or personal interview on purposive sampled targets

The purposes are

To know whether questions are well designed, difficult to answer, or have proper response choices

Engage the respondent?, too long? Clarity of instructions, find awkward wording