Research Design, Types Of Data, - PowerPoint Presentation

Slide1

Research Design, Types Of Data, Data Collection Methods

Zulkarnain Lubis

Slide2

Research Design and Tactics

Research

philosophy

Research approachesResearch strategiesTypes of DataQualitative vs QuantitativePrimary vs SecondaryCross Section vs LongitudinalData Collection MethodInterviewQuestionerObservation

Slide3

design of research

The research design is the master plan specifying the methods and procedures for collecting and analyzing the needed information

.

A detailed outline of how an investigation will take place. A research design will typically include how data is to be collected, what instruments will be employed, how the instruments will be used and the intended means for analysing data collected.A master plan that specifies the methods and procedures for collecting and analyzing the needed information

Slide4

Research Design and Tactics

The Research Onion

Saunders

et al, (2009)

Slide5

PositivismInterpretivism (Phenomenology)

RealismPragmatism

Research philosophy

Slide6

Positivism in general refers to philosophical positions that emphasize empirical data and scientific methodsVerified data (positive facts) received from the senses are known as empirical evidence; thus positivism is based on empiricismPositivism belongs to epistemology

 which can be specified as philosophy of knowing, whereas methodology  is an approach to knowing

Highly structured

methodologyUnder positivism, the science objects and scientific proposition should meet the requirements: Observable, Repeatable, Measurable, testable, predictableResearch philosophy: Positivism (1)

Slide7

Research philosophy: Positivism (2)In positivism studies the role of the researcher is limited to data collection and interpretation through objective approach and the research findings are usually observable and quantifiable.Quantifiable observations that lend themselves to statistical analysis Positivist studies usually adopt deductive approachThe researcher is independent and neither affects nor is affected by the subject of the researchStudies with positivist paradigm are based purely on facts and consider the world to be external and objective

Slide8

Research philosophy: Interpretivism (Phenomenology) (1)Interpretivism, also known as interpretivist involves researchers to interpret elements of the

studyInterpretivism integrates human interest into a studyInterpretivism is associated with the philosophical position of idealism, and is used to group together diverse approaches, including social constructionism, phenomenology and

hermeneutics

Interpretivism studies usually focus on meaning and may employ multiple methods in order to reflect different aspects of the issueInterviews and observations are the most popular primary data collection methods in interpretivism studies.  Secondary data research is also popular with interpretivism philosophy.

Slide9

Interpretivism may refer to: interpretivism (social science) an approach to social science that opposes the positivism of natural sciencequalitative research, a method of inquiry in social science and related disciplines

interpretivism (legal), a school of thought in contemporary jurisprudence and the philosophy of law

Interpretivists

avoid rigid structural frameworks such as in positivist research and adopt a more personal and flexible research structures (Carson et al., 2001), Interpretivists receptive to capturing meanings in human interaction (Black, 2006) and make sense of what is perceived as reality (Carson et al., 2001)Interpretivists believe the researcher and his informants are interdependent and mutually interactive (Hudson and Ozanne, 1988). Research philosophy: Interpretivism (Phenomenology) (2)

Slide10

Research philosophy: Interpretivism (Phenomenology) (3)Business situations are not only complex, they are unique, a particular set of circumstances and individualsTo discover ‘the details of the situation to understand the reality or perhaps a reality working behind them’, associated with “constructionism” or “social constructionism” – reality is socially constructed

People place different interpretations on the situation, in order to make sense of and understand motives, actions and intentions of other people

Slide11

Realism, in philosophy, the viewpoint which accords to things which are known or perceived an existence or nature which is independent of whether anyone is thinking about or perceiving them Based on a belief that reality exists, independent to human thoughts and beliefsRealism mainly concentrates in the reality and beliefs that are already exist in the environmentTwo main

approaches (McMurray, Pace and Scott 2004): Direct; what an individual feels, see, hear, etcCritical realism; individuals argue about their experiences for a particular situation (

Sekaran

and Bougie 2010)Research philosophy: Realism (1)

Slide12

Research philosophy: Realism (2)Scientific realism is the view that theories refer to real features of the world Its philosophical position is that reality exists independently of the researcher’s mind, that is, there is an external realitySocial objects or phenomena, external to or independent of individuals affect the way people perceive their world, whether the are aware of them or not

Business research is often a mixture between positivism and interpretivism, reflecting the stance of realism

Slide13

Pragmatism, despite many variants, essentially means that we come to know the world through the practicality or usefulness of objects (or concepts)Pragmatists contend that most philosophical topics—such as the nature of knowledge, language, concepts, meaning, belief, and science—are all best viewed in terms of their practical uses and successes.Research philosophy: Pragmatism (1)

Slide14

pragmatism and it is characterised, in most versions (1)Pragmatism treats knowledge, concepts and 

values as true if they are usefulThe pragmatists rejected the 

rationalist

 view that reality is static and fixed and preferred a view of a changing, dynamic realityPragmatism is primarily empiricist and inductive, testing hypotheses, prioritising experience, although not assuming that facts exist ‘out there’ waiting to be discovered Pragmatism is opposed to doctrines that hold that truth can be reached through deductive reasoning from a priori groundsPragmatists were content with probabilistic relationships rather than with deterministic ones.Pragmatists, though, opposed the notion of passive objectivity. 

Slide15

Pragmatism adopts a relative approach: truth is modified as discoveries are made and is relative to the time and place and purpose of inquiry The function of thought is to guide action not provide timeless abstract truths; pragmatists interpret ideas as instruments and plans of action rather than as images of reality

Thought is simply an instrument for supporting the life aims of the human organismThought is grounded in practical reality and has no real 

metaphysical

 significance, pragmatists protest against speculation concerning questions that have no application and no verifiable answers.In its ethical aspect pragmatism holds that knowledge, which contributes to human values, is real.Pragmatists also advocated human rights and individual freedompragmatism and it is characterised, in most versions (2)

Slide16

Positivist/ Postpositivist

Interpretivist/ Constructivist

Transformative

PragmaticExperimentalQuasi-experimentalCorrelationalReductionismTheory verificationCausal comparativeDeterminationNormativeNaturalisticPhenomenologicalHermeneuticInterpretivistEthnographicMultiple participant meaningsSocial and historical constructionTheory generationSymbolic interaction

Critical theory

Neo-

marxist

Feminist

Critical Race Theory

Freirean

Participatory

Emancipatory

Advocacy

Grand Narrative

Empowerment issue oriented

Change-oriented

Interventionist

Queer theory

Race specific

Political

Consequences of actions

Problem-centred

Pluralistic

Real-world practice oriented

Mixed models

Adapted from

Mertens

(2005) and Creswell (2003)

Paradigms: Language commonly associated with major research paradigms

Slide17

Paradigm

Methods (primarily)

Data collection tools (examples)

Positivist/ PostpositivistQuantitative. "Although qualitative methods can be used within this paradigm, quantitative methods tend to be predominant . . ." (Mertens, 2005, p. 12)ExperimentsQuasi-experimentsTestsScalesInterpretivist/ ConstructivistQualitative methods predominate although quantitative methods may also be utilised.

Interviews

Observations

Document reviews

Visual data analysis

Transformative

Qualitative methods with quantitative and mixed methods. Contextual and historical factors described, especially as they relate to oppression (Mertens, 2005, p. 9)

Diverse range of tools - particular need to avoid discrimination. Eg: sexism, racism, and homophobia.

Pragmatic

Qualitative and/or quantitative methods may be employed. Methods are matched to the specific questions and purpose of the research.

May include tools from both positivist and

interpretivist

paradigms.

Eg

Interviews, observations and testing and experiments.

Paradigms, methods and tools

Slide18

Classification of the research purposeExploratory research: Find out what is happening, to clarify your understanding of a problem, 3 ways for conducting; a search of the literature, interview experts in the subject, conducting focus group interviewsDescriptive studies:

Its object is to portray an accurate profile not persons, events or situations.Explanatory (causal) studies: Studies that establish causal relationships between variables

Slide19

Research Strategies ExperimentSurveyCase study

Action researchGrounded theory Ethnography

Archival research

Slide20

Research Strategies: Experiment An experiment will involve: Definition of a theoretical hypothesisSelection of samples from know populations

Random allocation of samplesIntroduction of planned intervention

Measurement on a small number of dependent variables

Control of all other variables

Slide21

Research Strategies: Survey Key Features:Popular in business research

Perceived as authoritativeAllows collection of quantitative dataData can be analysed quantitatively

Samples need to be representative

Gives the researcher independenceStructured observation and interviews can be usedSurveyTo collect a large amount of data from a sizeable population and standardize it to allow easy comparison Types: Questionnaires, Structured Interviews

Slide22

Research Strategies: Case study Key featuresProvides a rich understanding of a real life contextUses and triangulates multiple sources of data

A case study can be categorised in four ways and based on two dimensions: single case v. multiple case (more ability to generalize)

holistic case (choose 1 organization as a whole)

v. embedded case (some departments or activities)Case Studies:The documented history of a particular person, group, organization, or event.

Slide23

Research Strategies: Action research Key featuresResearch IN action - not ON action

focusing on the purposeInvolvement of practitioners in the research

The researcher becomes part of the organisation

Promotes change within the organisationCan have two distinct focus (Schein, 1999) – the aim of the research and the needs of the sponsor

Slide24

Research Strategies;

Grounded theory (Inductive deductive approach)

Key features:

Theory is built through induction and deductionHelps to predict and explain behaviourDevelops theory from data generated by observations Is an interpretative process, not a logical-deductive oneRepresents an inductive investigation in which the researcher poses questions about information provided by respondents or taken from historical records.The researcher asks the questions to him or herself and repeatedly questions the responses to derive deeper explanations.

Slide25

Research Strategies: Ethnography

(Inductive approach)

Key features

Aims to describe and explain the social world inhabited by the researcherTakes place over an extended time periodIs naturalisticInvolves extended participant observation such as studying gorillas in their natural habitatEthnographyRepresents ways of studying cultures through methods that involve becoming highly active within that culture.

Slide26

Research Strategies: Archival researchKey featuresUses administrative records and documents as the principal sources of data Allows research questions focused on the past

Is constrained by the nature of the records and documentsExample: historical research

Slide27

Research ApproachDeductive Deductive approach tests the validity of assumptions (or theories/hypotheses) in handInductiveInductive approach contributes to the emergence of new theories and generalizations

Slide28

Deductive methods

Inductive methods

Principles based on science

Movement is done from theory to dataCasual relationships between variables need to be explainedQuantitative type of data is mainly collectedMeasures of control are applied in order to ensure the validity of dataConcepts are operationalised in order to ensure the clarity of definitionsThe approach is highly structuredResearcher is independent from the research processSamples need to be selected of a sufficient size in order to be able to generalise research conclusions

The meaning of human attachment to events are aimed to be explored

Research context is understood in a deeper manner

Qualitative type of data is collected

More flexible approach to research structure to ensure provisions for changes during the research

Researcher is perceived to be a part of the research process

Research findings do not have to be generalised

 

the major differences between deductive and inductive research approaches

Slide29

Primary Data vs

Secondary Data

Primary data:

directly collected by researcher and his/her team Secondary: data collected by otherscross section vs time series/longitudinalcross section: the study of a phenomenon at a particular time

longitudinal:

It has the capacity to study

change and development

Types of Data

Quantitative

vs

Qualitative

Discrete: Nominal

Continuum; Ordinal, Interval,

ratio

The level of mathematical

Operations

Nominal :

= and



Ordinal :

= ,

, >, <

Interval :

= ,

 , > , <, + , -

Ratio :

= ,

, >, <, + , - ,  , 

Slide30

Time Horizons

Cross-sectional studies

the study of a phenomenon at a particular time. Because of time restrictions

Longitudinal studies it has the capacity to study change and development

Slide31

Comparing Qualitative and Quantitative Research

Slide32

Qualitative dataData that are not characterized by numbers but rather are textual, visual, or oral.Focus is on stories, visual portrayals, meaningful characterizations, interpretations, and other expressive descriptions.Quantitative dataRepresent phenomena by assigning numbers in an ordered and meaningful way.

Contrasting Exploratory and Confirmatory Research

Slide33

Quantitative business research

Descriptive and conclusiveAddresses research objectives through empirical assessments that involve numerical measurement and statistical analysis.

Qualitative

business researchExploratoryUses small versus large samplesAsks a broad range of questions versus structured questionsSubjective interpretation versus statistical analysisQualitative “versus” Quantitative Research

Slide34

Qualitative business researchResearch that addresses business objectives through techniques that allow the researcher to provide elaborate interpretations of phenomena without depending on numerical measurementIts focus is on discovering true inner meanings and new insights. Researcher-dependentResearcher must extract meaning from unstructured responses such as text from a recorded interview or a collage representing the meaning of some experience.

What is Qualitative Research?

Slide35

Qualitative research is useful when:It is difficult to develop specific and actionable decision statements or research objectives.

The research objective is to develop a detailed and in-depth understanding of some phenomena.The research objective is to learn how a phenomenon occurs in its natural setting or to learn how to express some concept in colloquial terms.

The behavior the researcher is studying is particularly context-dependent.

A fresh approach to studying the problem is needed.Uses of Qualitative Research

Slide36

Major Orientations of Qualitative Research

Phenomenology—originating in philosophy and psychologyEthnography—originating in anthropology

Grounded theory—originating in sociology

Case studies—originating in psychology and in business researchQualitative Research Orientations

Slide37

Data must be analyzed to produce information Computer software analysis is normally used for this process

Data should be carefully prepared for analysis Researchers need to know how to select and use different charting and statistical techniques

Quantative

data analysis: Key Points

Slide38

Preparing, inputting and checking data Choosing the most appropriate statistics to describe the data Choosing the most appropriate statistics to examine data relationships and trends Quantative

data analysis: Main Concerns

Slide39

Quantative data analysis: Main Considerations

Type of data (scale of measurement)

Data format for input to analysis software

Impact of data coding on subsequent analyses Case weighting Methods for error checking

Slide40

Data Sources

Secondary data

Documentary, survey, or an amalgam of both

Times series for longitudinal studiesCohort studies (compiling for the same population over time using a series of “snap-shots”)Area-based data setsPrimary dataExperiments and observational studyQuestionnaires/testsInterviewsFocus groups

Slide41

Secondary DataAvailability of secondary data sources

References in publications (books, journal articles)Within

organisations (unpublished sources)

Tertiary literature ( indexes and catalogues in archives or online)

Slide42

What did you find on the frustrating side as you looked for data on the state’s websites?Evaluating secondary data: – Limitations

Slide43

Evaluating secondary data: – LimitationsWhen was it collected? For how long?May be out of date for what you want to analyze.May not have been collected long enough for detecting trends.E.g. Have new anticorruption laws impacted Russia’s government accountability ratings?

Slide44

Evaluating secondary data: – LimitationsIs the data set complete?There may be missing information on some observationsUnless such missing information is caught and corrected for, analysis will be biased.

Slide45

Evaluating Secondary Data – LimitationsAre there confounding problems?Sample selection bias?Source choice bias?In time series, did some observations drop out over time?

Slide46

Evaluating Secondary Data – LimitationsAre the data consistent/reliable?Did variables drop out over time?Did variables change in definition over time?E.g. number of years of education versus highest degree obtained.

Slide47

Evaluating Secondary Data – LimitationsIs the information exactly what you need?In some cases, may have to use “proxy variables” – variables that may approximate something you really wanted to measure. Are they reliable? Is there correlation to what you actually want to measure?

Slide48

Evaluating Secondary Data – AdvantagesNo need to reinvent the wheel.If someone has already found the data, take advantage of it.

Slide49

Evaluating Secondary Data – AdvantagesIt will save you money.Even if you have to pay for access, often it is cheaper in terms of money than collecting your own data.

Slide50

Evaluating Secondary Data – AdvantagesIt will save you time.Primary data collection is very time consuming.

Slide51

Evaluating Secondary Data – AdvantagesIt may be very accurate.When especially a government agency has collected the data, incredible amounts of time and money went into it. It’s probably highly accurate.

Slide52

Evaluating Secondary Data – AdvantagesIt has great exploratory valueExploring research questions and formulating hypothesis to test.

Slide53

Evaluating secondary data: AdvantagesFewer resource requirements

UnobtrusiveLongitudinal studies may be feasible

Provision

of comparative and contextual dataUnforeseen discoveries may occurGenerally permanent and available

Slide54

Evaluating secondary data: DisadvantagesPurpose of data collection may not match the research needs

Access may be difficult or costlyAggregations and definitions may be unsuitable

No

real control over data qualityInitial purpose may affect data presentation

Slide55

Evaluating secondary data: Ensure that data sourcesEnable the research question(s) to be answered

Enable research objectives to be metHave greater benefits than their associated costs

Allow

access for research

Slide56

Suitability of secondary data: Overall suitability: points to considerPrecise suitability, including reliability and validity

- assessment of collection methods- clear explanation of collection techniquesMeasurement

validity

Measurement bias and deliberate distortion

Slide57

Coverage and unmeasured variables - ensure exclusion of unwanted data - ensure sufficient data remain for analysis Costs and benefits

Suitability of secondary data: Overall suitability: points to consider

Slide58

Primary DataPrimary data – data you collectPrimary Data - ExamplesSurveysFocus groupsQuestionnaires Personal interviewsExperiments and observational study

Slide59

Data collection choiceWhat you must ask yourself:Will the data answer my research question?

Slide60

Data collection choiceTo answer that You much first decide what your research question isThen you need to decide what data/variables are needed to scientifically answer the question

Slide61

Data collection choiceIf that data exist in secondary form, then use them to the extent you can, keeping in mind limitations.But if it does not, and you are able to fund primary collection, then it is the method of choice.

Slide62

InterviewsAn interview is a purposeful discussion between two or more people (Kahn and

Cannell, 1957)

Structured interviews: Personal (face to face) and Telephone

Semi-structured interviewsUnstructured interviews (in-depth)Standardized interviewsNon-standardized interviewsRespondent interviewsInformant interviews

Slide63

Standardized vs Non-standardized interviews

Slide64

Structured, Semi-structured, and Unstructured Interviews Related to the Research Type

Slide65

An unstructured, free-flowing interview with a small group (6-10 people) led by a moderator who encourages dialogue among respondents.Advantages:Relatively fastEasy to execute

Allow respondents to piggyback off each other’s ideas

Provide multiple perspectives

Flexibility to allow more detailed descriptionsHigh degree of scrutinyFocus Group Discussion

Slide66

Group Composition6 to 10 peopleRelatively homogeneousSimilar lifestyles and experiencesFocus Group Respondents

Slide67

ModeratorA person who leads a focus group interview and insures that everyone gets a chance to speak and contribute to the discussion.Qualities of a good moderator:Develops rapport with the groupGood listenerTries not to interject his or her own opinions

Controls discussion without being overbearing

The Focus Group Moderator

Slide68

Discussion guideIncludes written introductory comments informing the group about the focus group purpose and rules and then outlines topics or questions to be addressed in the group session.Planning a Focus Group Outline

Slide69

Welcome and introductions should take place first.Begin the interview with a broad icebreaker that does not reveal too many specifics about the interview.Questions become increasingly more specific as the interview proceeds.If there is a very specific objective to be accomplished, that question should probably be saved for last.A debriefing statement should provide respondents with the actual focus group objectives and answering any questions they may have.

Focus Group Discussion Guide

Slide70

Online focus groupA qualitative research effort in which a group of individuals provides unstructured comments by entering their remarks into an electronic Internet display board of some type.Focus blogA type of informal, “continuous” focus group established as an Internet blog for the purpose of collecting qualitative data from participant comments.

Interactive Media and Online Focus Groups

Slide71

Depth interviewA one-on-one interview between a professional researcher and a research respondent conducted about some relevant business or social topic.LadderingA particular approach to probing asking respondents to compare differences between brands at different levels.Produces distinctions at the:attribute level

benefit levelvalue or motivation levelDepth Interviews

Slide72

ConversationsAn informal qualitative data-gathering approach in which the researcher engages a respondent in a discussion of the relevant subject matter.Semi-structured interviewsWritten form and ask respondents for short essay responses to specific open-ended questions.AdvantagesAn ability to address more specific issues

Responses are easier to interpretWithout the presence of an interviewer, semi-structured interviews can be relatively cost effectiveConversations

Slide73

Questionnaire

A set of Questions designed to generate the statistical information from and data necessary for accomplishing a research project's objectives

Definition of Questionnaires: Techniques of data collection in which each person is asked to respond to the same set of questions in a predetermined order (Adapted from

deVaus. 2002)

Slide74

When to use questionnairesFor explanatory or descriptive researchLinked with other methods in a multiple-methods research design

To collect responses from a large sample prior to quantitative analysis

Slide75

Ensures standardization and comparability of the data across interviews – everyone is asked the same questions

Increases speed and accuracy of recording

Facilitates data processing

Allows the researcher to collect the relevant information necessary to address the management decision problemPurposes of the Questionnaire

To obtain information that cannot be easily observed or is not already available in written or electronic

form

Questionnaires enable researchers to measure

concepts/constructs

Slide76

Designing the QuestionnaireDetermine survey objectives: Plan what to measure.Decide on format. E.g. personal interview, telephone, self.

Formulate questions to obtain the needed informationDecide on the wording of questions

Decide on the question sequence and layout of the questionnaire

Using a sample, test the questionnaire for omissions and ambiguityCorrect the problems (pretest again, if necessary)

Slide77

Constructing the questionnaireMain considerationsOrder and flow of questionsQuestionnaire layout

Slide78

Choice of questionnaire: Related factorsCharacteristics of the respondents and access

Respondents answers not being contaminated or distortedSize of sample required for analysisType and number of questions required

Available resources including use of computer software

Slide79

Administering the questionnairePoints to considerInternet and intranet-mediated responsesPostal questionnairesDelivery and CollectionTelephone questionnairesStructured interviews

Slide80

Content - What should be asked? Wording - How should each question be phrased?Sequence

- In what order should the questions be presented?Layout - What layout will best serve the research objectives?

The Major Decisions in Questionnaire Design

The most difficult step is specifying exactly what information is to be collected from each respondent

Slide81

Initial

Considerations – problem, objectives, target population, sampling, etc.

Clarification of Concepts – select variables, constructs, measurement approach, etc.

Developing the QuestionnaireLength and sequence.Types of questions.

Sources of questions.

Wording, coding, layout and instructions.

Pre-testing the Questionnaire.

Questionnaire Administration Planning.

Steps in Questionnaire

Design

Slide82

Steps to design a questionnaire:Write out the primary and secondary aims of your study.Write out concepts/information to be collected that relates to these aims. Review the current literature to identify already validated questionnaires that measure your specific area of interest.

Compose a draft of your questionnaire.Revise the draft.

Assemble the final questionnaire

.

Slide83

Step 1: Define the aims of the studyWrite out the problem and primary and secondary aims using one sentence per aim. Formulate a plan for the statistical analysis of each aim.Make sure to define the target population in your aim(s).

Slide84

Step 2: Define the variables to be collectedWrite a detailed list of the information to be collected and the concepts to be measured in the study. Are you trying to identify:AttitudesNeedsBehavior

DemographicsSome combination of these conceptsTranslate these concepts into variables that can be measured. Define the role of each variable in the statistical analysis:

Predictor

ConfounderOutcome

Slide85

Step 3: Review the literatureReview current literature to identify related surveys and data collection instruments that have measured concepts similar to those related to your study’s aims.Saves development time and allows for comparison with other studies if used appropriately.Proceed with caution if using only a subset of an existing questionnaire as this may change the meaning of the scores. Contact the authors of the questionnaire to determine if a smaller version of the instrument exists that has also been validated.

Slide86

Step 4: Compose a draft [1]:Determine the mode of survey administration: face-to-face interviews, telephone interviews, self-completed questionnaires, computer-assisted approaches.Write more questions than will be included in the final draft.Format the draft as if it were the final version with appropriate white space to get an accurate estimate as to its length – longer questionnaires reduce the response rate.

Place the most important items in the first half of the questionnaire to increase response on the important measures even in partially completed surveys.Make sure questions flow naturally from one to another.

Slide87

Compose a draft [2]:Question: How many cups of coffee or tea do you drink in a day?Principle: Ask for an answer in only one dimension.Solution: Separate the question into two – (1) How many cups of coffee do you drink during a typical day?(2) How many cups of tea do you drink during a typical day?

Slide88

Compose a draft [3]: Question: What brand of computer do you own?(A) IBM PC(B) ApplePrinciple: Avoid hidden assumptions. Make sure to accommodate all possible answers.Solution:(1) Make each response a separate dichotomous item

Do you own an IBM PC? (Circle: Yes or No)Do you own an Apple computer? (Circle: Yes or No)(2) Add necessary response categories and allow for multiple responses.What brand of computer do you own? (Circle all that apply)

Do not own computer

IBM PCAppleOthers

Slide89

Compose a draft [4]: Question: Have you had pain in the last week?[ ] Never [ ] Seldom [ ] Often [ ] Very oftenPrinciple: Make sure question and answer options match.Solution: Reword either question or answer to match.How often have you had pain in the last week?[ ] Never [ ] Seldom [ ] Often [ ] Very Often

Slide90

Compose a draft [5]: Question: Where did you grow up?CountryFarmCityPrinciple: Avoid questions having non-mutually exclusive answers

Solution: Design the question with mutually exclusive options.Where did you grow up?House in the countryFarm in the countryCity

Slide91

Compose a draft [6]: Question: Are you against drug abuse? (Circle: Yes or No)Principle: Write questions that will produce variability in the responses.Solution: Eliminate the question.

Slide92

Compose a draft [7]:Question: Which one of the following do you think increases a person’s chance of having a heart attack the most? (Check one) [ ] Smoking [ ] Being overweight [ ] StressPrinciple: Encourage the respondent to consider each possible response to avoid the uncertainty of whether a missing item may represent either an answer that does not apply or an overlooked item.Solution: Which of the following increases the chance of having a heart attack?Smoking: [ ] Yes [ ] No [ ] Don’t know

Being overweight: [ ] Yes [ ] No [ ] Don’t knowStress: [ ] Yes [ ] No [ ] Don’t know

Slide93

Compose a draft [8]: Question: (1) Do you currently have a life insurance policy? (Circle: Yes or No)If no, go to question 3.(2) How much is your annual life insurance premium?Principle: Avoid branching as much as possible to avoid confusing respondents. Solution: If possible, write as one question.How much did you spend last year for life insurance? (Write 0 if none).

Slide94

Step 5: ReviseShorten the set of questions for the study. If a question does not address one of your aims, discard it.Refine the questions included and their wording by testing them with a variety of respondents.Ensure the flow is natural.Verify that terms and concepts are familiar and easy to understand for your target audience.Keep recall to a minimum and focus on the recent past.

Slide95

Step 6: Assemble the final questionnaire [1]:Decide whether you will format the questionnaire yourself or use computer-based programs for assistance:SurveyMonkey.comAdobe Live Cycle Designer 7.0GCRC assistanceAt the top, clearly state:The purpose of the studyHow the data will be usedInstructions on how to fill out the questionnaire

Your policy on confidentialityInclude identifying data on each page of a multi-page, paper-based questionnaire such as a respondent ID number in case the pages separate.

Slide96

Assemble the final questionnaire [2]:Group questions concerning major subject areas together and introduce them by heading or short descriptive statements.Order questions in order to stimulate recall.Order and format questions to ensure unbiased and balanced results.

Slide97

Assemble the final questionnaire [3]:Include white space to make answers clear and to help increase response rate.Space response scales widely enough so that it is easy to circle or check the correct answer without the mark accidentally including the answer above or below.Open-ended questions: the space for the response should be big enough to allow respondents with large handwriting to write comfortably in the space.Closed-ended questions: line up answers vertically and precede them with boxes or brackets to check, or by numbers to circle, rather than open blanks.

Use larger font size (e.g., 14) and high contrast (black on white).

Slide98

Open-ended Questions : place no constraints on respondents; i.e., they are free to answer in their own words and to give whatever thoughts come to mind.Closed-ended Questions : respondent is given the option of choosing from a number of predetermined answers.

Two Types of Questions:

Open-ended

Closed-endedQUESTIONNAIRE DESIGN

Slide99

Open-ended QuestionsTypically used in exploratory/qualitative studies.Typically used in personal interview surveys involving small samples.

Allows respondent freedom of response.Respondent must be articulate and

willing to spend time giving a full answer.

Data is in narrative form which can be time consuming and difficult to code and analyze.Possible researcher bias in interpretation.Narrative is analyzed using of content analysis. Software is available (e.g., NUD*IST).QUESTIONNAIRE DESIGN

Slide100

Open Ended Questions: key advantages

Wide range of responses and information can be obtained

Answers based on respondent’s not researcher’s frame of reference – consumer’s terms

Lack of influence. Don't channel respondents thinking Can help interpret closed-ended questions - why Particularly useful as introduction to survey or topic When it’s important to measure the salience of an issue When too many possible responses to be listed or unknown

Slide101

Ability and/or willingness of respondent to answer

Interviewer’s ability to record answers quickly or summarize accurately & probe effectively

Interviewer’s attitude influences response

Time consuming (interview sessions, tabulation, classification, assignment, validation) Difficulty in coding Require respondents to be articulate Respondents may miss important points Non-responseOpen-ended questions: Key disadvantages

Slide102

Examples of Open-ended Questions:How do you typically decide which restaurant you will eat at?

Which mutual funds have you been investing in for the past year?How are your investment funds performing

Do you think airport security is better now than it was six months ago?

Slide103

Examples of Open questionsPlease list up to three things you like about your job1…………………………………………

2…………………………………………3…………………………………………

Slide104

Closed-end Questions: Single Answer

Multiple Answer

Rank Order

Numeric Likert-Type Scales Semantic DifferentialQUESTIONNAIRE DESIGN

Slide105

Closed-end QuestionsTypically used in quantitative studies.Assumption is researcher has knowledge to pre-specify response categories.

Data can be pre-coded and therefore in a form amenable for use with statistical packages (e.g., SPSS, SAS) – data capture therefore easier.More difficult to design but

simplifies analysis.

Used in studies involving large samples.Limited range of response options.QUESTIONNAIRE DESIGN

Slide106

Ease of understanding

Requires less effort on part of interviewer and respondent

Ease of tabulation & analysis

Less error prone Less interviewer bias Less time consumingAnswers directly comparable from respondent to respondentClosed-ended questions (Fixed-alternative responses); Advantages

Slide107

Did you check your email this morning? __ Yes __ NoDo you believe Enron senior executives should be put in jail? __ Yes __ No Should the U.K. adopt the Euro or keep the Pound?

__ Adopt the Euro __ Keep the Pound

Which countries in Europe have you traveled to in the last six months?

__ Belgium __ Germany __ France __ Holland __ Italy __ Switzerland __ Spain __ Other (please specify) _____________How often do you eat at Samouel’s Greek Cuisine restaurant? __ Never __ 1 – 4 times per year __ 5 – 8 times per year __ 9 – 12 times per year __ More than 12 times per year

Examples of Closed-end Questions:

Slide108

Examples of question types: List questionsWhat is your religion?Please tick  the appropriate box

Buddhist  None Christian

 Other 

Hindu Jewish Muslim Sikh 

Slide109

Examples of question types: Category questionsHow often do you visit the shopping centre?Interviewer: listen to the respondent’s answer and tick

 as appropriate First visit

 Once a week

 Less than fortnightly to once a month 2 or more times a week Less than once a week to fortnightly Less often

Slide110

Examples of question types: Ranking questionsPlease number each of the factors listed below in order of importance to you in choosing a new car. Number the most important 1, the next 2 and so on. If a factor has no importance at all, please leave blank.Factor Importance

Carbon dioxide emissions [ ] Boot size [ ]

Depreciation

[ ] Price [ ]

Slide111

Examples of question types: Rating questionsFor the following statement please tick the box that matches your view most closelyAgree Tend to agree Tend to disagree Disagree

I feel employees’    

views have

influenced the decisions takenby management

Slide112

Examples of question types: Quantity questions What is your year of birth?(For example, for 1988 write:)

1

1

9

9

8

8

Slide113

Dichotomous Questions

Should the Alberta Government give consumers an energy rebate?

Agree

DisagreeAdvantages

Easy to administer and tabulate

Disadvantages

Prone to large amounts of error since polarized responses prevent gaining information on the range of variation

Fail to communicate any intensity of feeling

Slide114

Multiple Choice Questions

Are all possible alternatives included?

Too many alternatives

Position Bias

Scaled Response Questions

Closed ended questions where the response choices are designed to capture an intensity of feeling (Likert, Staple, Semantic differential)

Easy

to code and more powerful statistical tools

Main problem: Respondent misunderstanding

Slide115

Preparing Good Questions:

Use Simple Words.

Be brief.

Avoid Ambiguity. Avoid Leading Questions. Avoid Double-Barreled Questions.

Check Questionnaire Layout.

Prepare Clear Instructions.

Watch Question Sequence.

QUESTIONNAIRE DESIGN

Slide116

QUESTIONNAIRE DESIGN

 

 

Recently a survey was conducted by the United Nations using a sample from several different countries. The question asked was:

 

" Would you please give your opinion about the food shortage in the rest of the world?"

The survey was a huge failure. Why?

In

Africa

they did not know what 'food' meant.

In

Western Europe

, they did not know what 'shortage' meant.

In

Eastern Europe

they did not know what 'opinion' meant.

In

South America

they did not know what 'please' meant.

And in the

U.S.

, they did not know what 'the rest of the

world' meant.

Slide117

Avoid Position Bias:

Position Bias

:

“How important are flexible hours in evaluating job alternatives?” “What factors are important in evaluating job alternatives?”

No Position Bias

:

“What factors are important in evaluating

job alternatives?”

“How important are flexible hours in evaluating

job alternatives?”

QUESTIONNAIRE DESIGN

Slide118

QUESTIONNAIRE DESIGN

 

 

. . . are used to direct respondents to answer the right

questions as well as questions in the proper sequence.

“Have you seen or heard any advertisements for wireless

telephone service in the past 30 days?”

“If ‘No’, go to question #10.

“If ‘Yes’ , were the advertisements on radio or TV or both?”

“If the advertisements were on TV or on both radio and

TV, then go to question #6?

“If the advertisements were on radio, then go to

question #8.”

Following questions #6 and #8 the next question would be:

“Were any of the advertisements for ‘Sprint PCS’?”

Branching Questions:

Slide119

QUESTIONNAIRE DESIGN

 

 

Introducing and explaining how to answer a series of

questions on a particular topic.

Transition statements from one section (topic) of the

questionnaire to another.

Which question to go to next (branching or skipping).

How many answers are acceptable, e.g., “Check only

one response”

or

“Check as many as apply.”

Whether respondents are supposed to answer the

question by themselves, or can consult another

person or reference materials.

What to do when the questionnaire is completed, e.g.,

“When finished, place this in the postage paid

envelope and mail it.”

Issues – Self-Completion Instructions:

Slide120

QUESTIONNAIRE DESIGN

 

How to increase respondent participation.

How to screen out respondents that are not wanted and

still keep them happy.

What to say when respondents ask how to answer a

particular question.

When concepts may not be easily understood, how to

define them.

When answer alternatives are to be read to respondents

(aided response) or not to be read (unaided response).

How to follow branching or skip patterns.

When and how to probe.

How to end the interview.

Issues – Interviewer-Assisted Instructions:

Slide121

Identify response bias for below questions:

“Do you advocate a lower speed limit to save human lives?”

“When you visited the museum, how many times did you read the plaques that explain what the exhibit contained?”

“About what time do you ordinarily eat dinner?”“How important is it for stores to carry a large variety of different brands of this product?”“Would you favor increasing taxes to cope with the current fiscal crisis?”“Don’t you see some danger in the new policy?”“What small appliance, such as countertop appliances, have you purchased in the past month?”“When you buy ‘fast food,’ what percentage of the time do you order each of the following types of food?”“Do you like orange juice?”

QUESTIONNAIRE DESIGN

Slide122

Comments on Questions:

A loaded question because everyone wants to save lives. Also, it presumes that lower speed limits saves lives.

Too specific because respondents likely cannot remember the exact number of times.

Ambiguous because don’t know if dinner is lunch or evening.Not specific enough about types of stores.Overemphasis because refers to crisis.Leading question because uses “danger” in sentence.Answers likely to relate only to countertop appliances and not all small appliances.Over generalization because does not specify time period.Ambiguous because may like orange juice for themselves, or for their kids, but really do not know.

QUESTIONNAIRE DESIGN

Slide123

Objective: to identify possible shortcomings of questionnaire.Approaches – informal or formal.Can assess:

No

hard and fast rules.

ability to perform meaningful analysestime to complete the questionnaire cost of data collection which questions are relevant whether key questions have been overlooked

sources of bias

clarity of instructions

cover letter

clarity of questions

adequacy of codes and categories for pre-coded questions

quality of responses

likely response rate

Pre-testing Questionnaires:

QUESTIONNAIRE DESIGN

Slide124

Types of Scales:Metric

(interval & ratio)Likert

-type

Summated-Ratings (Likert)NumericalSemantic Differential Graphic-RatingsNonmetric (nominal & ordinal)CategoricalConstant Sum MethodPaired ComparisonsRank OrderSorting

MEASUREMENT SCALES

Slide125

Examples of Likert-Type Scales: “When I hear about a new restaurant , I eat there to see what it is like.”

Strongly Agree Neither Agree Disagree Strongly

Agree Somewhat or Disagree Somewhat Disagree

1 2 3 4 5MEASUREMENT SCALES – Metric

“When I hear about a new restaurant , I eat there to see what it is like.”

Strongly Strongly Agree Disagree

1 2 3 4 5

Slide126

Summated Ratings Scales: A scaling technique in which respondents are asked to indicate their degree of agreement or disagreement with each of a number of statements. A subject’s attitude score (summated rating) is the total obtained by summing over the items in the scale and dividing by the number of items to get the average.

Example

: “My sales representative is . . . . “ SD D N A SA Courteous ___ ___ ___ ___ ___ Friendly ___ ___ ___ ___ ___ Helpful ___ ___ ___ ___ ___ Knowledgeable ___ ___ ___ ___ ___

MEASUREMENT SCALES – Metric

Slide127

Alternative Approach to Summated Ratings scales:

“When I hear about a new restaurant , I eat there to see what it is like.”

Strongly Agree Neither Agree Disagree Strongly

Agree Somewhat or Disagree Somewhat Disagree 1 2 3 4 5“I always eat at new restaurants when someone tells me they are good.”Strongly Agree Neither Agree Disagree Strongly Agree Somewhat or Disagree Somewhat Disagree 1 2 3 4 5

MEASUREMENT SCALES – Metric

This approach includes a separate labeled Likert scale with each item (statement). The summated rating is a total of the responses for all the items divided by the number of items.

Slide128

Numerical Scales:Example:“Using a 10-point scale, where ‘1’ is ‘not at all important’ and ’10’ is ‘very important,’ how important is ______ in your decision to do business with a particular vendor.”

Note: you fill in the blank with an attribute, such as reliable delivery, product quality, complaint resolution, and so forth.

MEASUREMENT SCALES – Metric

Slide129

Semantic Differential Scales: A scaling technique in which respondents are asked to check which space between a set of bipolar adjectives or phrases best describes their feelings toward the stimulus object.

Example

:

“My sales representative is . . . . “ Courteous ___ ___ ___ ___ ___ Discourteous Friendly ___ ___ ___ ___ ___ Unfriendly Helpful ___ ___ ___ ___ ___ Unhelpful Honest ___ ___ ___ ___ ___ DishonestMEASUREMENT SCALES – Metric

Slide130

Graphic-Ratings Scales:A scaling technique in which respondents are asked to indicate theirratings of an attribute by placing a check at the appropriate point

on a line that runs from one extreme of the attribute to the other.

“Please evaluate each attribute in terms of how important the attribute is to you personally (your company) by placing an “X” at the position on the horizontal line that most reflects your feelings.” Not Importan Very ImportantCourteousness _____________________________________Friendliness _____________________________________Helpfulness _____________________________________Knowledgeable _____________________________________

MEASUREMENT SCALES – Metric

Slide131

Categorical scale:Categorical scales are nominally measured opinion scales that have two or more response categories.

“How satisfied are you with your current job?”

[ ] Very Satisfied [ ] Somewhat Satisfied [ ] Neither Satisfied nor Dissatisfied [ ] Somewhat Dissatisfied [ ] Very DissatisfiedNote: Some researchers consider this a metric scale when coded 1 – 5 .

MEASUREMENT SCALES – Nonmetric

Slide132

Constant-Sum Method:A scaling technique in which respondents are asked to divide

some given sum among two or more attributes on the basis oftheir importance to them.

“Please divide 100 points among the following attributes in terms of the relative importance of each attribute to you.” Courteous Service ____ Friendly Service ____ Helpful Service ____ Knowledgeable Service ____

Total

100

MEASUREMENT SCALES – Nonmetric

Slide133

Paired Comparison Method: A scaling technique in which respondents are given pairs of stimulus objects and asked which object in a pair they prefer most.

“Please circle the attribute describing a sales representative which you consider most desirable.”

Courteous versus Knowledgeable Friendly versus Helpful Helpful versus Courteous

MEASUREMENT SCALES – Nonmetric

Slide134

Sorting:A scaling technique in which respondents areasked to indicate their beliefs or opinions by

arranging objects (items) on the basis of

perceived importance, similarity, preference

or some other attribute.MEASUREMENT SCALES – Nonmetric

Slide135

Rank Order Method:A scaling technique in which respondents are resented with several stimulus objects simultaneously and asked

to order or rank them with respect to a specificcharacteristic.

“Please rank the following attributes on how important each is to you in relation to a sales representative. Place a “1” beside the attribute which is most important, a “2” next to the attribute that is second in importance, and so on.”

Courteous Service ___ Friendly Service ___ Helpful Service ___ Knowledgeable Service ___

MEASUREMENT SCALES – Nonmetric

Slide136

Practical Decisions When Developing Scales:

Number of items (indicators) to measure a concept?

Number of scale categories?

Odd or even number of categories? (Include neutral point ?)

Balanced or unbalanced scales?

Forced or non-forced choice?

(

Include Don’t Know ?)

Category labels for scales?

Scale reliability and validity?

Scale Development

Slide137

Balanced vs. Unbalanced Scales?

“

To what extent do you consider TV shows with sex and violence to be acceptable for teenagers to view?”

Balanced:   __ Very Acceptable __ Somewhat Acceptable

__ Neither Acceptable or Unacceptable

__ Somewhat Unacceptable

__ Very Unacceptable

Unbalanced:

__ Very Acceptable

__ Somewhat Acceptable

__ Unacceptable

Scale Development

Slide138

Forced or Non-Forced?

Very Very

Unlikely Likely

1 2 3 4 5 6 __ No Opinion

Scale Development

“How likely are you to purchase a laptop PC in the next six months?”

Slide139

Category Labels for Scales?

Verbal Label:

“How important is the size of the hard drive in selecting a laptop PC to purchase?” Very Somewhat Neither Important Somewhat Very Unimportant Unimportant

or Unimportant Important

Important

1 2 3 4 5

Numerical Label:

“

How likely are you to purchase a laptop PC in the next six months?”

Very Very

Unlikely Likely

1 2 3 4 5

Unlabeled:

“How important is the weight of the laptop PC in deciding which brand

to purchase?”

Very Very

Unimportant Important

___ ___ ___ ___ ___

Scale Development

Slide140

Choosing a Measurement Scale: Capabilities of Respondents.

Context of Scale Application.

Data Analysis Approach.

Validity and Reliability.MEASUREMENT SCALES

Slide141

Assessing Measurement Scales: Validity

Reliability

MEASUREMENT SCALES

Measurement Error : occurs when the values obtained in a survey (observed values) are not the same as the true values (population values)

Slide142

Error Due to Bias and ChanceBias - A systematic tendency to misrepresent the population.The object of any experimental design is to eliminate bias and reduce chance error as much as possible.

Slide143

Observation as A Data Collection Method‘Observation involves the systematic observation , recording, description analysis and interpretation of people’s behaviour’

Slide144

Observation

Considerations:

Methods – human/mechanical/electronicUseful where respondent cannot or will not articulate the answer.Cannot be used to measure thoughts, feelings

,

attitudes

, opinions, etc.

Slide145

Types of observationThe two main typesParticipant observation – emphasises the discovery of meaning attached to actions (qualitative) Structured observation

– is concerned with frequency of actions (quantitative)

Slide146

Participant ObservationDefinition: ‘Where the researcher attempts to participate fully in the lives and actions of subjects, enabling them to not merely observe what is happening but also feeling it

’ , Adapted from Gill and Johnson (2002)

Slide147

Participant observation: Points to considerUsed

both as the principle research method and in combination with other methodsResearchers become immersed in the research settingResearchers try to understand the process by which individual identity is constructed and reconstructed (symbolic interactionism)

Slide148

Researcher roles in participant observationTypology of participant observation researcher rolesSource: Gill and Johnson (2002)

Slide149

Choice of participant observer roleDetermining factorsPurpose of the research and time availableDegree of suitability felt by the researcherOrganisational

access Ethical considerations

Slide150

Data collection and analysisTypes of data generated by participant observationPrimary observationsSecondary

observationsExperiential data

Delbridge

and Kirkpatrick (1994)

Slide151

Data collection and analysisPoints to considerData may be classed as ‘descriptive observation’ and ‘narrative

account’ (Robson, 2002)

Data

recording method(s) will depend on the roleData collection and analysis may be part of the same processUse of analytic induction leads to redefinition of the original hypothesis

Slide152

Data collection and analysisPoints to considerThreats to validityThe perspective of the subject - not the researcherAdvantages

and disadvantages of participant observation are summarised in Table 9.1Saunders et al. (2009)

Slide153

Structured ObservationPoints to considerStructured observation is systematic and aims to establish straightforward facts

Structured observation was an important part of Mintzberg’s

(1973) study of managerial work

Proliferation of the Internet potentially widens the scope of participant observationSaunders et al. (2009)

Slide154

Structured observationData collection and analysisChoosing an ‘off the shelf’ coding scheduleDesigning your own coding schedule

Combining both types of scheduleUse of simple (manual) or complex (computer) methods of analysis

Slide155

Structured observationThreats to validity and reliabilitySubject errorTime errorObserver

effects and strategies to overcome this –habituation and minimal interaction Robson

(2002)

Slide156

Slide157

Slide158