Adapted from J Scott Armstrong from lectures related to Persuasive Advertising File TestingAds R13 In Lectures for AdPrinTesting Ads Updated January 2014 1 The most important word in the vocabulary of advertising is ID: 533377
Download Presentation The PPT/PDF document "Testing Ads" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Testing Ads
Adapted from J. Scott Armstrong fromlectures related to Persuasive Advertising.File: TestingAds-R13In Lectures for AdPrin/Testing AdsUpdated January 2014
1Slide2
The most important word in the vocabulary of advertising is
testTest your premise Test your headlines and illustrationsTest your media Test your level of expenditureNever stop testing and your advertising will never stop improving.David OgilvySlide3
How would you test this headline for Tylenol?
“When not playing is not an option.”3Slide4
Testing the effectiveness of advertisements
Assume that you were given 10 pairs of print ads and asked to identify which ad in each pair would be more persuasive. Each ad advertises a product with the same brand. In addition, you would like to suggest how to improve each of the ads.You have a budget of $20,000. Write different approaches. Spend at least five minutes if you already know how to solve this problem, and ten minutes if you do not know. If working in a group, exchange your solutions with a partner.When you finish, go to the next slide.4Slide5
Five approaches to testing advertisements:
Listed by EffectivenessAll of these methods involve comparisons of responses for two or more alternative advertisements:Unaided ratings by novices (used for Super Bowl).Unaided ratings by experts (used for Super Bowl).Copy testing: Test which ads gain the best response
.
Persuasion Principles Audit
: Rate how well ad applies
evidence-based persuasion principles
. This is analogous to how we evaluate medical doctors and engineers.
Field experiment
: Test direct-response ads to see which are most effective (e.g., as done by Bose radio ads).
Reference:
Armstrong et al (2013)
5Slide6
Benefits of unaided judgment by non-experts*
Very low costVery fastCombining helps to improve validity6Slide7
Unaided judgments by experts
Experts are typically poor at forecasting effects of different approaches in complex uncertain situations. Tetlock Seer-suckerTheoryExperts in consumer behavior were poorer than marketing practitioners and high-school students in predicting consumer behavior (Armstrong 1991).7Slide8
Persuasion Principles Audit
Freeware and training module available at AdPrin.comRequires some knowledge of the principles(See AdPrin.com & Persuasive Advertising)Requires good cognitive skills, logic, and patienceCreativity helps
Practice helps
People differ in their ability to do this task. I suspect that economists, lawyers, and engineers do well.
The audit leads to a Persuasion Principles Index (PPI)
8Slide9
AdPrin Audit procedure
Complete 90-minute AdPrin.com training modulePractice on a few adsRate print ads in about 30 minutes, motion ads in 40.Use five independent codersWhen ratings differ “substantially” for “relevant principles” seek consensusfor details see
Predictive Validity of Evidence-based Advertising
)
9Slide10
Benefits of the AdPrin Audit
Relatively low costRelatively fastLeads to evidence-based improvements10Slide11
Copy testing: Selecting a representative sample
Who to test? • A convenience sample• Mall intercepts or student subjects, or Mechanical Turk at Amazon. • Screening questions for similarity to the target market
2. How large a sample?
Start small, increase as the client desires or as you think proper.
3. What approach?
Experimental design, with r
andom assignment of subjects to two or more treatments.
11Slide12
Copy testing for advertisements
Compare responses to ads.Ads do not have to be in finished form. Rough copy generally produces realistic results on the relative magnitude of effects.High face validity convincing to clients.Are ads appropriate for all groups?Consider testing among members of interest groups.
T
o maintain security, ask subjects to role-play such interest groups.
12Slide13
Testing procedure
Prepare Test PackageDegree of finish of the adsContext (e.g., in a magazine)Number and length of exposuresDesign of the test (e.g., pre- and post- tests; within subjects v. across subjects).Use multiple measures of effectSlide14
Testing responses
Describe the subjects’ role (e.g., “Assume you are planning to buy a car”)Focus on behavioral change (e.g., purchase intentions, not “liking”
)
Use a scale
for
responses
when possible
(
e.g.
, use
“
0 to 100%
”
for intentions
)
Use alternative ways of asking the
questions
Pretest the
questions to ensure understanding
Test
elements of the ads
(
e.g., picture
alone
versus headline
alone
)
14Slide15
15
Test the headlineSlide16
16
Test the illustration: What is the product?Slide17
17
Complete adSlide18
Administering copy tests
Written testing instruments and instructions.Self-administration of tests by individuals.Those administering the instrument should be unaware of the hypotheses if they interact with the subjects. Prepare written instructions for administrators.
Take notes about
departures from the script.
18Slide19
What not to do for copy testing
Do not ask customers how they make decisions when purchasing products. They do not know even when making important decisions. (See meta-analysis by Nisbett & Wilson 1977). Even if they did know, they would not remember (Kaasa, Morris & Loftus 2011).Consumers are not experts at advertising, so do not ask them how an ad might be improved or if they liked an ad.
Do not use
focus groups
.
19Slide20
Review: Benefits of copy
testingUseful for comparing responses to ads.The ads do not have to be in finished form. Rough copy generally produces realistic results on the relative magnitude of the ads’ effects.Copy testing has high face validity and
is thus
convincing to clients.
Testing can also assess whether the ads are appropriate to all interest groups
.
20Slide21
Experimental Designs (Campbell and Stanley)
“BEFORE”MEASURESTREATMENT“AFTER” MEASURES“AFTER ONLY”“BEFORE-AFTER”
“BEFORE-AFTER WITH CONTROL”
EXP.
CONTROL
“FOUR GROUP-
SIX
STUDY
EXP I
EXP II
CONTROL I
CONTROL IISlide22
After Only Design
Use Time seriesPrepare (obtain) predictions with ad campaign Compare results with prediction
P
A
Provide Contingency PlansSlide23
Laboratory or field experiments?
Laboratory experiments are typically less expensive and offer more control.Field experiments are more realistic.Findings from each method are comparable.See Locke’s Generalizing from Laboratory to Field Settings23Slide24
What constitutes “evidence”
Reliability “Repeat the test on a similar population to determine if results are similar.” • Replication: Other researchers get same result by same procedure. In Of 20 replications marketing journals, 12 gave substantially different results and only 3 were fully in agreement. Hubbard & Armstrong 1994 • Inter-rater (different raters agree) • Across time (same rater)Validity “Test measures what it purports to measure.” • Face validity (experts agree); Very weak test. Try asking them to predict the outcomes before seeing outcomes. Then pay attention to the differences.
•
Construct validity
(different approaches) For example, Apple’s 1984 ad, was not liked by the experts and it did poorly on copy testing, but it did well on implementing principles.
•
Predictive validity.
Are the predictions testable? (For example, is a cold winter evidence for or against global warming? What will the temperature be in one hundred years?)Slide25
What would you do in this asbestos case?
492 X-rays submitted as court evidence in asbestos lawsuits. X-ray readers found lung damage in 96% of the cases. You have been hired as an expert witness for all cases. What would you do? Spend five minutes to answer this.If working in a group, share your ideas with a learning partner.When finished, go to the next slide.25Slide26
Resolving the asbestos case
Replicate: Ask independent experts to read the x-rays.Result? Only 4.5% of the X-rays were read as showing damage. (See Gitlin study)The plaintiffs will claim that you are biased. How would you protect against that? Click after you have written your answer.Take steps to ensure that the independent experts do not know why they are being asked to do the readings.26Slide27
Reporting on your research
Bring in research as needed to support recommendations.Use statistics to communicate information (See The Visual Display of Quantitative Information by E. Tufte).Do not use statistical significance. Use prediction intervals instead.
Report on shortcomings of your research.
Provide full disclosure on data and methods.
See checklist for
oral
and
written
reports.
27Slide28
Applications: Tylenol ad
Apply material from this lecture to test this Tylenol headline.“When not playing is not an option.”If working in a group setting, exchange your applications with a learning partner.Then click for some answersSlide29
Tylenol headline
It did poorly on copy tests.Also does poorly on principles. For example: 1. No brand name 2. Two negative words 3. Contains a period 4. Social responsibility and legal issues** For example, they might get sued if an unfortunate event occurs.29Slide30
Advice on learning
One study found that fewer than 10% of students were successful in applying new knowledge.This went to 20% if they actively applied what they were taught during a class session.It went to 90% when they worked with a learning partner and coached each other.30Slide31
Application session
Apply the principles from this session to an ad for your organization, to a small project that you are doing, or to an exercise in this course.31