Testing Ads - PowerPoint Presentation

Slide1

Testing Ads

Adapted from J. Scott Armstrong fromlectures related to Persuasive Advertising.File: TestingAds-R13In Lectures for AdPrin/Testing AdsUpdated January 2014

1Slide2

The most important word in the vocabulary of advertising is

testTest your premise Test your headlines and illustrationsTest your media Test your level of expenditureNever stop testing and your advertising will never stop improving.David OgilvySlide3

How would you test this headline for Tylenol?

“When not playing is not an option.”3Slide4

Testing the effectiveness of advertisements

Assume that you were given 10 pairs of print ads and asked to identify which ad in each pair would be more persuasive. Each ad advertises a product with the same brand. In addition, you would like to suggest how to improve each of the ads.You have a budget of $20,000. Write different approaches. Spend at least five minutes if you already know how to solve this problem, and ten minutes if you do not know. If working in a group, exchange your solutions with a partner.When you finish, go to the next slide.4Slide5

Five approaches to testing advertisements:

Listed by EffectivenessAll of these methods involve comparisons of responses for two or more alternative advertisements:Unaided ratings by novices (used for Super Bowl).Unaided ratings by experts (used for Super Bowl).Copy testing: Test which ads gain the best response

.

Persuasion Principles Audit

: Rate how well ad applies

evidence-based persuasion principles

. This is analogous to how we evaluate medical doctors and engineers.

Field experiment

: Test direct-response ads to see which are most effective (e.g., as done by Bose radio ads).

Reference:

Armstrong et al (2013)

5Slide6

Benefits of unaided judgment by non-experts*

Very low costVery fastCombining helps to improve validity6Slide7

Unaided judgments by experts

Experts are typically poor at forecasting effects of different approaches in complex uncertain situations. Tetlock Seer-suckerTheoryExperts in consumer behavior were poorer than marketing practitioners and high-school students in predicting consumer behavior (Armstrong 1991).7Slide8

Persuasion Principles Audit

Freeware and training module available at AdPrin.comRequires some knowledge of the principles(See AdPrin.com & Persuasive Advertising)Requires good cognitive skills, logic, and patienceCreativity helps

Practice helps

People differ in their ability to do this task. I suspect that economists, lawyers, and engineers do well.

The audit leads to a Persuasion Principles Index (PPI)

8Slide9

AdPrin Audit procedure

Complete 90-minute AdPrin.com training modulePractice on a few adsRate print ads in about 30 minutes, motion ads in 40.Use five independent codersWhen ratings differ “substantially” for “relevant principles” seek consensusfor details see

Predictive Validity of Evidence-based Advertising

)

9Slide10

Benefits of the AdPrin Audit

Relatively low costRelatively fastLeads to evidence-based improvements10Slide11

Copy testing: Selecting a representative sample

Who to test? • A convenience sample• Mall intercepts or student subjects, or Mechanical Turk at Amazon. • Screening questions for similarity to the target market

2. How large a sample?

Start small, increase as the client desires or as you think proper.

3. What approach?

Experimental design, with r

andom assignment of subjects to two or more treatments.

11Slide12

Copy testing for advertisements

Compare responses to ads.Ads do not have to be in finished form. Rough copy generally produces realistic results on the relative magnitude of effects.High face validity  convincing to clients.Are ads appropriate for all groups?Consider testing among members of interest groups.

T

o maintain security, ask subjects to role-play such interest groups.

12Slide13

Testing procedure

Prepare Test PackageDegree of finish of the adsContext (e.g., in a magazine)Number and length of exposuresDesign of the test (e.g., pre- and post- tests; within subjects v. across subjects).Use multiple measures of effectSlide14

Testing responses

Describe the subjects’ role (e.g., “Assume you are planning to buy a car”)Focus on behavioral change (e.g., purchase intentions, not “liking”

)

Use a scale

for

responses

when possible

(

e.g.

, use

“

0 to 100%

”

for intentions

)

Use alternative ways of asking the

questions

Pretest the

questions to ensure understanding

Test

elements of the ads

(

e.g., picture

alone

versus headline

alone

)

14Slide15

15

Test the headlineSlide16

16

Test the illustration: What is the product?Slide17

17

Complete adSlide18

Administering copy tests

Written testing instruments and instructions.Self-administration of tests by individuals.Those administering the instrument should be unaware of the hypotheses if they interact with the subjects. Prepare written instructions for administrators.

Take notes about

departures from the script.

18Slide19

What not to do for copy testing

Do not ask customers how they make decisions when purchasing products. They do not know even when making important decisions. (See meta-analysis by Nisbett & Wilson 1977). Even if they did know, they would not remember (Kaasa, Morris & Loftus 2011).Consumers are not experts at advertising, so do not ask them how an ad might be improved or if they liked an ad.

Do not use

focus groups

.

19Slide20

Review: Benefits of copy

testingUseful for comparing responses to ads.The ads do not have to be in finished form. Rough copy generally produces realistic results on the relative magnitude of the ads’ effects.Copy testing has high face validity and

is thus

convincing to clients.

Testing can also assess whether the ads are appropriate to all interest groups

.

20Slide21

Experimental Designs (Campbell and Stanley)

“BEFORE”MEASURESTREATMENT“AFTER” MEASURES“AFTER ONLY”“BEFORE-AFTER”

“BEFORE-AFTER WITH CONTROL”

EXP.

CONTROL

“FOUR GROUP-

SIX

STUDY

EXP I

EXP II

CONTROL I

CONTROL IISlide22

After Only Design

Use Time seriesPrepare (obtain) predictions with ad campaign Compare results with prediction

P

A

Provide Contingency PlansSlide23

Laboratory or field experiments?

Laboratory experiments are typically less expensive and offer more control.Field experiments are more realistic.Findings from each method are comparable.See Locke’s Generalizing from Laboratory to Field Settings23Slide24

What constitutes “evidence”

Reliability “Repeat the test on a similar population to determine if results are similar.” • Replication: Other researchers get same result by same procedure. In Of 20 replications marketing journals, 12 gave substantially different results and only 3 were fully in agreement. Hubbard & Armstrong 1994 • Inter-rater (different raters agree) • Across time (same rater)Validity “Test measures what it purports to measure.” • Face validity (experts agree); Very weak test. Try asking them to predict the outcomes before seeing outcomes. Then pay attention to the differences.

•

Construct validity

(different approaches) For example, Apple’s 1984 ad, was not liked by the experts and it did poorly on copy testing, but it did well on implementing principles.

•

Predictive validity.

Are the predictions testable? (For example, is a cold winter evidence for or against global warming? What will the temperature be in one hundred years?)Slide25

What would you do in this asbestos case?

492 X-rays submitted as court evidence in asbestos lawsuits. X-ray readers found lung damage in 96% of the cases. You have been hired as an expert witness for all cases. What would you do? Spend five minutes to answer this.If working in a group, share your ideas with a learning partner.When finished, go to the next slide.25Slide26

Resolving the asbestos case

Replicate: Ask independent experts to read the x-rays.Result? Only 4.5% of the X-rays were read as showing damage. (See Gitlin study)The plaintiffs will claim that you are biased. How would you protect against that? Click after you have written your answer.Take steps to ensure that the independent experts do not know why they are being asked to do the readings.26Slide27

Reporting on your research

Bring in research as needed to support recommendations.Use statistics to communicate information (See The Visual Display of Quantitative Information by E. Tufte).Do not use statistical significance. Use prediction intervals instead.

Report on shortcomings of your research.

Provide full disclosure on data and methods.

See checklist for

oral

and

written

reports.

27Slide28

Applications: Tylenol ad

Apply material from this lecture to test this Tylenol headline.“When not playing is not an option.”If working in a group setting, exchange your applications with a learning partner.Then click for some answersSlide29

Tylenol headline

It did poorly on copy tests.Also does poorly on principles. For example: 1. No brand name 2. Two negative words 3. Contains a period 4. Social responsibility and legal issues** For example, they might get sued if an unfortunate event occurs.29Slide30

Advice on learning

One study found that fewer than 10% of students were successful in applying new knowledge.This went to 20% if they actively applied what they were taught during a class session.It went to 90% when they worked with a learning partner and coached each other.30Slide31

Application session

Apply the principles from this session to an ad for your organization, to a small project that you are doing, or to an exercise in this course.31