UW CSE454 todo Add vickrey auction Landing page analysis Context discussion very weak Slides 2630 weak Thanks To Mike Mathieu mikefrontseatorg Thanks To Dr Andrei Broder ID: 781799
Download The PPT/PDF document "Computational Advertising" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Computational Advertising
UW CSE454
Slide2todo
Add
vickrey
auction
Landing page analysis
Context discussion very weak
Slides 26-30 weak
Slide3Thanks To:
Mike
Mathieu
mike@frontseat.org
Slide4Thanks To:
Dr. Andrei
Broder
Dr
. Evgeniy
Gabrilovich Dr
.
Vanja
Josifovski
Slide52012 Global Ad Spend
$530 Billion
“Half the money
I spend on advertising
is wasted;
the trouble is I don't know which half.
”
-- John Wanamaker (attributed)
[1838-1922]
Slide66
US online advertising spending
(source: eMarketer.com, November 2010)
Year
Online
Online %
of total media
2009
$22.7B
13.9%
2010
$25.8B
15.3%
2011
$28.5B
16.7%
2012
$32.6B
18.3%
2013
$36.0B
19.8%
2014
$40.5B
21.5%
Slide77
measurability and reach
No more coupon codes
Flexible ad targeting +
conversion tracking
Experimentation rules !
What changed in 100 years?
Slide88
What is “Computational Advertising”?
A new scientific sub-discipline that provides
the foundation
for building online ad
retrieval
platforms
Find the optimal ad for a given user
in a
specific context
Information retrieval
Statistical modeling & machine learning
Large-scale text analysis
Microeconomics
Computational Advertising
Slide9Ad
Industry Structure
Advertiser
Audience
Media
$
$$$
$$$$$
$$
Slide10The Great Divide
Brand
Direct Response
Emotions
Indirect benefits
Banners, TV, stadiums
Transactions
Gross profits
Search,
coupons, 1-800, radio, mail
Slide11Slide12ü
ü
ü
ü
ü
ü
ü
ü
ü
ü
ü
ü
ü
ü
ü
ü
Slide1313
Anatomy of an ad
http://research.yahoo.com/sigir10_compadv
Display URL
Landing URL
Bid phrases:
{SIGIR 2010,
computational advertising, Evgeniy Gabrilovich, ...}
Bid:
$0.10
Landing page
Creative
Title
Tutorial at
SIGIR 2010
Information Retrieval Challenges in
Computational Advertising
research.yahoo.com/tutorials/
sigir
Slide1414
So when do advertising dollars actually change hands
?
CPM
= cost per thousand
impressions
Typically used for
graphical/banner
ads (brand advertising)
CPC
= cost per click
Typically used for textual
ads
CPT/CPA
= cost per transaction/actionAlso known as referral fees or affiliate fees
Risk
To the advertiser
Shared
To the search engine
Slide15Online Advertising Risks
Balance
of Risk
Publisher
Advertiser
Cost Per Impression (CPM)
Cost Per Click (CPC)
Cost Per
Action (CPA)
Revenue Share
Subscription / Sponsorship
Slide16Ad
Industry Structure
Advertiser
Audience
Media
$
$$$
$$$$$
$$
Slide17Conversion Funnel
Ad Impressions
Clicks
Conversions
Revenue
Slide18Monetizing Traffic
Search-Paid
Search-Free
Affiliates
Display Ads
Syndication
Email
Mobile
Offline
LANDING PAGES
$ per
Transaction
Conversion Rate
TRAFFIC
CTR
CR x
CPA
= RPV
Gross Margin
Slide19Share of Voice Costs $$$
0%
Cost Per
Action
Reach
100%
Slide20Conversion Potential vs. Price
Slide21Real World Example
Slide22Real World Example
Impressions
4.4M
Clicks
2078
CTR=0.0469%
CPC=$0.65
eCPM
=$0.31
CPRegClick
=$19.69
CPReg
=$46.76
69
RegClick
Registrations
29
RefSrc
on URL
Drop cookie
Pass
RefSrc
upon conversion
Match with ad spend
Calculate CPA
Slide23Bid Management
Term
Clicks
CPC
Pos
CR
Leads
CPA
AvgPrice
Revenue
Spend
GM
Nursing School
5,000
$1.00
1
5%
250
$20.00
$7.50
$
1,875
$5,000
-63%
Nursing Schools
5,000
$2.00
3
20%
1,000
$10.00
$30.00
$
30,000
$10,000
200%
Total
10,000
$1.50
2
12.5%
1,250
$12.00
$25.50
$
31,875
$15,000
113%
Optimized
8,000
$2.43
1
22%
1,760
$11.05
$30.00
$
52,800
$19,440
172%
Slide24Bid Management
Term
Clicks
CPC
Pos
CR
Leads
CPA
AvgPrice
Revenue
Spend
GM
Nursing School
5,000
$1.00
1
5%
250
$20.00
$7.50
$
1,875
$5,000
-63%
Nursing Schools
5,000
$2.00
3
20%
1,000
$10.00
$30.00
$
30,000
$10,000
200%
Total
10,000
$1.50
2
12.5%
1,250
$12.00
$25.50
$
31,875
$15,000
113%
Optimized
8,000
$2.43
1
22%
1,760
$11.05
$30.00
$
52,800
$19,440
172%
Slide25Ad
Industry Structure
Advertiser
Audience
Media
$
$$$
$$$$$
$$
Slide2626
Beyond keyword matching
Matching ads is relatively simple for explicitly bid
keywords
Exact match
Covering only those is
not enough
– advertisers need volume !
Broad match
(or
advanced
match
)
Suppose your ad is
“Low prices on Seattle hotels”Naïve approach: bid on all queries that contain the word
“Seattle”
Problems
‘
Seattle's
Best Coffee
Chicago
’
‘
Alaska
cruises start
point
’
Ideally:
bid on
queries
related to
Seattle as a travel
destination
The system should facilitate concept-level ad matching
Slide2727
Exact match (query = bid phrase)
Broad match via query rewrites
Content match: reduce the problem to exact match
(extract bid phrases from pages)
Essentially record lookup
An ultra-brief history of approaches to Web advertising
The new approach:
knowledge-based
ad retrieval
Elaborate query expansion
Ad indexing and scoring using all the info available
Bid phrases, title, creative, URL, landing page, etc.
Akin to document indexing in IR
2
nd
pass relevance reordering (re-ranking)
Using features not available to the 1
st
pass model (e.g., set-level features, click history)
The old school:
database-style
ad matching
Learning to rank
Slide2828
Our approach to sponsored search
Query
Front end
Candidate ads
Revenue reordering
Ad slate
Ad query generation
First pass retrieval
Relevance reordering
Ad search engine
Ad query
Miele
<Miele, appliances, kitchen,
“appliances repair”, “appliance parts”,
Business/Shopping/Home/Appliances
>
Rich query
Use bid
phrases
and
landing
pages to
augment the ads (cf. query expansion)
Slide2929
Ads vs. Web pages
Very short
Optimized for presentation, not for indexing
creatives have low SNR
Legacy bid-phrase-centric definition dictated by the exact match scenario
very limiting today
Complex structure
Not overly short (at least more often than not
)
Simple structure: sections/subsections and (optional) HTML markup
Ads
Web pages
Slide3030
Smaller corpus
Much broader notion of relevance (relatedness)
Different (but rich) information is available
bids, budgets, landing pages, conversion rates,
elaborate nested structure of campaigns, …
Huge corpus
Mainly aiming at pages that subsume all the query terms
Strict notion of relevance
Anchor text and other valuable signals are available
Ad retrieval vs. Web search
Ad retrieval
Web search
Slide31RPV Optimization:
Problems with Sort by CPC
Example Term: "
mba
"
Ad Title
Univ. of
Phx
: Online MBA
Univ. of Washington MBA
Ad Body
100% online university. Fully accredited.
Foster School of business. Top 30 ranked.
CPC
$10.00
$0.50
CTR
0.01%
4%
Position
#
1
#10
RPV
$0.0010
$0.0200
Slide32RPV Optimization:
Problems with Sort by CPC
Example Term: "
mba
"
Ad Title
Univ. of
Phx
: Online MBA
Univ. of Washington MBA
Ad Body
100% online university. Fully accredited.
Foster School of business. Top 30 ranked.
CPC
$10.00
$0.50
CTR
0.01%
4%
Position
#
1
#10
RPV
$0.0010
$0.0200
Slide33RPV Optimization
Sort by (
CPC_Bid
x CTR)
Slide34One does not
have
to show ads!
Roughly
half
of the queries have no ads
Repeatedly
showing non-relevant ads can have detrimental long-term
effects
Modeling actual (short- and long-term) costs of showing non-relevant ads is very difficult
Goal: predict
when (not) to show
ads
34
Learning
when (not) to advertise
(
CIKM 2008,
Broder
et al.)
Should we show ads at all
Slide3535
Two approaches:
Thresholding vs. Machine Learning
Global threshold
on relevance scores of
individual ads
Only show ads
with scores above
the threshold
Problem:
Scores
are not necessarily comparable across queries!
Learn
a binary prediction model for
sets of ads
Features defined over
sets of ads
rather than individual ads
Relevance
(word overlap, cosine similarity between ad and query/page etc.)
Result set cohesiveness
(coefficient of variation of ad scores, result set clarity, entropy)
Should we show ads at all
Slide3636
Features
Relevance features
Word overlap, cosine similarity between ad and query/page
Vocabulary mismatch features
Translation models
PMI between query/page terms and bid terms
Ad-based features
Bid price (higher bids
often indicate
better ads)
Result-set
cohesiveness features
Coefficient of variation of ad scores (
std
/mean) Result set clarity If the set of ads is very cohesive and focused on 1-2 topics, the relevance language model is very different from the collection modelEntropy
Slide3737
Estimating
advertizability
of tail queries for sponsored search
[Pandey et al. SIGIR
2010]
The
previous model
mainly looked at the candidate set of
ads
An alternative approach is to look at the words in the queries
“download”, “buy”, “insurance”, “flight”, “hotel”
amenable to advertising
“weather”, “free”, “university”
low ad CTRParameters (= word propensities to attract ad clicks) are estimated from training data
Slide3838
Binary classifier (relevant / non-relevant ads)
Baseline: text overlap features (query/ad)
Click history (query/ad) with back-off
Click propensity in query/ad translation
Incorporating click history
(WSDM 2010, Hillard et al.)
Cold start (i.e., no click history) is OK
Using click data to overcome synonymy
Query = “
running gear
”
Ad = “
Best jogging shoes
”
Results
Query coverage
9%
Ads per query
12%
CTR
10%
Same # clicks on fewer ads
Counting clicks for query/ad word pairs
IBM Model 1
Bayes’ rule
Should we show ads at all
Slide3939
Ready to buy or just browsing ?
Classifying research- and purchase-oriented sessions
Inferring
eye gaze position from
observable
actions
Keystrokes, GUI (scroll/click), mouse movement, browser (new tab, close, back/forward)
Research vs. purchase classification (in lab): F1 = 0.96
Ad clickthrough
in sessions classified as
Purchase
> 2X
compared to sessions classified as ResearchPredicting future ad clicks: F1 = 0.07 0.17 (
+141%
)
Incorporating multi-modal interaction data (SIGIR 2010, Guo & Agichtein)
Should we show ads at all
Slide40Keyword Opacity
Impr
CTR
Clicks
CPC
CR
Leads
CPA
Spend
Nursing School
100,000
5%
5,000
$1.00
5%
250
$20.00
$5,000
Nursing Schools
10,000
50%
5,000
$2.00
20%
1,000
$10.00
$10,000
Total
110,000
9%
10,000
$1.50
12.5%
1,250
$12.00
$15,000
MatchDriver
110,000
9%
10,000
$2.00
12.5%
1,250
$16.00
$20,000
Slide41Landing Page Analysis
What?? No “Christmas”
Slide42Landing Page Analysis
No “Christmas” here either!
Slide43Ad
Industry Structure
Advertiser
Audience
Media
$
$$$
$$$$$
$$
Slide44End Users
Don’t bug me
Unless I like what
you have to offer
Slide45Ads as another source of content for enriching Web search results
45
“I do not regard advertising as entertainment or an art form, but
as a medium of information….”
Slide46Better Matching
Context detection
GPS, location
App vs. content
In-game
Info seeker vs.
transactor
Calendars/schedules/events
Social networks/status
Twitter - now
Behavioral – esp. w/knowledge of specific site behaviors
ContextualPrivacyGoogle “AOL search data”
Slide4747
Textual advertising
Sponsored Search
Ads
driven by search keywords
a.k.a
.
“keyword driven ads
” or
“paid search
”
Content Match
Ads driven
by the content of a web
page
a.k.a
. “
context driven ads
” or “contextual
ads
”
Textual advertising on the Web is strongly related
to NLP and information retrieval
Slide48Context?
Flowers
Mentos
gum
Trial Prep
Credit score
Cosmetics
Hampton Inns
WeightWatchers
Vacation Home Rentals
Home Depot Web Hosting
WebMD
Colon Cleanse – Warning
My Teeth Aren’t Yellow
Classmates.com
Slide49Testing
Slide50Nine Differences
A
B
upgrade!
Lost 90% revenue….
Reverting coupon code increased CR 6.5%
Slide51Testing
Slide52A/B Split Test
Slide53Testing
Sample Size, margin of error, confidence
x
=
Z
(
c
/
100
)2r(100-r)
n = N x/((N
-1)
E
2
+ x) E = Sqrt[(N - n)
x
/
n
(
N
-1
)
]
Slide54Sample Size Problems
So many ideas, so little to sample…
Disproportionate advantage to scale
Multivariate testing
Taguchi Method
Method for calculating signal-to-noise ratio of different parameters in an experimental design
Allows optimization with A/B test of each cross-product
00
Slide55Testing
Slide56Repetition
Slide57Slide58Professional Photos
Before
After
We observed an immediate 30% increase in conversion rates
Slide59Fact Sheet Design
Existing Schools (n=1,428)
CR
Best
51.1%
Worst
0.4%
Average
11.6%
Test
# Schools
CR Lift
Professional photo
1
30%
More RFI buttons
3
21%
Marketing voice, more programs listed
1
28%
Photos + Marketing voice, more programs
1
50%
Slide60The Great Divide
Brand
Direct Response
Emotions
Indirect benefits
Banners, TV, stadiums
Transactions
Gross profits
Search,
coupons, 1-800, radio, mail
Slide61Graphical ads – all over the web!
61
Slide62Advertisers: basic principles
Display advertisers aim to
Reach
audiences of interest with certain frequency
Achieve certain
performance
of the campaign
In marketing terms, display advertising aims for both
Brand marketing that raises the awareness for a brand
Direct marketing
For both reach and performance campaigns, the key challenge is to select the right audience, i.e. to
target the right usersMany advertisers have multiple simultaneous campaigns with different goalsIn all cases, ultimate goal is to maximize ROI
62
Slide63The tasks in targeting
63
Targeting can be decomposed into three related tasks
User profile generation
Audience selection
: find the best audience for a given ad
Performance prediction
: find the best ad for a given impression
User profile generation
Understanding the user based on his past activity and available metadata
Determining what are the predictive features
Audience selection
Define the objective
Training data,
Modeling methods
Can be done offline/near-line and by third party.
Slide64Overview
Demographic targeting
Geo targeting
Behavioral targeting
Retargeting
64
Slide65Using demographics in advertising
Important indicator of people’s interest and potential of a conversion
Imagine you want to sell a $50K sports car. Who do you target?
Used widely in traditional advertising:
TV, magazines, etc. maintain very detailed statistics of their audience
Common classic dimensions:
Age
Gender
Income bracket
Location
Interests (“Golf enthusiast”)
65
Slide66Obtaining Demographic Information
User supplied demographic information
Most reliable – if filled correctly
In some cases 15-20% of users born on 1
st
of January
Most users see very little incentive to fill the form
Privacy concerns
But CC data, shipping address, etc, are ~100% reliable.
Inferred demographic information
Guess demographics based on browsing/querying behavior
74% women/ 58% of men seek health or medical info online
34% women/ 25% men seek religious info online
Wider reach – virtually every user
66
Slide67Geo targeting
Goal: determine user location
Home
Current
Often wrong
Inputs
Registration data
IP (Main source!)
Browser default language
Search language
Etc …Lots of papers/results, but no time to discuss …
67
Slide68Behavioral Targeting
A technique used by publishers and advertisers to increase campaign effectiveness based on a given user’s
historical behavior
:
Previous searches/search sessions
Previous browsing activity
Previous ad-clicks
Previous conversions
Declared demographics data
Etc.
Utility – everyone wins! (at least in theory
)
Advertisers: get a more appropriate/receptive audience, increased conversion rate, better ROI
Publishers: can ask for a premium
Users: see more interesting ads
68
Main techniques
Slide69Basic search retargeting scheme
User searches for shoes on the XYZ engine
69
Site ABC sends ad request + XYZ cookie to XYZ
XYZ creates shoes ad based on XYZ cookie that remembers “shoes”
Slide70Basic browse retargeting scheme
User Joe views skiing site ABC that contains some XYZ produced ad or just “beacon”
Joe’s XYZ cookie captures visit to ABC
Now on site DEF Joe sees ski ads
70
Sends ad request + XYZ cookie to XYZ
XYZ creates skiing ad based on XYZ cookie that remembers “skiing”
Alternative: XYZ puts ad for ABC
Slide71Example: dapper.net
71
Slide72Google Analytics
Why
Slide73Opportunities Today
Conversions
Low-RPV
Waste
Simplicity
Risk
Scaling local, hyperlocal
Data exchanges
Under-monetized sites
Context
Slide74Summary
Conversions
Risk
Context
Testing