Paired Comparisons - PowerPoint Presentation

Slide1

Paired Comparisons

Testing DimensionsSlide2

Studying attributes of items in a

cultural domain

Free listing is a way to define the contents of a

domain.

Pile

sorts and triad tests are ways to measure

similarity.

B

ut

we want to do more. We

want

to describe the attributes of the relations among the elements of a domain.

To evaluate the items on an attribute, we can use several techniques: ratings, rankings, and paired comparisons.Slide3

Rating scales vs. paired comparisons

Ratings are the most common way to measure attributes in the social sciences

How dangerous is [this illness]:

1 2 3 4 5

This

produces many tied scores

How many points should we allow?

An even number or an odd number of choices?

In some research, you

don

’

t

want to allow people to

equivocate

Scales

are often the best we can do for measuring attributes, but for short lists of items, rank-ordered data is better.Slide4

Paired comparisons

Paired comparisons produce perfect rank-ordered lists.

There are N(N-1)/2 pairs of items, so there are 6 pairs in 4 items:  

AB

AC AD BC BD CD

Each element occurs

N – 1 times

and each element is paired with each other element.

You can count how many times each element “wins” in a paired comparison of items. Slide5

How to do

PC

’s in

Anthopac

Produce a person-by-pair matrix.

Convert this matrix into a person-by-item matrix where cells indicate the number of times, for each person, each item “won” in its contest.

This matrix can be summed by columns to produce a vector, indicating, for each item, over all informants, how many times the item won.

Anthropac

does all this when it imports paired comparison data.Slide6

The method of paired comparisons is an alternative way to get

rank orderings of

a list of items in a domain.

There are

n

(

n

–1

)/2

pairs of

things in a list of things.

Suppose 5 colors

:

red, green

, yellow, blue, and brown.

The next slide shows

the paired comparison

test to

find

an

informant’s rank-ordered preference for these five colors

.Slide7

In each of the following pairs of colors, please circle the one you like best

:

RED GREEN

RED YELLOW

RED BLUE

RED BROWN

GREEN YELLOW

GREEN BLUE

GREEN BROWN

YELLOW BLUE

YELLOW BROWN

BLUE BROWNSlide8

Order effects

The pairs of colors are listed in such a way that you can see how the 10 of them exhausts the possibilities for five items.

For a real test, scramble the order of the pairs to ensure that the order of the items in a list doesn’t influence the choices that informants make.

Use

Anthropac

to do this.Slide9

Some more examples of paired comparisons

Choose the animal in each pair that is more [vicious, exotic, expensive]

Choose the illness in each pair that is more [scary, hard to treat, life threatening]

Choose the food in each pair that is [better for you, harder to find, best for children]

Choose the crime in pair that [you’re most afraid of, deserves more punishment, harder to stop] Slide10

How Anthropac calculates the rank order in paired comparisons

For each informant, count

up how

many times

each item in a list ‘‘wins

’’—that

is, how many times it was circled.

For a list of illnesses you expect cancer to win over most other illnesses.

It

gets

interesting

when

compare

the average rank ordering

across ethnic groups of, say, high blood pressure and diabetes.Slide11

Plusses and minuses of paired comparisons

Plus: People make one judgment at a time -- easier than rank ordering a list of items by staring at all the items at once.

Plus: You can read a list of pairs to nonliterate informants.

Minus: With 20 items, informants make separate 190 judgments.

With 60 pile-sort cards, informants make 1770 judgments – but it’s all in their heads.