Transcript Andrewcs286r

The Value of Reputation on eBay:
A Controlled Experiment
Andrew Berry
11/25/08
Internet Market
• No outside instrument of reputation
• Temptation for sellers to misrepresent
products is great
• Temptation to sloth
– Example: Ship slowly after receiving payment
• Buyers are forced to assume risk
– Should lower the price buyers are willing to
pay
Internet Reputation Systems
• Necessary to substitute for traditional
seller reputation mechanisms
• Inform buyers whether potential trading
partners are trustworthy
• Deter opportunistic behavior
– Past actions affect future business
– Open record of transaction history
Internet and Reputation
• Information can be tallied costlessly on a
continuous basis
• Written assessments are easily assembled
• Information can be costlessly transmitted
across many customers
Prior Studies
• Observational studies of a set of items
whose sellers had varying reputations
• Studies correlate reputations with auction
outcomes
• Most studies found that buyers paid more
to sellers with better reputations
Observational Studies
• Can only examine reputation in markets
for standardized goods
• Plagued by omitted variable bias
– Discussion: What factors could lead to
OMVB?
Prior Work
• Shows that reputation affects:
– Probability of a sale
– Price
– Probability that bidders enter an auction
– Number of bids in the auction
– Assessment of a seller’s trustworthiness
Confounds with Observational
Studies
• Private email communications
– Can influence buyer willingness to bid high
– Unobservable to researcher
• Layout aesthetics
• More experience may mean higher quality
Advantages to Field Experiments
• Automatically controls for confounds
• Ability to investigate reputation for nonstandard goods with unavailable book
value
eBay Reputation System
• To leave feedback a transaction must have
occurred
• Buyer and seller can rate each other
• Opportunity for a one-line text comment
• Rated individuals can respond to
comments that they feel are unfair
eBay Reputation System
• A buyer can click the net score in order to
see a detailed breakdown
• Scroll to see individual comments
• Users may change identity by registering
again
• No search mechanism to find negatives
General Page
Feedback Scores and Stars
• The Feedback score is the number in
parentheses next to a member’s user ID
• Next to the Feedback score, you may also see a
star
– A Feedback score of at least 10 earns you a yellow
star
– The higher the Feedback score, the more positive
ratings a member has received
– As your Feedback score increases, your star will
change color accordingly, all the way to a silver
shooting star for a score above 1,000,000
Feedback Profile
Key Areas
• Positive Feedback Ratings
– The percentage of positive ratings left by
members in the last 12 months.
– This is calculated by dividing the number of
positive ratings by the total number of ratings
(positive + neutral + negative).
Feedback Profile
Key Areas
• Recent Feedback Ratings
– The total number of positive, neutral, and
negative Feedback ratings the member has
received in the last 1, 6, and 12 months
Feedback Profile
Key Areas
• Detailed Seller Ratings
– provide more details about this member’s
performance as a seller
– Five stars is the highest rating, and one star is
the lowest
– These ratings do not count toward the overall
Feedback score and they are anonymous
– Sellers cannot trace detailed seller ratings
back to the buyer who left them
Feedback Profile
Key Areas
• All Feedback
– Provides feedback from all transactions
– Detailed user comments from transaction
history
eBay Reputation Trends
• Half of the buyers provide positive
feedback
• This positive feedback is similar to saying
“thank you” in everyday discourse
• Sellers receive negative feedback only 1%
of the time
• Buyers receive negative feedback only 2%
of the time
Halftime
• Thought Questions:
– 1. What do you think are biggest factors that
account for so much positive feedback and so
little negative feedback? Is the reputation
system that good or is there something else at
play?
– 2. You’ve seen the eBay interface. Is there too
much information to digest? What do buyers
and sellers actually look at?
Experimental Setup
• 8 eBay identities
– STRONG
• Net score of 2000 with one negative feedback
– NEW
• 7 new eBay identities with no feedback
• Matched 200 items sold by STRONG with
one of the new sellers
Experimental Setup
• Vintage postcards sold
– Asymmetry between seller and buyer about
condition
– No established book value to guide buyers
• 12 week experiment
• 5 new sellers presented 20 lots each for
sale
• 2 sellers presented 50 lots each
Experimental Setup
• To prevent customers from identifying the
experiment:
– Lots listed in a category that has thousands of
lots for sale
– New sellers had slightly different format for
listings
– Each half of each matched pair was listed at
different times
Second Experiment
• Tested the effects of negative feedback
• 3 week experiment
• Purchased lots from three of the new
sellers to give negative feedback
• Two categories of negative comments
– Item did not match description
– Item was in worse condition than listed
Second Experiment
• Negative feedback was displayed at the
top of the comments page
• 35 more matched pair lots
Hypotheses
• Hypothesis 1:
– Buyers will view an established seller as less
risky and pay more
• Hypothesis 2:
– New sellers with negative feedback will reap
lower profits than those without negative
feedback
• Thoughts on these hypotheses or the
experimental setup?
Imperfect Observation
• Neither STRONG nor NEW sell
– Gives little information
• Either STRONG or NEW sells
– Provides a lower or upper bound on the ratio
of a buyer’s willingness to pay
• Both STRONG and NEW sell
– Ideal situation
Slight Detour
• Censored Normal Regression Models
– Arise when the variable of interest is
observable in certain conditions
– OLS is biased when the variable is
unobservable
– Use these models when the independent
variable is known, but the dependent variable
is not
• Allows us to include data where either
NEW or STRONG sold
Slight Detour
• Why don’t we just use data where both
sell?
– Reduce the sample size too much
– Truncation Bias
• New sellers sold fewer lots
– Observations of sold lots for NEW reflect more extreme
points than for STRONG
Results
• Sign Test
– If STRONG sells but NEW does not, the sign is
positive
– If both sell, the observed difference is used
• One sided sign test approaches significance
• Probability of sale was not independent of two
sellers
• STRONG sold 63% of time
• NEW sellers sold 56% of the time
Results
• Censored normal estimation
– Parametric Estimate
– Used lots where either or both sellers sold
• Estimated mean difference is significant
– P = .044
• Suggests buyers are willing to pay 8.1%
more for lots sold by STRONG
Results
• Second experiment shows negatives in a brief
reputation don’t necessarily hurt revenues
• NEW sellers without negatives sold 16 of 35 lots
• NEW sellers with negatives sold 14 of 35 lots
• No significant differences
• Sellers without negatives often received lower
prices when they did sell
– Favored sellers without negatives 9 times
– Favored sellers with negatives 11 times
Threats to Validity
• Experiment 1
– Differences in listing quality
– Repeat customers and private reputation
– Multiple purchases
• Experiment 2
– Small sample size
– Profile Design
– Timing of negative feedback
Discussion
• Validity of results?
• Is the percentage of negative feedback
more valuable?
– Dewally and Edgerington (2006)
• Do buyers click through profiles or merely
rely on overall score?
• How do we test if the market is over or
underestimating reputation?
Discussion
• Given the results, what moral hazards
does this pose for the structure of the
eBay market?