Transcript 2: Poster_Ceren - Computer Science

Strategic State Estimation in Uncertain and
Mixed Multiagent Environments
Roi Ceren
Computer Science Department
The University of Georgia
[email protected]
Introduction
Approach
Results
Humans, in generating probability estimations for their
predictions, often lack justification. In realistic situations, such
The experiments were conducted through
the University of Georgia Psychology
department using the student research
pool (RP) and Air Force ROTC.
Participants are introduced to a flight
simulator (FlightGear) with a custom
environment developed by our lab, known
as GaTAC (Georgia Testbed for
Autonomous Control of Vehicles).
Results were observed utilizing information contained within the
experimental spreadsheet. The ratio between believed and stated
odds (the X parameter) was analyzed. The closer X is to 1, the
as military operations, judging uncertainty can be difficult. As
current research does not apply to complex situations, we
designed three studies in order to establish and encourage
field-valid probability assessments with effective update
mechanisms. Utilizing UAV simulation, we were able to present
a complex, real environment to subjects. Each study seeks to
identify causal sources of poor probability assessments and
seeks to rectify the associated cause. Our most recent progress
on this research involves the completion of the first study, where
we investigated the impact of subjective expressions of
probabilities and potential methods to correct them.
Background
In the realm of decision making, uncertainty serves to
obfuscate optimal or correct choices. Human judgment
suffers from cognitive biases, which presents unique
obstacles to efficient and meaningful probability estimations.
This is especially apparent in complex environments, such
as an UAV pilot in a multiagent environment.
The goal of these ARO studies is to identify potential loci of
prediction errors and establish potential mechanisms for
more effective field-valid assessment and update
techniques. The project consists of three studies:
The current wave of
research (Study 1) attempts
to highlight disparities
between a verbal
expressions of probabilities
and the actual belief of a
given individual.
Participants controlled a UAV, flying over the
Bagram Air Force Base, and were tasked with the goal of
reaching a designated target. Success was incentivized with a
$3 payout for each victory in this game.
Participants experienced two phases of
experimentation: training and test
phases. In the training phase,
participants were given the opportunity
to grow accustomed to the interface. In
the test phase, participants experienced
15 trials. During each decision point,
players filled out a questionnaire
quantifying their probability
assessments.
The experimental group was given the
opportunity at each decision point to
validate their probability by using a
random number generator (called a
bingo cage) as an opportunity to make
a move with no risk, based on their
estimate.
For each decision point, if the player
chose to make a move without the
random event, the probability of
winning the random event increases.
Inversely, choosing the random event
decreases the likelihood of winning the
random event. [1] Ideally, the probability
estimates for experimental player will
converge on a validated estimate.
closer the believed probability assessment is to the stated
probability assessment.
Population mean of X for RP: 0.07178
95% confidence interval: (0.000007457, 690.97)
Population mean of X for ROTC: 1.191
95% confidence interval: (0.8542, 1.6608)
Both of these intervals contain 1 for X, indicating that the players
may not have been significantly inflating or deflating their
verbal estimations. Additionally, the tighter bound on the ROTC
participants indicate a more consistent, better behaved research
pool.
Contributions
These experiment teaches us about the limits of human probability
estimations and potential mechanisms that can assist in promoting
better estimation. This study offers ways to quantify and analyze
an individual’s cognitive biases as well as establish better
assessment and update techniques.
References
1. J. E. Mazur. Estimation of indifference points with an adjusting-delay procedure. Journal of
the Experimental Analysis of Behavior, 49:37–47, 1988.
2. B. DeFinetti. Foresight: Its logical laws, its subjective sources. In H. Kyburg and H. Smokler,
editors, Studies in Subjective Probability, pages 93–158. New York City, New York, 1964.
3. D. Kahneman, P. Slovic, and A. T. (Eds.). Judgment under Uncertainty: Heuristic and Biases.
Cambridge University Press, 1982.
4. D. Kahneman and A. Tversky. On the psychology of prediction. Psychological Review,
80:237–251, 1973.
Acknowledgments
This work was performed in collaboration with Profs. Prashant Doshi
(CS), Adam Goodie (Psychology) and Dan Hall (Statistics). We
acknowledge the support of a grant from the Army RDECOM, grant #
W911NF-09-1-0464, to Prof. Prashant Doshi (PI). Special thanks to
Ekhlas Sonu for his work on GaTAC, the FlightGear community for quick,
timely assistance, and Matthew Meisel for his help in running and
maintaining the experiment.