Transcript Noonan - Georgia Institute of Technology

Dumb Supermodels?
On the catwalk between social
and physical models
Douglas S. Noonan
School of Public Policy
Georgia Institute of Technology
On models and “supermodels”
• The topic:
– nature of models, and
– models of nature
• From the perspective of an economist and
policy analyst….
Some basic questions
• What is the air quality in a particular place,
time? The climate?
• What was the air quality in a particular
place, time? The climate?
• What will be the air quality in particular
place, time? The climate?
• What are people doing right now?
• What were they doing? What will they do?
What we don’t know
• We have surprisingly few “facts” or direct
empirical observations.
– Most of our information, our knowledge of the world
actually takes the form of estimates
• I don’t know what temperature it is in the hallway, but I can
guess.
• Based on a model that combines:
– theory
» knowledge of (or assumptions about) constraints and
forces in the system that produce temperature
– other data or measures
» “nearby” (spatially, temporally) values
• Ultimately, I rely on an estimate of the phenomenon, which
derives from a model of it
What we don’t know
• There is a lot we don’t know.
• There is a lot that we estimate – using models
– We estimate
• Atlanta’s population and AQI yesterday
• the composition and behavior of Atlanta’s auto fleet
• the concentration of CO emitted from a tailpipe of a car
driven past a remote sensor
• its contribution to local air quality
• etc.
Some interesting questions
• What are the determinants of local air
quality? Of climate change?
• What are the impacts of changes in local
air quality? Of climate change?
• What is the optimal change in local air
quality? In the climate?
• How can we design policy mechanisms to
improve local air quality? To mitigate
climate change?
The nature of models
• These big questions involve big, complex
systems
– Some simplification required for tractability
– Some simplification desirable for generalization
• So, we undertake to develop a model … a
simplified representation of reality that captures,
we hope, the essential elements and that
explains the phenomena of interest.
– How do we do?
Some prominent models
• Models of:
» at various scales, resolution (spatial, temporal)
– weather
– air quality
– climate
– transportation
– agricultural, industrial production
– innovation
– (public) health
– demographics
Why model?
• It seems like an obvious question, especially to
most modelers
–
–
–
–
basic research
commercial interests
policy relevance
some other “public interest”…
• More practically, models that explain
phenomena also get used in other ways
– for estimating values
– for forecasting
• the results used pervasively for decision-making by
individuals, firms, public agencies, policymakers, etc.
Example model
• Suppose that we are interested in
modeling the effects of “air quality action
days”
– effects on what?
•
•
•
•
•
Health of those exposed
Emitters’ behavior
Others’ behavior
NAAQS attainment status
Air quality realized
Example model
• Suppose that we are interested in
modeling the effects of “air quality action
days”
– effects on what?
•
•
•
•
•
Health of those exposed
Emitters’ behavior
Others’ behavior
NAAQS attainment status
Air quality realized
Here be Supermodels
• We might thus imagine a “supermodel” that
incorporates multiple models
– Atmospheric models
» mixing and movement of airborne chemicals
– Meteorological models
» local weather variations
– Economic models
» industrial emissions, (energy) supply models
– Transportation models
» emissions quantity, location, timing
– Other models
• psychology, epidemiology, etc.?
Endogeneity
• An essential modeling question is the extent to
which influences from one system/model
depend on another
– Are emissions exogenous? Is weather exogenous?
Etc.
• A supermodel might well include values or
estimates from multiple models
– But do those estimates depend on each other?
“Smart” Supermodels
Physical models
Social models
(atmospheric,
meteorological,
etc.)
(economic,
transportation,
etc.)
Here be Supermodels
• We might thus imagine a “supermodel” that
incorporates multiple models
– Atmospheric models
» mixing and movement of airborne chemicals
– Meteorological models
» local weather variations
– Economic models
» industrial emissions, (energy) supply models
– Transportation models
» emissions quantity, location, timing
– Other models
• psychology, epidemiology, etc.?
Dumb Supermodels
• A “dumb supermodel” might be thought of
as a model that incorporates multiple
models independently
– e.g.,
T = T(Q, W, A)
A = A(Q)
» So, we could rewrite it just as: T=T(Q,W,A(Q))
– But what about a “smart supermodel” that
endogenizes air quality?
Smart Supermodels
• A “smart supermodel” might be thought of
as a model implicitly defined by multiple
interdependent models
– e.g.,
T = T(Q, W, A)
A = A(Q) + 
Q = Q(W, T)
W=W
– In this case, inputs to the model are seen to
depend on the output…
Air Quality Alert Impacts
• A dumb supermodel could estimate the impacts
of A on T
• and thus on Q (one of the policy objectives)
– It would take A and Q as exogenous to T
• A smart supermodel could also estimate the
impacts of A on T
– It would simultaneously estimate a system with
endogenous A, Q, T, and other phenomena
• finding any exogeneity, a “natural experiment”, or something
to identify the system poses the big research challenge
Air Quality Alert Impacts
• In the Atlanta context, we have daily
variation in predicted and actual ozone
concentrations
Air Quality Alert Impacts
• In the Atlanta context, we have daily
variation in predicted and actual ozone
concentrations
• The predictions (“smog alerts”) aim to
affect behavior
– Reduce (or re-time) emission-causing
behavior like driving autos
– Enable people to avoid exposure
• Similar programs in ~250 cities in US
Air Quality Alert Impacts
• Atlanta (and other cities) include these
voluntary programs in their SIPs
• What affect do these programs have
– on behavior?
– on actual air quality?
• Henry & Gordon at GSU looked into it, and
found large effects in the 1998
Air Quality Alert Impacts
• I combine 2001 data on:
– travel (from ARC’s household travel diary),
– weather (actuals and forecasts from NWS),
– air quality (actuals and forecasts for EPD).
• Trip behavior estimated based on traveler
and environmental characteristics.
– Weather affects both behavior and air quality
Implications and Impacts
• Ozone levels and forecasts poor predictors of travel
– No O3 effect on household’s number of driving trips
– Driving trip length falls in O3 shocks; unrelated to O3 alerts
– O3 alerts have positive & insig. effect on household-miles-driven
– Higher O3 forecasts associated with earlier departure times (~3
min./ppb)
– O3 alerts (actual or forecast) associated with later departure
times (~34 min.)
– Employees of firms with alternative commute perks never took
those modes on alert days
– Elderly never biked/walked on alert days
0
.02
.04
.06
.08
.1
Histogram for departure time, all days
0
5
10
15
departure time, decimal hours
20
25
0
.02
.04
.06
.08
.1
Histogram for departure time, red-alert days
0
5
10
15
departure time, decimal hours
20
25
Implications and Impacts
• Ozone levels and forecasts poor predictors of travel
– No O3 effect on household’s number of driving trips
– Driving trip length falls in O3 shocks; unrelated to O3 alerts
– O3 alerts have positive & insig. effect on household-miles-driven
– Higher O3 forecasts associated with earlier departure times (~3
min./ppb)
– O3 alerts (actual or forecast) associated with later departure
times (~34 min.)
– Employees of firms with alternative commute perks never took
those modes on alert days
– Elderly never biked/walked on alert days
Outdoor activities/exercise
• Logits on trip involving outdoor activity
– Negative effect of forecasted prob. of rain
– Negative effect of forecasted ozone levels
– No effect of ozone alerts
Air Quality Alert Impacts
• Questions remain:
– Does the model generating “smog alerts”
depend on behavioral changes in response to
other determinants of ozone levels?
Behavioral changes in response to the alerts
themselves?
Smart or Dumb Supermodels
• In practice, how well supermodels integrate
physical and social systems remains an open
question…
…even when the original purpose for the model
is to explain the impact of one system on the
other
– Commonly, effects are modeled as one-sided, where
environmental systems respond to human influences
or vice versa
Smart Supermodels
(“on the catwalk”)
• Making estimates (projections, forecasts)
using Supermodels smartly involves:
– modeling environmental systems
– modeling social systems
– modeling the interdependence between them
• These feedback mechanisms are often vastly more
complex and misunderstood than the outcomes
(e.g., environmental quality, social welfare) the we
ultimately care about
Other Supermodels
• Air pollution and epidemiology
• Global climate change
• Environmental Kuznets Curves
Other Supermodels
• Air pollution and epidemiology
Health = H(Enviro)
Classic doseresponse curve
vs.
Health = H(Enviro, Indiv)
Enviro = E(Indiv, Health)
Indiv = Indiv
Here, environmental
quality is not
exogenous; possibly
chosen based on
individual traits
Other Supermodels
• Air pollution and epidemiology
• Global climate change
Climate = C(economic activity)
vs.
Climate = C(economic activity)
Economic activity = E(Climate)
Other Supermodels
• Air pollution and epidemiology
• Global climate change
• Environmental Kuznets Curves
On the Catwalk
• The mechanism linking the physical and social
systems is crucial, even if it is not the object of
study itself
– What feedback mechanisms are there?
– How do the models capture them?
– More attention need be paid to explicitly modeling
adaptations of natural, social systems to one another
• Malthus wasn’t the first modeler to make the mistake
On the Catwalk
• Ceteris paribus assumptions, while
convenient, often contradict the premise of
the modeling exercise
– “Given current status/trends in X, then we
forecast Y will have this [fate]”
– We give the forecast to inform our efforts to
change X or somehow improve the [fate]
• The forecast is premised on an adaptation that
violates the “given current status/trends”
assumption
Dumb Supermodels and
Trying to be Wrong
• Many dumb supermodels do not incorporate
adaptation or feedbacks, leading to stark
estimates or forecasts
– Strategically, very effective
• A model that shows “what bad would happen (in the absence
of X)” is a good way to motivate X
– Yet if supermodels’ estimates are to be used in
decision-making, these dumb supermodels will be
wrong and biased
• Bad choices, policy result
• Credibility fades