Transcript FASOM

FASOM
Hamburg
July 1-3, 2008
Topics
1. FASOM Basics
2. FASOM Equations
3. Analyzing FASOM
4. Modifying FASOM
I FASOM Basics
Model Scope
•
•
•
•
•
Natural Resources
Agriculture
Forestry
Processing
Food, Fibre, Timber, Energy Markets
Purpose and Objectives
To study and advice policy makers about the
agricultural and forestry sector response
to
a) Policies
b) Environmental change
c) Technical change
d) Socioeconomic change
Policy Scope
•
•
•
•
Climate and other environmental policies
Research subsidies
Agricultural policies
Trade policies
Simultaneous assessment
Structural Changes
a)
b)
c)
d)
Policies
Environmental change
Technical change
Socioeconomic change (population,
preferences)
Methodology
•
•
•
•
•
•
•
Partial Equilibrium
Bottom Up
Constrained Welfare Maximization
Dynamic Optimization
Integrated Assessment
Mathematical programming
GAMS
Limits
Model Structure
Limits
Resources
Inputs
Supply
Functions
Land Use
Products
Markets
Processing
Technologies
Demand
Functions,
Trade
Technologies
Environmental
Impacts
Limits
FASOM = Large Linear Programs
Max
c X
a X
j
j
ij
j

bi
for all i
Xj

0
for all j
j
s.t.
j
Programming
Declare and Assign
• Indexes
• Exogenous data
• Endogenous data
• Equations (endogenous/exogenous)
Run Programs and Display Results
Major Indexes (Sets)
(decl_basicsets.gms)
•
•
•
•
•
•
Regions
Periods
Altitudes
Species
Resources
Cohorts
•
•
•
•
•
Farm sizes
Farm types
Technologies
Products
Substances
Sets
• Identify scope
• Efficient but dangerous and difficult
• Super sets, Sub sets, Tuples
• decl_basicsets.gms
• 2_activesets.gms
• calc_tuples.gms
Exogenous Data
•
•
•
•
•
Resource endowments
Technologies (Inputs,Outputs, Costs)
Demand functions
Environmental Impacts
Policies
Exogenous Data
• Parameters (Tables, Scalars)
• decl_parameters.gms
• Data_*.gms
Data Subdirectories
•
•
•
•
•
•
EPIC
FINDUS
FADN
OSKAR
FAO
BIODIV
+ others
Endogenous Data
• Land use decisions (control variables),
Impacts (state variables), Shadow prices
of constraints
• Positive and Free Variables
• (Integer Variabales)
• 4_eufasom_model.gms
Equations
• Understanding them is
• to understand FASOM
Equations
• Objective Function
• Resource Restrictions
• Technological Restrictions
• Environmental Accounts
• Others
Major Parameters
FOREST_DATA (PERIOD,REGION,SOILTYPE,SOILSTATE,SPECIES,FARM,AGE,ALLTECH,ALLITEM)
PEREN_DATA (PERIOD,REGION,SOILTYPE,SOILSTATE,SPECIES,FARM,AGE,ALLTECH,ALLITEM)
CROP_DATA (PERIOD,REGION,SOILTYPE,SOILSTATE,SPECIES,FARM,ALLTECH,ALLITEM)
ECO_DATA (PERIOD,REGION,SOILTYPE,SOILSTATE,ALLTECH,ALLITEM)
LIVE_DATA (PERIOD,REGION,ANIMAL,ALLTECH,ALLITEM)
FEED_DATA (PERIOD,REGION,PRODUCT,ALLTECH,ALLITEM)
PROCESS_DATA (PERIOD,REGION,ALLTECH,ALLITEM)
PRODUCT_DATA (PERIOD,REGION,PRODUCT,ALLITEM)
MARKET_DATA (PERIOD,REGION,ALLITEM,SDTYPE,SDITEM)
TRADE_DATA (PERIOD,REGION,REGION,PRODUCT,ALLITEM)
RESOURCE_DATA (PERIOD,REGION,ALLRESOURCE,SDITEM)
STOCK_DATA (PERIOD,REGION,PRODUCT,ALLITEM)
LUC_DATA (PERIOD,REGION,SOILTYPE,SPECIES,ALLCHANGE,ALLITEM)
Major Variables
RESOURCE_VAR (PERIOD,REGION,RESOURCE)
FOREST_VAR (PERIOD,REGION,SOILTYPE,SOILSTATE,SPECIES,OWNER,AGE,FORTECH)
PEREN_VAR (PERIOD,REGION,SOILTYPE,SOILSTATE,SPECIES,FARM,AGE,CROPTECH)
CROP_VAR (PERIOD,REGION,SOILTYPE,SOILSTATE,SPECIES,FARM,CROPTECH)
LIVE_VAR (PERIOD,REGION,ANIMAL,LIVETECH)
FEED_VAR (PERIOD,REGION,ANIMAL,PRODUCT,FEEDTECH)
ECO_VAR (PERIOD,REGION,SOILTYPE,SOILSTATE,ECOTECH)
PROCESS_VAR (PERIOD,REGION,PROCTECH)
LUC_VAR (PERIOD,REGION,SOILTYPE,SPECIES,CHANGE)
DEMAND_VAR (PERIOD,REGION,PRODUCT)
SUPPLY_VAR (PERIOD,REGION,PRODUCT)
TRADE_VAR (PERIOD,REGION,REGION,PRODUCT)
Major Equations
OBJECTIVE_EQU
RESOURCEBAL_EQU (PERIOD,REGION,RESOURCE)
RESOURCEMAX_EQU (PERIOD,REGION,RESOURCE)
SOILSTATE_EQU (PERIOD,REGION,SOILTYPE,SOILSTATE)
FORINVENT_EQU (PERIOD,REGION,SOILTYPE,SOILSTATE,SPECIES,OWNER,AGE,FORTECH)
LUC_EQU (PERIOD,REGION,SOILTYPE,SPECIES,CHANGE)
LUCLIMIT_EQU (PERIOD,REGION,SOILTYPE,SPECIES,CHANGE)
STOCK_EQU (PERIOD,REGION,STOCK)
EMIT_EQU (PERIOD,REGION,SUBSTANCE)
PRODUCTBAL_EQU (PERIOD,REGION,PRODUCT)
MINFEED_EQU (PERIOD,REGION,ANIMAL,NUTRIENT)
MAXFEED_EQU (PERIOD,REGION,ANIMAL,NUTRIENT)
Managing the Model Scope
• 2_active_sets.gms
– index elements on/off
• decl_scope.gms
– components on/off
FASOM File Structure
• Main directory (full access)
• Data subdirectories (restricted access)
–
–
–
–
–
–
–
–
Agriculture (FADN)
Crop environmental impacts (EPIC)
Bioenergy (ENFA)
Nature Reserves (HABITAT)
Forest Stands (OSKAR)
Forest Industry (FINDUS)
FAOSTAT
Dynamics
FASOM File Naming (Main Dir)
•
•
•
•
•
•
•
Data files (data_*.gms)
Declaration files (decl_*.gms)
Input calculation files (calc_*.gms)
Program files (#_*.gms)
GAMSCHK files (*.gck), Solver option files (*.opt)
Report files (rep_*.gms)
Special program files (z#_*.gms)
* =
# =
description
1,2,...,k
FASOM Sub Directories
• Store original data files in provider format
• Convert data to FASOM format (names,
resolution, unit consistency)
• Automated programs, slow, infrequent use
• Protect main FASOM directory from
information floods
• Confidentiality (FADN), IPR
FASOM Execution
•
•
•
•
•
•
•
1_load_data.gms (primary data files)
2_active_sets.gms (determine model size)
3_precompute.gms (calculation files)
4_eufasom_model.gms (equations)
5_base_report.gms (base solution report)
6_scenario.gms (repeated solves)
7_results.gms (scenario differences)
Save / Restart
When using GAMSIDE, each program segment must
restart from previous one and save for the next
• 1_load_data.gms (save .\t\a1)
• 2_active_sets.gms (restart .\t\a1 save .\t\a2)
• 3_precompute.gms (restart .\t\a2 save .\t\a3)
• 4_eufasom_model.gms (restart .\t\a3 save .\t\a4)
• 5_base_report.gms (restart .\t\a3 save .\t\a4)
• ...
1_load_data.gms
* Indexes
$include decl_basicsets.gms
$include decl_foresttuple.gms
$offlisting
* Load Data
$include data_crop2000.gms
$include data_faowood.gms
$include data_faocrop.gms
$include data_faoworldcrop2002.gms
$include data_oatsprice.gms
$include data_feed.gms
$include data_land.gms
$include data_cropsupply.gms
$include data_averagecropemission.gms
$include data_euepic.gms
$include data_euluc.gms
$include data_uscrops.gms
$include data_peat.gms
$include data_forprocess.gms
$include data_forproduct.gms
$include data_formarket.gms
$include data_treelog.gms
$include data_rotlength.gms
$include data_forestrent.gms
$include data_forest_set.gms
$include data_forest_nor.gms
$include data_forest_lrg.gms
$include data_forest_sml.gms
2_active_sets.gms
•
•
•
•
•
Define active set elements
Region
Period
Product
Cohort
• Can limit model scope to 1 Region, 1
Period, 1 Product
3_precompute.gms
•
•
•
•
•
•
Assumptions and Definitions (Discounting)
Deal with Data Gaps
Calculations (Data, Tuples)
Checks
Calibrations (Zero Marginal Profits!)
Experiments
• Check whether statements can be moved to
subdirectory
4_eufasom_model.gms
•
•
•
•
•
•
Variable declarations
Equation declarations
On/off switches for some equations
FASOM equations (see part II)
First solve statement
4_eufasom_model.gck
5_base_report.gms
• Declaration of report indexes and
parameters
• Display of FASOM base solution
– Current period (?=? observation)
– Future periods (?=? econometric forecast)
6_scenario_loop.gms
• Define scenarios
– Number (Set)
– Assumptions (Parameters)
– Declare report
• Loop over scenario set
– Restore all parameters to original values
– Implement scenario assumptions
– Solve
– Save solution
7_scenario_report.gms
• Display scenario results
• Comparison between base line solution
and scenario solution
8_scenario_graphs.gms
• Display scenario results graphically
• Need gnupltxy installed
II FASOM Equations
4_eufasom_model.gms
Understanding FASOM Equations
a) GAMS code (indexed equation block)
b) DISPLAYCR output (individual
equations)
c) Display equation components
Condition(al)s
•
•
Restrict cases (very important, many
indexes)
Use GAMSCHK to check
PRODUCTBAL_EQU(REGION,PERIOD,PRODUCT)
$ PRODUCT_TUPLE(REGION,PERIOD,PRODUCT)..
•
calc_tuples.gms
Constrained Optimization
Max
s.t.
z = f(X) ... objective function
G(X) <= 0
... constraints
FASOM = Large Linear Programs
Max
c X
a X
j
j
ij
j

bi
for all i
Xj

0
for all j
j
s.t.
j
Objective Function
(Normative Economics)
Maximize
+ Area underneath demand curves
- Area underneath supply curves
- Costs
± Subsidies / Taxes from policies
Maximum equilibrates markets!
Alternative Objective Function
To get „technical potentials“ from land use
for alternative objective u, simply use
Maximize
u
s.t. all constraints
Examples: u = Carbon Sequestration, Wheat production, Mire area
Is FASOM linear?
• Input prices increase with increasing input
use (scarcity of resources)
• Output prices decrease with increasing
output supply (saturation of demand)
• Hence, FASOM has non-linear objective
function but is solved as Linear Program
using linear approximations
Linear Approximation?
• For well behaved functions: yes
– Concave benefit / convex cost functions
– Decreasing marginal utilities
– Increasing marginal costs
• For ill behaved: no, need integer variables
– Fixed cost (Investment)
– Minimum habitat requirements (Biodiversity)
Linear Approximation in FASOM
• Use step variables (xi) ... with i steps
• Convexity Restriction
Sum(i, xi) = 1
• Identity Restriction
Sum(i, aixi) = x
How many
steps?
DEMAND_VAR(REGION,PERIOD,PRODUCT)
DEMAND_STEP(REGION,PERIOD,PRODUCT,STEP)
DEMAND_CONVEXITY(REGION,PERIOD,PRODUCT)
DEMAND_IDENTITY(REGION,PERIOD,PRODUCT)
Correct Convex Combination?
q
2
3
4
5
100q-0.5q^2
198.0
295.5
392.0
487.5
Welfare
Quantity
Share
Share
0.5
0.5
0.5
343.75
3.5
0.5
342.75
3.5
Model will automatically choose closest neighbor points
Constraints
PRODUCTBAL_EQU
•
•
•
•
Very important
Multi-input, Multi-output
Negative coefficients - Inputs
Positive coefficients - Outputs
RESOURCEBAL_EQU
• Sum resource uses over all technologies,
species, farm structures into an
accounting variable
RESOURCE_VAR(REGION,PERIOD,RESOURCE)
RESOURCEMAX_EQU
• Represent resource limits (endowments)
RESOURCE_VAR(REGION,PERIOD,RESOURCE)
≤
RESOURCE_DATA
(REGION,PERIOD,RESOURCE,”Maximum”)
LUC_EQU
LUC_EQU(REGION,PERIOD,SOILTYPE,SPECIES,CHANGE)
$ LUC_TUPLE(REGION,PERIOD,SOILTYPE,SPECIES,CHANGE)..
• Land use accounting equation
• Combines individual and aggregated
accounting
LUCLIMIT_EQU
LUCLIMIT_EQU(REGION,PERIOD,SOILTYPE,SPECIES,CHANGE)
$(LUC_TUPLE(REGION,PERIOD,SOILTYPE,SPECIES,CHANGE) AND
LUC_DATA(REGION,PERIOD,SOILTYPE,SPECIES,CHANGE,
"MAXIMUM"))..
• Land use change limits
• Should be based on land characteristics
• Currently uses rough assumptions
FORINVENT_EQU
• Forest distribution this period depends on
forest distribution in last period and
harvest activities
• Note:
– Oldest cohort transition
– Initial distribution
– Harvested forests can immeadiatly be
reforested
INTIALFOREST_EQU
• Thinning regime cannot be switched
• Initial thinning distribution unknown
• Let model decide, which thinning regime to
use for initial forests
REPLANT_EQU
• Restricts tree species that can be
replanted after harvest (can have
agricultural break inbetween)
• Regulated by tuple
SPECIESSEQU_MAP(OLDSPECIES,SPECIES)
• Currently restrictive
SOILSTATE_EQU
• To portray important unstable soil properties
• Carbon sequestration effect depends on soil
carbon level
• Equations are implemented, EPIC data are
not yet established
SOILSTATE_EQU(REGION,PERIOD,SOILTYPE,SOILSTATE)
SOILSTATE_EQU
• To portray important unstable soil
properties
• Carbon sequestration effect depends on
soil carbon level
• Equations are implemented, EPIC data
are not yet established
SOILSTATE_EQU
• Contains soil state transition probabilities
Probability_Data(REGION,SOILTYPE,SOILSTATE,SPECIES,
OWNER,COHORT,ALLTECH,POLICY,OLDSTATE)
• Transition probabilities are calculated from
EPIC based carbon functions
Soil Carbon Transition Probabilities
SOC1 SOC2 SOC3 SOC4 SOC5 SOC6 SOC7 SOC8
SOC1 0.81 0.19
SOC2
1
SOC3
0.09 0.91
SOC4
0.31 0.69
SOC5
0.5
0.5
SOC6
0.74 0.26
SOC7
1
No-till Wheat Fallow
SOC8
0.04 0.96
Carbon Functions
Soil Organic Carbon (tC/ha/20cm)
45
40
35
Wheat-Lucerne 3/3
30
Wheat-Lucerne 6/3
25
20
No-till wheat-fallow
15
Tilled wheat-fallow
10
5
0
10
20
30
Time (years)
40
50
STOCK_EQU
STOCK_EQU(REGION,PERIOD,STOCK)
$(STOCK_TUPLE(REGION,PERIOD,STOCK) AND
STOCK_DATA(REGION,PERIOD,STOCK,"DecompRate"))..
• Represents dynamics of dead wood (14 types)
• Linked to emission accounting
PRODUCTINVENT_EQU
PRODUCTINVENT_EQU(REGION,PERIOD,PRODUCT)
• Represents different product life span of
forest products
• Linked to carbon emissions from forest
products
EMIT_EQU
EMIT_EQU(REGION,PERIOD,SUBSTANCE)
$ EMIT_TUPLE(REGION,PERIOD,SUBSTANCE)..
• Accounting equation
• Contains direct emissions and emissions
from stock changes
Resource Balance
•
•
•
* Resource Equations
LUC_EQU(REGION,PERIOD,SOILTYPE,SPECIES,CHANGE) Land Use Change Equation
LUCLIMIT_EQU(REGION,PERIOD,SOILTYPE,SPECIES,CHANGE) Land Use Change Limits
•
•
•
•
* Environmental Externality Accounts
* POPULATION_EQU(REGION,PERIOD,ECOSYSTEM,ANIMAL) Wildlife Population Accounting
Equation
* HABITAT0_EQU(REGION,PERIOD,ECOSYSTEM,ANIMAL) Habitat Requirement Equation
* HABITAT1_EQU(REGION,PERIOD,ECOSYSTEM,ANIMAL) Habitat Requirement Equation
•
•
•
•
•
•
•
•
•
* Production Equations: Jointly model agricultural and forestry products
HARVESTINERTIA_EQU(REGION,PERIOD) Harvest Inertia Equation
MINFEED_EQU(REGION,PERIOD,ANIMAL,MINNUTRIENT) Minimum Feed Intake
MAXFEED_EQU(REGION,PERIOD,ANIMAL,MAXNUTRIENT) Maximum Feed Intake
CROPMIX_EQU(REGION,PERIOD,SPECIES) Crop Mix Restrictions
TOTALCROPMIX_EQU(REGION,PERIOD) Crop Mix Identity
PRODUCTMIX_EQU(REGION,PERIOD,ALLPRODUCT) Product Mix Restrictions
ANIMALMIX_EQU(REGION,PERIOD,SPECIES) Animal Mix Restrictions
* PLANTCAP_EQU(REGION,PERIOD,COHORT,PROCTECH)
III Analyzing FASOM
Why Large Models?
• Land use is diverse and globally linked
• We want both high resolution and large
scope
• More computer power tempts larger
models
• Data availability better
Large Model Effects
• Indexed data, variables, and equations
• Dimensions need to be carefully
conditioned
• More things can go wrong
• Less intuition in model drivers
• Causes of misbehavior difficult to guess
• Higher probability that some errors are not
discovered
Pre-Solution Analysis
• Generic variable and equation checks
• About 30 different types
• Easy in GAMS through use of GAMSCHK,
possible with other software
Example
• Nonnegative Variable Xj occurs only in <=
constraints
• All aij coefficients are nonnegative
• All objective function coefficients (cj) are
nonpositive
•  Optimal Xj = 0! (Maximization problem)
More Generic Checks
Generic Errors
Caused by:
• Missing/misspecified data
• Missing/misspecified conditions
• Missing/misspecified equations
Correction (rule of thumb):
• All individual variables need the same
condition in all individual equations
Generic Checks
• Large models frequently reveal unchecked
generic errors
• Errors may or may not impact optimal
solution
• Correction always increases model speed
• Detection and correction of generic errors
also applicable to non-linear models
Post-Solution Analysis
of “Misbehaving” Solutions
Infeasible equations
 empty solution
Unbounded variables
 objective reaches infinity
Non-sensible solutions
Infeasibility
•
•
•
•
Very easy to detect
Easy to fix
Use artificial variables
Analyze equations with non-zero artificial
variables
• Look for constraints with high shadow
prices
Unbounded Solution
•
•
•
•
•
Very easy to detect
Easy to fix
Add artificial bounds
Look for high variable levels
Examine equations coefficients of
unbounded variables
Non-sensible solution values
•
•
•
•
•
•
Detection requires context knowledge
Problem analysis more challenging
Large deviations easier
Must employ post optimality checks
Start at the symptom
Go backwards alternating between
variables and equations
Linear Program Duality
Max
c X
a X
j
j

Z
ij
j

bi
for all i
Xj

0
for all j
U b
U a

Z
j
s.t.
j
Min
i
i
i
s.t.
i ij
i
Ui
 cj
for all j

for all i
0
Reduced Cost
Z
i a ij  Ui  c j   X
j
Shadow prices
Technical
Coefficients
Objective
Function
Coefficients
Complementary Slackness
Opt. Slack Variable Level
Shadow Price
*

 0
 U A  C X
*
 0
U
*'
 b  AX
*'
Reduced Cost
Opt. Variable Level
Reduced Cost Equality
• Links variables with individual equations
• Positive aij*Ui  costs (Maximization)
• Negative aij*Ui  benefits (Maximization)
Ex: Reduced Cost Computation
##
CROP_VAR(Alabama,Corn,DryLand,BASE,W3-8Land,Vent,NONE,ND,Nbase)
SOLUTION VALUE
7.22224
UPPER BOUND
EQN
OBJT
ProductBal(Corn,Southeast)
0.100000E+10
Aij
Ui
Aij*Ui
134.90
1.0000
134.90
-108.41
2.5413
-275.50
MAXLAND(Alabama,W3-8Land)
1.0000
122.45
122.45
LABOR(Southeast)
2.4700
2.2800
5.6316
MIXREG(Corn,Alabama,All)
1.0000
28.496
28.496
MIXREGTOT(Alabama,All)
1.0000
-15.973
-15.973
EMITACCT(Southeast,FuelCarbon)
-0.35077E-01
0.0000
0.0000
EMITACCT(Southeast,DrygCarbon)
-0.44493E-01
0.0000
0.0000
EMITACCT(Southeast,FertCarbon)
-0.95131E-01
0.0000
0.0000
EMITACCT(Southeast,PestCarbon)
-0.19033E-02
0.0000
0.0000
EMITACCT(Southeast,SoilCarbon)
-0.32381E-01
0.0000
0.0000
EMITACCT(Southeast,N2ODeNit)
-0.61070E-05
0.0000
0.0000
EMITACCT(Southeast,N2OFert)
-0.96429E-03
0.0000
0.0000
TRUE REDUCED COST
0.0000
Row Summing
##
FEEDINGBAL_EQU(Pacific,turkeypro0)
VAR
LVSTBUDGET(CaliforniN,turkey,0,BASE)
LVSTBUDGET(CaliforniS,turkey,0,BASE)
LVSTBUDGET(Oregon,turkey,0,BASE)
REGPROCESS(Pacific,trkpromix4)
=L=
RHS COEFF
SLACK EQUALS
SHADOW PRICE
Aij
Xj
Aij*Xj
0.10000E-01 0.60560E+06 6056.0
0.10000E-01 25233.
252.33
0.10000E-01 34428.
344.28
-1.0000
6652.6
-6652.6
=L=
0.0000
0.0000
0.86001
Fixing Non-sensible Models
• Zero/large variables: look at cost and
benefits of these variables in individual
equations
• High shadow prices: indicate resource
scarcity (check endowments, technical
coefficients, units)
• In general, analyst needs a combination of
mathematical and context knowledge
Conclusions
• Large mathematical programming models
are not necessarily black boxes
• Drivers for individual results can be traced
and understood
• Generic misspecifications can and should
always be corrected
• Systematic post-optimality analysis is by
far better and faster than intuition and
guesswork
IV Modifying FASOM
Watch out
• Unit consistency (5 year)
• Documentation