Transcript ArcGIS Geoprocessing - Advanced Scripting

ArcGIS Geoprocessing
Advanced Scripting With Python
Corey Tucker
Nathan Warmerdam
Workshop Outline
• Review of scripting languages
– What is supported? What is recommended? Why?
• Data Access with cursors
– Review types and options
– Working with feature geometry
• Creating and using objects for parameter values
• How to create a script tool
• Making scripts more efficient
– Use of layers and table views
– Calling models
• Effective Error Handling
• Summary
Scripting Languages
• Many to choose from for geoprocessing tasks
–
–
–
–
–
Python
VBScript
JScript
Perl
Others
• Any language that supports COM IDispatch may
be used
• Python is the most commonly used by our users
and is what is most commonly documented
Scripting Languages
Python
• Why is Python recommended by ESRI
–
–
–
–
–
–
–
–
It is free!
Established user community
Simple when it needs to be
Easy to maintain
Scalable
Cross platform (windows, UNIX, Linux)
Many IDE’s to choose from
Supports object-oriented and modular designs
Scripting Languages
Python
• What version is supported?
– ArcGIS installs version 2.1, but all versions from 2.1
up to 2.4.1 are supported
• Why 2.1?
– Due to installation problems of PythonWin for later
versions of Python, ESRI had to ship version 2.1.
• How can I upgrade my version of Python
– Download install for 2.4.1
– Download PythonWin for 2.4.1
– Read Knowledge Base document #26872 from
support.esri.com
ArcGIS 9.2
Bulletin
• A native Python module for ArcGIS will be
included
–
–
–
–
Based on version 2.4.1
Eliminates need for PythonWin
Allows better error messaging
Cross platform support
• Can be used to run scripts on Unix and Linux that have a 9.2
ArcGIS Engine installation
– 9.0 and 9.1 scripts will just work. Older Dispatch
methodology will still be supported.
Scripting Documentation
and Samples
• Writing Geoprocessing Scripts with ArcGIS
– PDF in Documentation Library
– Explanation of methods and samples
• ArcGIS Online Help
– Same content as PDF
– Specific help and samples for all of the
geoprocessor’s methods and properties
• Geoprocessing Programming Guide
– Logical layout of scripting objects with their methods
and properties
– Quick Reference
Section 1: Cursors
Accessing Data
with Cursors
• Geoprocessing workflows commonly require record by record
access of field values
– Including the geometry field
• This is done using an object that points to a record in a table
or feature class called a cursor
• Examples
– Using a field value in an logical expression
if row.soiltype == “loam”:
row.group = 2
– Moving values from one table or feature class to another
if row1.parcelid == row2.parcelid:
row1.owner = row2.owner
– Accessing properties of a feature’s geometry
print row.shape.FirstPoint
Accessing Data
with Cursors
• There are three types of cursors
– Search Cursor
• Read-only access
– Update Cursor
• Read/Write/Delete access but no new records
– Insert Cursor
• Read/Write access with capability of creating new records
Accessing Data
with a Search Cursor
• A row object is returned from the search cursor
object
• Fields are accessed as properties of the row
object
• Use the row object’s GetValue and SetValue
methods if your field name is a variable
• Destroy the row and cursor objects to remove
read locks on the data source
Accessing Data
with a Search Cursor
from win32com.client import Dispatch
gp = Dispatch("esriGeoprocessing.gpDispatch.1")
rows = gp.SearchCursor("D:/St_Johns/data.mdb/roads")
row = rows.Next()
# Print concatenated values of road name and road type
while row:
print row.name + row.GetValue("type")
row = rows.Next()
# Delete the row and cursor objects so no locks remain
del row
del rows
Accessing Data
with Cursors
• A where clause may be used to limit the records
returned by the cursor
– Same as defining a definition query on a layer
from win32com.client import Dispatch
gp = Dispatch("esriGeoprocessing.gpDispatch.1")
rows = gp.SearchCursor("D:/St_Johns/data.mdb/roads",
“[neighborhood] = "Shea_Heights")
row = rows.Next()
# Print concatenated values of road name and road type
while row:
print row.name + row.GetValue("type")
row = rows.Next()
del row
del rows
Reading Feature
Geometry
• You must understand the hierarchy for geometry in order
to use it
– A feature class is made of features
– A feature is made of parts
– A part is made of points
• In Python terms
– A single part feature looks like this
[pnt, pnt, pnt]
– A multipart polygon feature looks like this
[[pnt, pnt, pnt],[pnt, pnt, pnt]]
– A single part polygon feature with a hole (inner ring) looks like
[[pnt, pnt, pnt, ,pnt, pnt, pnt]]
Reading Feature
Geometry
• A null point is used as a separator between rings (holes)
in a polygon part
• Use the PartCount property to get the number of parts
for a feature
• Use the GetPart method to retrieve the desired part
# Reading lines
x = 0
while x < feat.PartCount:
roadArray = feat.GetPart(x)
pnt = roadArray.Next()
while pnt:
print pnt.x + "," + pnt.y
pnt = RoadArray.Next()
x = x + 1
Reading Feature
Geometry
• Features may be projected on-the-fly to another
coordinate system using the Spatial Reference
parameter
• Create the Spatial Reference by describing another
dataset
# Describe features class with a GCS
desc = gp.Describe("D:/data.mdb/geo_roads")
# Create search cursor, using GCS spatial reference
rows = GP.SearchCursor("D:/data.mdb/roads","",
desc.SpatialReference)
row = rows.Next()
Demo 1 – Reading Feature Geometry
Reading Feature
Geometry - Tips
• Use field values instead of the properties of the
geometry object if speed is really important
– Length
– Area
• If you want the extent or centroid of a complex
feature try creating simple features in a new
feature class
– Features to Point
– Feature Envelope to Polygon
• Using fields or simple geometry is faster that
querying the geometry object of a complex
feature
Writing Feature
Geometry
• Insert cursors must be used to create new
features
rows = GP.InsertCursor("D:/data.mdb/roads“)
row = rows.NewRow()
• Use the Point and Array objects to create
feature parts
• A part may be used to set a geometry field
– A multipart feature is an array containing other
arrays, where each array is a part
Writing Feature
Geometry
# Open an insert cursor for the feature class
cur = GP.InsertCursor(fcname)
# Create the array and point objects needed to create a feature
lineArray = GP.CreateObject("Array")
pnt = GP.CreateObject("Point")
# Add two points to the array
pnt.x = 358331
pnt.y = 5273193
lineArray.Add(pnt)
pnt.x = 358337
pnt.y = 5272830
lineArray.Add(pnt)
# Create a new row, or feature, in the feature class
feat = cur.NewRow()
# Set the geometry of the new feature to the array of points
feat.shape = lineArray
# Insert the feature
cur.InsertRow(feat)
Writing Feature
Geometry
• The geoprocessor checks the validity of the
geometry before it is inserted
– Problems such as invalid ring order or ring orientation
are corrected automatically
– Uses the same process found in the Check and
Repair Geometry tools
• Writing features requires an exclusive schema
lock on the feature class
– Use the TestSchemaLock method to see if the data is
being used by another process
ArcGIS 9.2
Bulletin
• Cursor performance will be FASTER!
– Much faster for Update operations
• New option to limit the returned fields on a row
• New option to sort the rows on a field or set of
fields
• 9.0 and 9.1 scripts will just work
• Speed enhancements will be included in Service
Pack 1 for ArcGIS 9.1
Section 2: Complex Parameters
Tools and Parameters
• Each geoprocessing tool has a collection of
parameter values
• Parameters define how the tool will work
Using Objects for
Parameters
• Most tool parameters are easy to define using a
string or a number, such as a data path or buffer
distance
• Some parameters are more complex and have
complicated text string representations,
– spatial reference or field mapping
• Instead of using text strings, use objects
Using Objects for
Parameters
• Objects either created directly by
– The geoprocessor’s CreateObject method
– Another object, such as the descriptive object created
by the Describe method
– A tool, such as Create Spatial Reference
Using Objects for
Parameters
• Creating a geodatabase feature dataset
from win32com.client import Dispatch
gp = Dispatch("esriGeoprocessing.GPDispatch.1")
# Describe a dataset that has the desired SR
desc = gp.Describe("D:/St_johns/city.mdb/city_boundary")
# Create the FDS using the describe object's SR Object
gp.CreateFeatureDataset("D:/St_johns/city.mdb“,"NewFDS",
desc.SpatialReference)
SR = Spatial Reference
Using Objects for
Parameters
• Value Tables
– Used to define any tool parameter that accepts more
than one value (e.g. Merge)
– Multivalue parameters may be expressed as
• A string ("value1;value2;value3")
• As a Value Table
– Use LoadFromString method to load a multivalue
strting that may be passed as a script argument
– Use ExportFromString to create an output argument
value
• Useful for script tools
Using Objects for
Parameters
• Overlay tools, such as Union and Intersect use
Value Tables with multiple columns
– Difficult to parse multivalue string when more than
one column is used
("D:/data/new roads" 1;D:/data/soils 2)
• Value Tables with multiple columns are easy to
navigate
Section 3: Script Tools
Creating Tools
from Scripts
• Why?
– The script is generic and can be used with other data
• Script can use arguments from the user
– You want to use a script in ModelBuilder
• Incorporate another system with a script wrapper or do
branching
– You want to easily share your script
• Not everyone knows how to run a stand-alone script
• Puts a familiar face on your work
Creating Tools
from Scripts
• Step 1: Create argument variables
– Can use sys.argv[ ], but this is subject to windows
limitations (1024 characters)
– Should use GetParameterAsText, especially with
multivalue parameters
Creating Tools
from Scripts
• Step 2: Messaging
– Return informative messages during execution of the
script
– Return error messages when a problem arises
Creating Tools
from Scripts
• Step 3: Add the script to a toolbox
– Give the tool a name, label and description
– Set the tool source and use relative paths option if
you plan on sharing the tool
– Define the parameters that will correspond to your
script
Creating Tools
from Scripts
• Step 3: Defining Parameters
– Parameters have several properties
•
•
•
•
•
•
•
Name: What you see on the dialog and on the command line.
Type: Is it required, optional or derived?
Direction: Is the data being used (input) or created (output)?
Multivalue: Do you want a list of values or just one?
Default: Is there a default value?
Environment: Does an environment provide a default value?
Domain: Do you want to provide a choice or limit input
values?
• Dependency: Does this parameter depend on another?
Creating Tools
from Scripts
• Step 3: Derived Parameters
– All tools that will be used in ModelBuilder should have
an output
– If the script updates an input dataset, create a derived
parameter and set its dependency to the input
parameter
Value
Creating Tools
from Scripts
• Step 3: Parameter Dependency
– Some parameter types have built-in behavior when
there is a parameter dependency
• Fields with an Input table or feature class
– Fields will be populated automatically in the dialog
• Derived parameter with an input parameter
– The derived parameter value will automatically be set to the
value of the input parameter it depends upon
Demo 2: Creating A Script Tool
Creating Tools
from Scripts
• AML is a supported source for script tools
– Need to change the default behavior of an AML to
support this
• By default it must execute with arguments when opened
• Use the RegisterAMLAsExecutable registry file
to make this change
– File is in the ArcGIS\ArcToolbox\Scripts folder
• Any ArcInfo workstation module is supported
– AML must start in the Arc module and then call other
modules
– Display windows and form menus are supported
Section 3: Tips for Efficiency
Efficiency Tips
• When working with selections, use layers and
views instead of using the Select tool
– Layers and views are in memory and avoid costs of
writing to disk
• Use Make Feature Layer, Make Raster Layer or
Make Table View
• Use Select By Attribute to update the selection
or provide a where clause when creating the
layer or view
Efficiency Tips
• Call models from your scripts
– Instead of rewriting a model as a script, reference the
model’s toolbox and call it as a tool
– Simplifies code
from win32com.client import Dispatch
gp = Dispatch("esriGeoprocessing.gpDispatch.1")
# Add custom toolbox
gp.AddToolbox("D:/Data/St_Johns/MyTools.tbx")
gp.MyRoadsModel("D:/Data/St_Johns/urban_roads",100,
"C:/temp/results")
Efficiency Tips
• Use Python Lists and Dictionaries for frequent
access of attributes
– Instead of reading through a table multiple times with
a cursor, read it once and load the values into
memory
– Use a list for an ordered collection of values
– Use a dictionary if you have an unordered collection
of values and want to access values using a key
• Great for working with coordinates
– Many examples in the source scripts of the Spatial
Statistics tools
Section 4: Error Handling
Error Handling
• The Geoprocessor is just another COM Dispatch
object to Python, or any other language
– Error messages are less than helpful
• Tool messages must be retrieved from the
geoprocessor
• Scripts must add error handling routines to catch
errors and return tool messages
Error Handling
• Non-tool methods on the Geoprocessor do not
return error messages
– Cursors, Creating Lists, Describe, etc
• Use the Raise statement to trigger specific
exceptions
– Especially useful when using tools and native
methods together
– Provides more meaningful context for messages
Error Handling
try:
# Get the input feature class and make sure it contains polygons
input = sys.argv[1]
dsc = gp.Describe(input)
if dsc.ShapeType != "polygon": raise "ShapeError"
# Get the new field name
fieldname = sys.argv[2]
# Make sure shape_length and shape_area fields exist
if gp.ListFields(input,"Shape_area").Next() and \
gp.ListFields(input,"Shape_length").Next():
# Add the new field and calculate the value
gp.AddField(input, fieldname, "double")
gp.CalculateField(input,fieldname,
"[Shape_area] / [Shape_length]")
else:
raise "FieldError"
except "ShapeError":
print "Input does not contain polygons"
except "FieldError":
print "Input does not shape area and length fields"
except:
print gp.GetMessages(2)
Error Handling
• Use the sys module to access python syntax
errors
– Useful when using exec or eval functions
– Must import the sys module
except:
msgs = gp.GetMessages()
if msgs:
AddMsgAndPrint(msgs,2)
value = sys.exc_info()[1]
AddMsgAndPrint(value.msg + ": " + value.text ,2)
Demo 3: Error Handling
Additional support
• For additional information about geoprocessing including service
pack information, knowledge base articles, and recent information
about issues go to:
http://support.esri.com/geoprocessing
Additional technical sessions
• ArcGIS Geoprocessing: Introduction to ModelBuilder
Wed 3:30 PM Room: 6E
• ArcGIS Geoprocessing: Advanced ModelBuilder
Thu 8:30 AM
• ArcGIS Geoprocessing: An Introduction
Room: 6E
Wed 1:30 PM Room: 6E
• ArcGIS Geoprocessing: Developing Geoprocessing
tools with ArcObjects
Thu 10:30 AM Room: 6E
• ArcGIS Geoprocessing: Using Spatial Statistics in
ArcGIS
Thu 1:30 PM
Room: 8
Thu 3:30 PM
Room: 5-B
• ArcGIS Extension: 3D Analyst Geoprocessing Tools
Island demo’s & Demo Theatre sessions
Island Demo’s
• Working with Layers and Table Views in Geoprocessing
Wed 4:00 -4:30pm
• How to convert pGDB feature classes to .shp in a model
Thu 11:30 – 12noon
Demo theatre sessions
•
How to use the Geoprocessing Environment Settings
T1 - Tue 10:30 – 11:15am
T1 - Wed 1:30 – 2:15pm
•
Using Spatial Statistics for Crime Analysis
T1 - Thu 9 -10am
•
Geoprocessing Tool Documentation and Metadata
T1 - Thu 11:15 - 12noon
Thank you!
Please fill out your survey form
Questions?