Transcript - LearnGroup

Lecture 9: Geoprocessing with
Python
Dr. Taysir Hassan Abdel Hamid
Associate Professor,
Information Systems Dept.,
Faculty of Computers and Information
Assiut University
April 17, 2016
INF424: GIS. Dr. Taysir Hassan A. Soliman
Outline
•
•
•
•
•
More on Strings
File I\O
ArcPy essential Vocabulary
ArcPy Modules
Arcpy.mapping
INF424: GIS. Dr. Taysir Hassan A. Soliman
INF424: GIS. Dr. Taysir Hassan A. Soliman
Strings Methods
INF424: GIS. Dr. Taysir Hassan A. Soliman
File I\O
•
•
•
•
Often read/write data from files
Open / close files
Read data from a file
Writing data from a file
INF424: GIS. Dr. Taysir Hassan A. Soliman
• f.close()
INF424: GIS. Dr. Taysir Hassan A. Soliman
GeoProcessing with Python
1. Every script using Arc functions must start
with import arcpy
• import arcpy
2. Get user inputs with ‘GetParameterAsText’
'''script.py
1
'''
2
import arcpy
3
4
# get input feature class
5
inputFeatureClass =
6
arcpy.GetParameterAsText(0)
This gets the first user entered parameter as text.
INF424: GIS. Dr. Taysir Hassan A. Soliman
INF424: GIS. Dr. Taysir Hassan A. Soliman
Essential ArcPy Vocabulary
Arcpy Classes
Arcpy
Functions
Arcpy
Modules
StandAlone
Python Scripts
Python
Window
Python Script
tool
Python
Toolbox
INF424: GIS. Dr. Taysir Hassan A. Soliman
What is Arcpy?
• ArcPy is a Python site package that provides a useful
and productive way to perform geographic data
analysis, data conversion, data management, and
map automation with Python.
• ArcGIS applications and scripts written using ArcPy
benefit from being able to access and work with the
numerous Python modules developed by GIS
professionals and programmers from many different
disciplines.
INF424: GIS. Dr. Taysir Hassan A. Soliman
• ArcPy provides access to geoprocessing tools as well as
additional functions, classes, and modules that allow you to
create simple or complex workflows.
• Geoprocessing tools are functions available from arcpy—that
is, they are accessed like any other Python function.
• However, to avoid confusion, a distinction is always made
between tool and nontool functions (such as utility functions
like ListFeatureClasses()).
INF424: GIS. Dr. Taysir Hassan A. Soliman
• Tools are documented differently than
functions.
• Every tool has its own tool reference page in
the help system. Functions are documented in
the ArcPy documentation.
• Tools return a Result object; functions do not.
• Tools produce messages, accessed through a
variety of functions such as GetMessages().
Functions do not produce messages.
INF424: GIS. Dr. Taysir Hassan A. Soliman
Python and Arcpy
• Python is an open-source programming
language that was conceived in the late 1980s
by Guido van Rossum and introduced in 1991.
Python is supported by a growing and varied
user community and provides easy readability,
clean syntax, dynamic typing, and an
extensive collection of standard and thirdparty libraries.
INF424: GIS. Dr. Taysir Hassan A. Soliman
Arcpy
• ArcPy (often referred to as the ArcPy site
package) provides Python access for all
geoprocessing tools, including extensions, as
well as a wide variety of useful functions and
classes for working with and interrogating GIS
data.
• Using Python and ArcPy, you can develop an
infinite number of useful programs that
operate on geographic data.
INF424: GIS. Dr. Taysir Hassan A. Soliman
• This next example shows how to execute
the Buffer tool. When run in the Python
window, the code will be transferred into the
transcript section of the window along with
the result.
• arcpy.Buffer_analysis("c:/data/Portland.gdb/st
reets", "c:/data/Portland.gdb/steets_buffer",
"500 METERS")
INF424: GIS. Dr. Taysir Hassan A. Soliman
Python and ArcMap
• Python is used to automate geoprocessing tools such as the
following which are in the ArcToolbox:
– Analysis, cartography, conversion, data management, editing,
geocoding
• Python also allows more advanced processing such as
looping through records in a database and reading and
writing them
• It also allows manipulating layers in a map
• It allows creating and manipulating geometries (point, line,
polygon)
INF424: GIS. Dr. Taysir Hassan A. Soliman
arcpy
• arcpy was introduced to ArcGIS in its 10th version
– It is downloaded with your ArcGIS!
– You can find it under your ArcGIS folder, e.g. at:
Program Files > ArcGIS >Desktop10.1
• You just need an editor such as PythonWin, IDLE, ArcMap
Python window, or even Calculator field to access arcpy
• If after typing import arcpy in front of the Python prompt
you do not get an error, then you have the arcpy!
>>> import arcpy
# Got no error? Then you can slither!
INF424: GIS. Dr. Taysir Hassan A. Soliman
Arcpy Programming Environments
• Python code for ArcGIS geoprocessing tasks
can be written using many different
development environments
Notepad,
Notepad++,
Wordpad
Text
Editor
Python
Win
IDLE
Third
Party
INF424: GIS. Dr. Taysir Hassan A. Soliman
Visual python,
PyScripter
New Python Window in ArcGIS 10
INF424: GIS. Dr. Taysir Hassan A. Soliman
About Python Window
• New to ArcGIS
• Embedded Interactive Python Window in ArcGIS
• Great for:
–
–
–
–
Testing small blocks of Python code
Learning Python
Building quick and easy workflows in Python
Execution of tools
INF424: GIS. Dr. Taysir Hassan A. Soliman
INF424: GIS. Dr. Taysir Hassan A. Soliman
INF424: GIS. Dr. Taysir Hassan A. Soliman
Python Window
INF424: GIS. Dr. Taysir Hassan A. Soliman
INF424: GIS. Dr. Taysir Hassan A. Soliman
Options at ESRI
Model
Builder
AcrObjects
Python Scripts
INF424: GIS. Dr. Taysir Hassan A. Soliman
Arcpy classes
• A class is analogous to an architectural
blueprint.
• The blueprint provides the framework for how
to create something.
• Classes can be used to create objects, often
referred to as an instance. ArcPy classes, such
as the SpatialReference and Extent classes,
are often used as shortcuts to complete
geoprocessing tool parameters
INF424: GIS. Dr. Taysir Hassan A. Soliman
Arcpy functions
• In ArcPy, all geoprocessing tools are provided
as functions, but not all functions are
geoprocessing tools.
• In addition to tools, ArcPy provides a number
of functions to better support geoprocessing
Python workflows.
INF424: GIS. Dr. Taysir Hassan A. Soliman
Stand-alone Python script
• A stand-alone Python script is a .py file that
can be executed from the operating system
prompt, a Python Integrated Development
Environment (IDE), or by double-clicking
the .py file in Windows Explorer.
INF424: GIS. Dr. Taysir Hassan A. Soliman
Python script tool
• A Python script tool is a Python script that has
been added to a geoprocessing toolbox.
• Once added as a script tool, the script tool
becomes like any other geoprocessing tool—it
can be opened and executed from the tool
dialog box, used in the Python window and
ModelBuilder, and called from other scripts
and script tools.
INF424: GIS. Dr. Taysir Hassan A. Soliman
INF424: GIS. Dr. Taysir Hassan A. Soliman
Example
import arcpy
roads = "c:/base/data.gdb/roads"
output = "c:/base/data.gdb/roads_Buffer"
# Run Buffer using the variables set above and
pass the remaining
# parameters in as strings
arcpy.Buffer_analysis(roads, output,
"distance", "FULL", "ROUND", "NONE")
INF424: GIS. Dr. Taysir Hassan A. Soliman
ArcPy Modules
Data
Access
arcpy.da
Arcpy
Modules
Time
module
Arcpy.time
Mapping
Arcpy.mp
Geostastistical
Analysis
ArcGIS Spatial
Analyst Extension
ArcGIS
Network
Analyst
Extension
Arcpy.ga
Arcpy.sa
Arcpy.na
INF424: GIS. Dr. Taysir Hassan A. Soliman
INF424: GIS. Dr. Taysir Hassan A. Soliman
arcpy Opens Modules
• arcpy can open many modules
– Module: is a self contained collection of functions and
classes that does something
• These modules include:
– Data access module (arcpy.da)
– Mapping module (arcpy.mapping)
– Geostatistical Analyst module (arcpy.ga)
– ArcGIS Spatial Analyst extension module (arcpy.sa)
– ArcGIS Network Analyst extension module (arcpy.na)
INF424: GIS. Dr. Taysir Hassan A. Soliman
INF424: GIS. Dr. Taysir Hassan A. Soliman
INF424: GIS. Dr. Taysir Hassan A. Soliman
INF424: GIS. Dr. Taysir Hassan A. Soliman
…
import arcpy
# imports ArcGIS geoprocessing functionality
import arcpy.mapping # imports only the mapping module
import os
# imports Python’s core operating system
import sys # variables/functions used or maintained by the interpreter
# import env from arcpy and set the workspace environment
from arcpy import env # ability to control ArcGIS environment
env.workspace = “C:\data”
from arcpy.management import *
# Content imported into namespace. Can use content without prefix
INF424: GIS. Dr. Taysir Hassan A. Soliman
Properties and Methods
• Manipulating ArcObjects requires knowing their properties
• Properties/Methods of two ArcObjects are shown below:
Map
Feature Class
Properties
layer count
Name
Spatial reference
Map scale
Extent
Methods
Add layer
Clear Selection
Select feature
Properties
Shape type
Spatial reference
Extent
Methods
Create feature
Remove feature
INF424: GIS. Dr. Taysir Hassan A. Soliman
•
Statements
A statement is a task that does not return a value, e.g., print, import, and
if statements:
import arcpy
print “bye”
# import functionality from ArcGIS Spatial Analyst
import arcpy.sa
from arcpy.sa import *
arcpy.CheckOutExtension (“Spatial”)
# Check the Spatial Analyst License
import arcy.mapping as map
• If statement checks if a condition is true or false
• For loop is another statement, loops through a list
for fc in fclist:
print fc
INF424: GIS. Dr. Taysir Hassan A. Soliman
Environments
• We set the environment for tools to use them
• This includes setting the current workspace,
output spatial reference, extent, raster
analysis setting (cell size, mask)
arcpy.env.workspace --- sets out the workspace
arcpy.env.outputCoordinateSystem --- output
coordinates
arcpy.env.extent
arcpy.env.cellSize
arcpy.evn.mask
INF424: GIS. Dr. Taysir Hassan A. Soliman
arcpy.env.extent
• # Set the extent environment using a keyword.
arcpy.env.extent = "MAXOF"
INF424: GIS. Dr. Taysir Hassan A. Soliman
Example
import arcpy
# Check the current raster cell size and make sure it is a
certain size
# for standard output #
arcpy.env.workspace = "c:/avalon/data"
if arcpy.env.cellSize < 10:
arcpy.env.cellSize = 10
elif arcpy.env.cellSize > 20:
arcpy.env.cellSize = 20
arcpy.HillShade_3d("island_dem", "island_shade", 300)
INF424: GIS. Dr. Taysir Hassan A. Soliman
arcpy.evn.mask
• Tools that honor the Mask environment will
only consider those cells that fall within the
analysis mask in the operation.
• Illustration
INF424: GIS. Dr. Taysir Hassan A. Soliman
INF424: GIS. Dr. Taysir Hassan A. Soliman
Example
# add buffer around the road feature class with given distances
import arcpy
arcpy.env.workspace = ”C:\data\City.gdb” #sets the workspace
fc = “Roads”
#variable feature class
distanceList = [“100 meters”, “200 meters”, 400 meters”] # distances
# loops through each distance in the distanceList
# takes the first distance and puts it in variable dist, and repeats it 3 times
for dist in distanceList:
outName = fc+” _”+ dist
arcpy.Buffer_analysis (fc, outName, dist)
# outputs the feature class, its output name_distance
# breaks out of the loop
print “Buffering completed!”
INF424: GIS. Dr. Taysir Hassan A. Soliman
Another example
•
•
•
•
Clipping a road feature in ArcMap
Click on the Python Window icon in ArcMap
Clear the window from old code, if necessary
Type the following code:
# clip Roads feature class from the Mineral_Spring city polygon,
# output the clipped feature class into RoadsClip
arcpy.clip_analysis (“Roads”, “Mineral_Springs”, “RoadsClip”)
# Run it
# You can see in ArcMap the new added RoadsClip feature class
# which was created and automatically added to the table of content
INF424: GIS. Dr. Taysir Hassan A. Soliman
Tool messages
• When we execute a tool, there might be three types of messages:
– Informative messages
– Warning messages
– Error messages
try:
# start try block
arcpy.Buffer (“C:/ws/roads.shp”, “C:/outws/roads10.shp”, 10)
# print the tool messages
except arcpy.ExecuteError:
print arcpy.GetMessages (2)
# any other error
Except Exception as e:
print e.message
INF424: GIS. Dr. Taysir Hassan A. Soliman
Functions
• Functions perform useful tasks, such as:
– Accessing geoprocessing tool messages (GetMessages)
– Listing data for batch processing, e.g.:
• ListFeatureClasses, ListFields, plus nine other list functions
– Retrieving a dataset’s properties (Describe)
import arcpy
# Set the workspace for ListFeatureClasses function
arcpy.env.workspace = “c:/test”
# For each feature class, create a scratch name and clip
for fc in arcpy.ListFeatureClasses ():
outName = arcpy.CreateScratchName (“clipped_” + fc, “”,
“featureclass”, arcpy.env.workspace)
arcpy.Clip_analysis(fc,
“boundary”, outName)
INF424: GIS. Dr. Taysir Hassan A. Soliman
Dealing with Functions/Methods
• Assigning a value to a property:
# object.property = value
for example:
env.workspace = “C:/Temp”
• Return the value of a property:
# object.property
for example:
print “The workspace is “ + env.workspace
• Use a method:
# object.method (arg1, arg2, …) e.g., put a buffer for a road:
arcpy.Buffer_analysis (“c:/input/roads.tif’,
“c:/output.gdb/buffer_output, 100)
INF424: GIS. Dr. Taysir Hassan A. Soliman
Geometry objects
• We can create, delete, move, and reshape features
# Create a geometry object and put it in the variable g
g = arcpy.Geometry ()
# Run the Copy Features tool. set the output to the geometry object
# Return a list of geometry objects (lines, streets)
geometryList = arcpy/CopyFeatures_management
(“c:/data/streets.shp”, g)
# Loop through each geometry, totaling the lengths of the streets
# for geometry in geometryList:
length += geometry.length
# Note: x +=1 means x=x+1
print “Total length: %f” % length
INF424: GIS. Dr. Taysir Hassan A. Soliman
Mapping module
• arcpy.mapping
– Is used to open and manipulate existing map documents
(.mxd) and layer files (.lyr)
• It is used to query and alter the contents
– Find a layer with data source X and replace with Y
– Updata a layer’s symbology across many MSDs
– Generate a report listing document information
• Data sources, broken layers, spatial reference, etc.
• Can print, export, or save the modified document
• Allows adding, removing, and rotating data frames, and adding
and removing layers
• We can manipulate properties of map documents and layers
• See next slide for an example
INF424: GIS. Dr. Taysir Hassan A. Soliman
Manipulate map documents
# Set data frame properties and export to a TIFF
import arcpy
# Find the Project.mxd map document
mxd = arcpy.mapping.MapDocument (”c:\Project\Project.mxd”)
# make a list of all the data frames in it; iterate through the frames
for df in arcpy.mapping.ListDataFrames (mxd):
df.rotation = 0
# rotate the map
df.scale = 24000
# set the scale
outFile = ”c:\Project\Output\\” + df.name + “.tif”
arcpy.mapping.ExportToTIFF (mxd, outFile, df)
del mxd
INF424: GIS. Dr. Taysir Hassan A. Soliman
Example 2
# Modify map document properties, save changes to a layer file, and
save changes to the map document
import arcpy
mxd = arcpy.mapping.MapDocument (“input.mxd”)
df = arcpy.mapping.ListDataFrames (mxd)
df.scale = 24000
df.rotation = 2.7
for lyr in arcpy.mapping.ListLayers(mxd):
if lyr.name == “Landuse”:
lyr.visible = True
lyr.showLabels = True
lyr.saveACopy(“output.lyr”)
mxd.save()
del mxd
INF424: GIS. Dr. Taysir Hassan A. Soliman
Manipulating Layers
• We can change properties of a layer, such as its:
– name, source data, visibility (make it on or off), transparency,
label, definition query, display order, etc.
import arcpy
# open a file on our hard drive
lyrFile = arcpy.mapping.Layer (”C:\Project\Date\Streets.lyr”)
# we are going to change its name, from streets to highways
for lyr in arcpy.mapping.ListLayers(lyrFile):
if lyr.name.lower() == “highways”:
# turn its label on
lyr.showLabels = True
lyr.saveACopy (r”C:\Project\Data\StreetsWithLabels.lyr”)
# now the changed layer is saved as different layer
del lyrFile
INF424: GIS. Dr. Taysir Hassan A. Soliman
Adding a Python script as a tool
• We can add a script as a tool to a toolbox
• These are called script tools
• They become a new tool with all the properties of a tool, e.g.,
– It will return messages, access to all environment settings,
and automatically add the output to our map (to the table
of contents in ArcMap)
• Can easily be shared, e.g., email it to other users
– they can double click it to run it)
• Tools automatically create dialog boxes (created for us by
ArcGIS)
• We can even add the tool into the toolbar and menus
INF424: GIS. Dr. Taysir Hassan A. Soliman
References
• http://pro.arcgis.com/en/pro-app/toolreference/analysis/clip.htm
• http://pro.arcgis.com/en/proapp/arcpy/geoprocessing_and_python/usingfunctions-in-python.htm
• https://www.eeducation.psu.edu/geog485/node/162
• http://resources.arcgis.com/en/help/main/10.
2/index.html#//002z00000008000000
INF424: GIS. Dr. Taysir Hassan A. Soliman
INF424: GIS. Dr. Taysir Hassan A. Soliman