Transcript ppt

An Example of radar DSP Product Processing
Newfel Mazari
Hongjie Xie
What is Geoprocessing ?
Why Geoprocessing ?
 Geoprocessing tasks can be time intensive since they are
often performed on a number of different datasets or on
large datasets with numerous records.
 Using ArcGis Tools to process GIS Data




ArcMap
ArcCatalog
ArcInfo
Scripting
Geoprocessing and Scripting
 Scripting is an efficient method of automating
geoprocessing tasks.
 Scripting allows the execution of simple processes (a single
tool) or complex processes (piggybacked, multitool tasks
with validation).
 Scripts are recyclable, meaning they can be data
nonspecific and used again.
 Script: is a set of computing instructions, usually
stored in a file and interpreted at run time.
Tools in Arcgis
 All commands (otherwise known as tools) are
maintained in toolsets within the ArcGIS toolboxes.
 A toolbox can contain tools, toolsets, and scripts and is
organized according to the collection of geoprocessing
commands it contains.
 A toolset can contain tools, toolsets, and scripts and is
organized according to the geoprocessing commands it
contains.
Types of Tools in Arcgis
 System tool
 Model Tool
 Script Tool
 Custom Tool
System Tools
Usually, these tools are installed
and registered when you install
ArcGIS.
Although third-party developers
can also create and register
system tools.
System tools are sometimes called
function tools by developers.
Model Tools
These tools are created by user with Model Builder. Some of
the tools in the system toolboxes are model tools.
Script Tools
Created by user with a scripting language editor (typically an
enhanced text editor).
Some of the tools in the system toolboxes are script tools.
Custom tools
 Built by system developers and have
their own unique user interface for
creating the tool.
 The ArcGIS Data Interoperability
extension contains custom tools.
Scripting for ArcGis
 ArcGIS 9 includes scripting support for many of
today's most popular scripting environments, such as:




Python,
VBScript,
JScript,
Perl.
Python
 Open Source
 Object Oriented
 Multi-Platform
 Good Debugging Tools
 Installed with ArcGis
 ESRI samples provided
Python
• Python is simple to learn because of its clean syntax and simple, clear
concepts.
• Python supports object-oriented programming in an easy-to-understand
manner.
• Documenting Python is easy because it has readable code.
• Complicated data structures are straightforward to work with in Python.
• Python is simple to integrate with C++ and Fortran.
• Python can be seamlessly integrated with Java.
• Python is free from the Web and has a widespread community.
Geoprocessing Concepts for Arcgis
 Object: almost anything can be an “OBJECT”
 Attributes
 Methods
 Geoprocessor: component of ArcObjects
 manages all the geoprocessing functions available within ArcGIS.
 It is an object that provides a single access point and environment
for the execution of any geoprocessing tool in ArcGIS, including
extensions.

Using the native arcgis scripting module, depending on
version of software version you have.
Resources for Geoprocessing with Pyhton
 Arcgis Desktop Help
 Arcgis Libraries
 www.python.org
How to start: Basic Concepts
 Define you geoprocessing problem
 Complex Problem
“Divide and Conquer”
 Write and draw a chart of required steps
 Conceptualize the problems and solutions
 Look at ArcGis Desktop Help
 Find tools in ArcGis
Concepts Continued
 The easiest way to start is to create a model, and export
it to a python script.
 Look carefully at syntaxes
Concepts again
 Each Tool has a scripting help associated with it
Tools Aliases
An Example of Geoprocessing
An Example: DSP data processing
Radar Cells (grids)
DSP Example
DSP Example: Radar Cells and 50
Gages
Conceptualize the Problem
Extract radar Values According to Rain Gauge Locations
 What are the available tools “solutions” within ArcGis
Radar Cells: Polygon Feature Class
Gages: Point Feature Class
Shapefiles
Conceptualize the Problem
Gages have
- Ids
- Coordinates
Radar Attributes
Hundreds of files similar to this one
Records from two radars
KEWX
KDFX
What are the Tools we can USE ?
 Analysis
 Cartography
 Conversion
 Management
 Geostatistical Analyst
 Spatial Statistics
Tools We can Use
 To make sure that both radar shape file and
gages shape file are on the same Coordinate
System
Tools We can Use
Define Projection Tool
Intersect Tool
Export Feature Attributes to ASCII
First Step: Create Your own ToolBox
Add Necessary tools to your own Toolbox
Second Step: Create a Model
My Model
Activate the Model
Test Run the Model
Check the results
Open the ASCII file
Strategy for Work
 Divide to 3 Steps

Define Projection

Intersect

Export Attributes to ASCII file
Export to Script: Define Projection
Export to Script: Intersect
Export to Script: Export Attributes
to ASCII
Syntax
Arguments
Lets Look at the one script: Define Projection
Script Title and General
Information
# ----------------------------------------------------------------# DefineProjection.py
# Created on: Thu Apr 02 2009 04:48:01 PM
# (generated by ArcGIS/ModelBuilder)
# -----------------------------------------------------------------With PythonWin 2.5 IDLE
Comments are shown in
red color
IDLE: Interactive Development Environment
Import Necessary Modules
Comments Line
################ Imports ##################
import arcgisscripting as ARC
import sys as SYS
import os as OS
import locale as LOCALE
ArcObjects
Operating System
File and Folders Path
The locale module opens access to
the POSIX locale database and
functionality
Create the Geoprocessor Object
# Create the Geoprocessor object
GP =ARC.create(9.3)
Load Required Toolboxes
## Load required toolboxes...
GP.toolbox = "management"
# Load required toolboxes...
GP.AddToolbox( "C:/Program Files/ArcGIS/ArcToolbox/Toolboxes/
Data Management Tools.tbx" )
Script Arguments and Workspace
# script arguments
inputWS = "C:\\testFolder\\test1"
outputWS = "C:\\testFolder\\test2"
# set the workspace
GP.workspace = "C:\\testFolder\\test1“
Set the Script to go through a list of
files
# Get a list of feature classes in the workspace.
fcs = GP.ListFeatureClasses()
List all files in the worksapce folder
for fc in fcs:
# Process: Define Projection...
………………Statement for Tool
……………….Line 2 of statements
……………….Line 3
Define the Tool Arguments and parameters
# Process: Define Projection...
#coordsys = "Coordinate Systems/Geographic Coordinate Systems/North
America/North American Datum 1983.prj“
coordsys = "Coordinate Systems/Geographic Coordinate
Systems/World/WGS 1984.prj “
#GP.DefineProjection_management(fcs,"GEOGCS['GCS_WGS_184',DATUM['D_
WGS_1984',SPHEROID['WGS_1984',6378137.0,298.257223563]],PRIMEM['Green
wich',0.0],UNIT['Degree',0.0174532925199433]]")
GP.defineprojection(fc, coordsys)
Add Debugging Messages
# Get the resulting messages and print them
print fcs
print GP.GetMessages()
It is important to print error messages and
processes for DEBUGGING purposes
Same Procedure for the Intersect
Script
# Load required toolboxes...
gp.AddToolbox("C:/Program
Files/ArcGis/ArcToolbox/Toolboxes/Analysis Tools.tbx")
# Script Arguments
inputWS = "C:\\testFolder\\test1"
precip_gages_wgs84 = "C:\\testFolder\\test2\\precip_gages_wgs84.shp"
outputWS = "C:\\testFolder\\test3"
# Set the workspace
gp.workspace = "C:\\testFolder\\test1"
Intersect Script: Continued
for fc in fcs:
# set the output name for each feature class to be the same
as the input
output = outputWS + "\\" + "kewx_" + fc
# for each file in the list run the Intersect
gp.Intersect_analysis(fc + ";"+ precip_gages_wgs84, output,
"ALL", "", "INPUT" )
#get the resulting message and print them
print gp.GetMessages()
Dealing with files in Python
 Python has built in function to work with files;
 List of files in a folder
 Open and read files


Read all file at once
Read line by line
 Write to files
 Read lines, search inside lines
 Read bytes..
 Create files for reading and/or writing
Searching for files in folders
 Search for files and get a list.
# import required python modules
import sys, string, os, shutil
# Define the path of the serch directory or folder
InputFolder = "C:\\testGP\\test1\\"
OutputFolder = "C:\\testGP\\test3\\“
ListFiles = open(OutputFolder + "\\" + “MyList.txt", "w")
listfiles = os.listdir(InputFolder)
for fileNames in listfiles:
print fileNames
MyList.write(fileNames + "\n")
print finished listing files
Reading and Writing in files
 Create a new file:
open("C:\\testGP\\output.txt", "w")
 Read entire file into a single string: S = input.read()
 Read next line:
S = input.readline()
 Read entire file into list of line strings:
L = input.readlines()
 Write string S into file:
output .write(S)
 Write all line strings L into file: output.writelines(L)
 Manual close of file: output .close()
Export Attributes to ASCII Script
# Load required toolboxes...
gp.AddToolbox( "C:/Program Files/ArcGis/ArcToolbox/Toolboxes/Analysis
Tools.tbx“ )
gp.AddToolbox( "C:/Program Files/ArcGis/ArcToolbox/Toolboxes/Spatial
Statistics Tools.tbx“ )
gp.AddToolbox("C:/Program Files/ArcGis/ArcToolbox/Toolboxes/Data
Management Tools.tbx" )
# Script Arguments
outputWS = "C:\\testFolder\\test4"
inputWS = "C:\\testFolder\\test3"
outputWS = "C:\\testFolder\\test4"
outputASC = "C:\\testFolder\\test4“
# Set the workspace
gp.workspace = "C:\\testFolder\\test3"
Export Attributes Continued
# create a text file for combining and writing all outputs
file1 = open( "C:\\testFolder\\test4\\kewxradar.txt", "w“ )
# set file2 as the feature class attributes exported to text file
# create text files for all outputs with Zero counts and Rain Counts
file3 = open( "C:\\testFolder\\test4\\ZeroCountKewx.txt", "w“ )
file4 = open( "C:\\testFolder\\test4\\RainCountKewx.txt", "w“ )
for fc in fcs:
# set the output name for each feature class to be the same as the input
output = outputWS + "\\" + fc[0:18] + ".txt"
# get the count of each intersection file if count>0 then export to ASCII
count =gp.GetCount_management(fc)
Attributes Export: Setting the files names,
Outputs, Tool parameter…………
if count >=1:
# set the timestamp for each file
timestamp = fc + " "
print output + " The count is = " + count
# write the timestamp to a text file
file1.write(timestamp )
#Process: Export Feature Attribute to ASCII...
gp.ExportXYv_stats(fc, "value", "Space", output)
#open the ASCII files, read and write its contents to another text file
file2 = open (output,"r" )
text = file2.read()
file1.write( " " + text )
file2.close()
else:
print output + "count for rainfall radar cells value = " + count + "nothing to
export“ )
print " the script will proceed to the next file "
file3.write( output + "\ t" + "Count = 0" + "\n " )
Export Attributes Continued
else:
print output + "count for rainfall radar cells value = " +
count + "nothing to export“ )
print " the script will proceed to the next file "
file3.write( output + " \ t" + "Count = 0 " + " \ n " )
file1.close()
print gp.GetMessages()
Test your Script… Test again
 Python is case sensitive for variable names and strings
 Indentation is a very sensitive issue in Python
 Note the difference in the use of forward slash “ \” for
system paths
 Double Forward Slashes “ \\ “ for workspaces paths

Test Again, run and SAVE your Module
So Let’s give it a try!