Transcript Thu G04 1330 Kim - OSGeo Server

The New Zealand Institute for Plant & Food Research Limited
Use of Cloud computing in
impact assessment of
climate change
Kwang Soo Kim and Doug MacKenzie
Outline
• Introduction
• Cloud computing
• Case study:
Calculation of rainfall frequency in the 21st century
• Results
• Conclusions
The New Zealand Institute for Plant & Food Research Limited
Climate Change
The New Zealand Institute for Plant & Food Research Limited
Impact of climate change
• Geographical distribution of species across a wide
range of ecosystems (Walther et al. 2002)
• The timing of blooming for temperate zone species
(Root et al. 2003)
• Crop production in positive or negative ways in
different regions in the 21st century (Rosenzweig et al.
2001).
Simulation models has been used to
assess the impact of Climate change
The New Zealand Institute for Plant & Food Research Limited
Computational scale of impact assessment
study
• The spatial resolution of GCMs has increased
» The typical resolution used in the IPCC TAR was about 250 km
(Houghton et al. 2001).
» In the AR4, many of those models had higher spatial resolution
(Miller et al. 2006).
» NCAR-CCSM3 had spatial resolution of 150 km.
• An ensemble prediction system has been used
» Probabilistic forecasts of climatic events are generated
» Murphy et al. (2004) used a 53-member ensemble of models to
determine the range of climate changes.
• Climate change scenarios
» The Special Report on Emission Scenarios (SRES)
» B1, A1B and A2
The New Zealand Institute for Plant & Food Research Limited
Cloud computing
• A paradigm of computing in which virtualized resources are
provided as a service over the Internet (Gruman & Knorr, 2008)
• Computing resources “As a Service”
»
»
»
»
Infrastructure as a service (IaaS)
Platform as a service (PaaS)
Software as a service (SaaS)
Data storage as a service (dSaaS)
• Utility computing
• Distributed computing
The New Zealand Institute for Plant & Food Research Limited
Amazon web service –
Elastic Compute Cloud (EC2)
• Customers can rent computers on which to run their own
computer applications
• Scalable deployment of applications by creating virtual machines
• A customer can create, launch, and terminate server instances as
needed
• Customers are charged by the hour for active servers
»
»
»
»
»
»
»
Hourly charge per virtual machine ($0.10 to $1.2 per hour)
Data transfer charge ($0.10 to $0.17 per gigabyte)
Allocated and unused Elastic IP address
Storage using Amazon Elastic Block Store (EBS)
Additional transfer charges using Elastic Load Balancing
Using Amazon's CloudWatch service to monitor your virtual machine
Using Amazon's Elastic Load Balancing which distributes load
among selected virtual machine
The New Zealand Institute for Plant & Food Research Limited
Science Clouds
• The Science Clouds project was initiated by the University of
Chicago (UC) and the University of Florida (UFL)
(http://workspace.globus.org/clouds/)
» EC2-style cloud computing
» members of the scientific community to lease resources for short
amounts of time
• The Science Clouds do not require users to directly pay for usage
» Verify the person asking for an allocation is indeed a member of the
scientific community
» Ask for a short writeup of the scientific project.
» Based on the project the individual is allocated a small (testing),
middle (development), or large (science) hour credit on the Science
Clouds.
The New Zealand Institute for Plant & Food Research Limited
Objective
• Calculate monthly rainfall frequency in the 21st century
» Daily precipitation (P) was calculated
P = RF * 86400
» RF = daily rainfall flux
» It was assumed that rainfall occurred on a day when P > 0.254 mm
The New Zealand Institute for Plant & Food Research Limited
Climate projection data
• The daily sets of GCM outputs were obtained from the WCRP
CMIP3 multi-model database (https://esgcet.llnl.gov:8443).
The New Zealand Institute for Plant & Food Research Limited
System architecture
The New Zealand Institute for Plant & Food Research Limited
System configuration
• Amazon web service EC2
» Small instances for Server and Client images
• Server
» Fedora 8
» MySQL database server
» Network file system (NFS) service
• Clients
» Ubuntu 8.10
» A script to download a daily climate change dataset from the internet.
» The data process program
» using NetCDF file format
» search and extract subsets of the original dataset.
The New Zealand Institute for Plant & Food Research Limited
Results
• The running time using 10 client instances was about 32 hr.
• Downloading the climate data
» Between 4.8 hr and 9.1 hr
» Total time was 66 hr
• Database transaction
» Between 15.2 hr and 24.8 hr
» Total time was 209 hr
The New Zealand Institute for Plant & Food Research Limited
Running time
The New Zealand Institute for Plant & Food Research Limited
File size and transfer rate
The New Zealand Institute for Plant & Food Research Limited
Database transaction
The New Zealand Institute for Plant & Food Research Limited
Computation cost (USD)
• CPU:
» Downloading/Processing data: (275 hr + 32 hr) x $ 0.1 = $ 30.7
» Processing/Downloading results: 42 hr x $ 0.1 = $ 4.2
• Transfer:
» 70 GB x $ 0.1 = $ 7
• Storage:
» Climate data storage: 30 GB x $ 0.1 = $ 3
» Database storage: 10 GB x $ 0.1 = $ 1
• Total cost
» $ 46
The New Zealand Institute for Plant & Food Research Limited
Conclusions
• Cloud computing could provide inexpensive and temporary
computing resources to analyse large-scale scientific data for the
climate change impact assessment.
» In a 10 processor-core configuration, our approach would be up to 10
times faster than the calculation on a single processor core machine.
» The costs for processor core use, data transfer and temporary
storage were about $35, $7 and $4, respectively.
• Cloud computing have benefits
» Running time
» Local storage resources
» Network resources.
The New Zealand Institute for Plant & Food Research Limited
Acknowledgement
• New Zealand’s Foundation for Research,Science and
Technology through contract CO2X050, Better Border Biosecurity
(B3) (www.b3nz.org).
The New Zealand Institute for Plant & Food Research Limited
The New Zealand Institute for Plant & Food Research Limited
Questions ?
Email: [email protected]