Transcript presentation

P2P Distributed
Computing Platform:
Korea@Home
C.Y. ‘Connor’ Park
[email protected]
KISTI Supercomputing Center
Overview


Korea@Home (KatH) : A Software
Platform for Distributed Computing
using Volunteer Computer Resources
Features



-2-
Hybrid Peer-to-Peer architecture
Secure PKI-based Security and Digital
Signature
Web Services with SOAP messaging
16th APAN Meetings, BUSAN, KOREA
Motivation



To take advantage of and stimulate
‘Advanced Network Infrastructure in
Korea’
To meet the growing needs of
computing power for applications
Funded by ‘Ministry of Information and
Communication (MIC)’
-3-
16th APAN Meetings, BUSAN, KOREA
Infrastructure


Over 10,000,000 Broadband network (xDSL,
Cable) households - 2002. 11.
Over 58.4% of PCs with Pentium3+
Own PCs
Not own PCs
100%
80%
23.1
21.4
100%
17.7
10.7
82.3
89.3
2001.12
2002.12
60%
76.9
78.6
40%
20%
20%
0%
0%
2001.12
-4-
Internet-unable
80%
60%
40%
Internet-enable
2002.12
16th APAN Meetings, BUSAN, KOREA
Intention of Participation
Korea@Home Understanding
Intention of Providing Resources
Providing with
Don’t (2.9%)
Compensation (7.2%)
Don’t Know (33.3%)
Well-Understanding
(44.9%)
Keep up
Providing (59.5%)
Providing later (30.4%)
Understanding
roughly (21.8%)
• 67.7% of Netizen already know what is Distributed Computing and
Korea@Home.
• 89.9% of Netizen have intention of providing their PC resources.
-5-
16th APAN Meetings, BUSAN, KOREA
Similar Projects

Science


Internet


Folding@Home,
FightAIDS@Home


MoneyBee
Games

Distributed.net,
RSAttack576
Grub, CycleTrader
Financial

Cryptography


SETI@Home,
climateprediction.net
Life Science



ChessBrain
Miscellaneous

DALiWorld
Mathematics

Mersenne Prime Search,
Pi(x) Project
from http://www.aspenleaf.com/distributed/index.html
-6-
16th APAN Meetings, BUSAN, KOREA
Project History

2002. 4.



2002. 8.



Second pilot production and test
2003. 4.


First pilot production and test
Client/Server Architecture Model
2002. 12.


MIC launched the pilot project ‘Korea@Home’
Application : Virtual Screening for New Drug Design
Regularize the project “P2P Internet-based Distributed
Computing Platform Development”
2003. 9.

-7-
Announcement of Hybrid P2P-style Distributed Computing
Platform
16th APAN Meetings, BUSAN, KOREA
Architecture
Agent
Agent
Cooperation
Agent
Cooperation
Distribution
Distribution
Cooperation
Distribution
Agent
Internet
Distribution
Subscription
Job Request
Resource
Management
-8-
Large-scale
Application
16th APAN Meetings, BUSAN, KOREA
Design Goals

Application Developers (Job Requester)

Convenience


Flexibility


The platform is general enough to be used by
various applications.
Correctness

-9-
They should be able to use easily.
The platform should return correct results.
16th APAN Meetings, BUSAN, KOREA
Design Goals

PC agents (Resource Providers)

Transparency


Reliability


It’s reliable not to make any security and privacy
problem.
Reward

- 10 -
PC users are not disturbed in their own works.
There is any kind of compensation, even just in
mind.
16th APAN Meetings, BUSAN, KOREA
Design Goals

Platform

Scalability


Adaptability

- 11 -
Performance should be improved according to
the growth of agents.
It should put up with the dynamic features of
Internet.
16th APAN Meetings, BUSAN, KOREA
Implementation Choices
Make server as simple as possible,
and Make PC agents more intelligent

Web Services


Request-Response Model


SOAP messaging over https
It is Agents’ duty to make most decisions.
PKI-based certification and Digital
Signature for files
- 12 -
16th APAN Meetings, BUSAN, KOREA
Functional Classification

Resource Management



Scheduling and Job distribution



Authentication, Certification
Simple accounting
Job distribution with hybrid P2P style
Result gathering
Client Management




- 13 -
Client authentication
Application preparation to work
Digital signing
Result returning to client
16th APAN Meetings, BUSAN, KOREA
Job Distribution
Agent
Agent
Agent
 Authentication
Internet
 Working Specification
Request
• Is there any job for
me to work?
• Who has the job
Agent
data?
 Getting data, working,
and returning result to
server
KatH Server
- 14 -
16th APAN Meetings, BUSAN, KOREA
Networking Issues

Network Security


Variable Connection



Dial-up, cable, xDSL, T1/E1, …
IPv6, WiFi, Bluetooth, …
Network Unobtrusiveness


DoS, Spoofing, Sniffing, …
Background/shared communication
Transparency (Firewall/NAT)
- 15 -
16th APAN Meetings, BUSAN, KOREA
2002 Pilot Projects



Application : Virtual Screening for New Drug
Design
Client-Server Architecture
An Agent




Receives about 400KB-size file
Executes during 20 minutes per working unit
Returns about 190KB-size file
Total number of working job units : 50,000
- 16 -
16th APAN Meetings, BUSAN, KOREA
Virtual Screening
Target Protein
Chemical DB
Share
Analysis
Disease Protein
Clinical Demo New Drug Mass-produce
Ligand DB
Virtual High-Throughput Screening
+
+
Client 1
+
+
Client 2
Server
+
Calculating Active Site PMI
Calculating Ligand PMI
+
Looking for
Drug Candidate
Client 3
Internet
Structure Analysis
- 17 -
16th APAN Meetings, BUSAN, KOREA
Experimental Results

1st Test




1,217 installed agents
2002.8.22-9.2
Max 53.5 Gflops / Avg. 31.78 Gflops
2nd Test



6,472 installed agents
2003.2.6-2.14
Max 644 Gflops / Avg. 470.6 Gflops
※ World Top Rank 124 - 629.7 Gflops:
IBM xSeries Cluster Xeon 2.8 GHz - Myrinet/ 192
(from http://www.top500.org, 2003.6)
- 18 -
16th APAN Meetings, BUSAN, KOREA
2003 Progresses


P2P data distribution
New Applications




Virtual Screening for New Drug Design
Global Risk Management for financial
investment
3D image rendering
Supplementary plug-in

- 19 -
Remote backup service
16th APAN Meetings, BUSAN, KOREA
Future works





Make agents more intelligent with P2P
MPI-like API for application developer
Fault-tolerance : checkpointing &
rollback recovery
IPv6 enables
Open-Source
- 20 -
16th APAN Meetings, BUSAN, KOREA
Thanks a lot! ^^
[email protected]
- 21 -
16th APAN Meetings, BUSAN, KOREA