Transcript ClusterView

ClusterView
Jason Hill
[email protected]
Senior Research Assistant
U. C. Berkeley
High Level Overview of
ClusterView


Cluster View is a data collection system
used to monitor networked computers.
GOALS:
To build an extensible system that can be
used to monitor heterogeneous collections of
computers.
 To provide web based access to collected
data.

Topics of Discussion




General System Design
Collection Classes & Communication
How to collect new data
Web Access….
Collection System Design

Each nodes starts with an I-space
 base



level assumption
All collection code written in Java.
Collection classes are instantiated on every
node.
Centralized collection process records data
from nodes into SQL Server database.
Communication…



All communication between data center and
nodes is done over ninja RMI.
Each collection object presents itself as an
I-space service that is accessed by the data
center.
The data center itself can be an I-space
service or a standard Java application.
System Diagram
I-spaces
Cluster Nodes
Ninja RMI
Data Center
Collection Objects
HTTP
JDBC
ODBC
SQL Server
Web Server
HTTP
Internet
Communication… (cont.)



Data Center connects to the database using
JDBC.
Web Server connects to database using
ODBC.
Classes are uploaded into the I-space
nodes via the web server using HTTP by the
data center.
Current Examples




Top Information
Currently Running Processes (PS)
CPU usage (SAR, 5 second average)
A Short Demo
Collection Cycle




Query database to gain information about
hosts.
Make RMI calls.
Store data.
Use stored procedures.
Where is the “server”?




The traditional roles of client and server
have been reversed.
The traditional clients are now servers.
The traditional server or data center is now
both a client and a server.
This change adds a lot of flexibility.
How to extend ClusterView

Extensibility...
 1)
Create collection classes.
 --
 2)
must implement ninja RMI interface.
Create a data center class.
 3) Define any necessary database tables.
 4) Place collection classes on web server.
What types of data can you
collect?



High sampling rate?
 Buffer multiple samples at each node before
group upload.
Slow collection time?
 30 X 5 sec/sar = 2.5 minutes!!!
 ClusterView Collection time?  5 sec total
 Background collection.
Irregular samples?
 Have nodes push the data into the data center.
HTML Access

Center piece of the web access is the
GeneralQuery.asp page.
 It
passes query onto SQL Server.
 Determines column name based on SQL
Server output.
 Allows the construction of any query you can
think of.
The power of a Database -Custom Queries & Security

Custom Query Example: What if you asked:
 “Tell
me the username, command, average
CPU used, runtime, and time of any process
that has used more that 30% of a node’s
CPU.”
 (click Here to See the results…)
 Find out just what you want to know.

Security -- Try the query “Delete from
PsData” -- you get “permission denied”.
Getting out of the way….


What if you don’t want the node to be
monitored?
How do you turn things off?
 Ans.:


Right from the web!!
What about all that data?
Memory footprint = 5.4 MB - 3.5MB resident.
What This Means


This gives you the power to collect any
information you can record or compute in
Java.
Create a simple collection class and it will
be automatically distributed to every node of
the NOW cluster.
Next Steps





Security!!!!
Why have a data center at all? What about a
ninja service as the collection method?
HTML interfaces allows universal, generic data
access but what about a more interactive query
tool?
How can Queries be stored so users can define
them from the web?
Notification system if important events are
detected.