Transcript Breakout: XNAT System Administration

XNAT System
Administration
Chip Schweiss
[email protected]
June 26, 2012
XNAT System Administration
1.
2.
3.
4.
System Requirements
Java, Tomcat & PostgreSQL Essentials
Scaling Vertically & Horizontally
An Automation Demonstration with VMware
and Puppet
5. ZFS file system
6. Network Monitoring
XNAT System Requirements
• Highly dependent on size of data set and
volume of users
• Minimum for a working build:
– 2 GB free RAM
– Tomcat 6 on Java 6
– PostgreSQL 9.0 (9.1 not yet supported)
XNAT System Requirements
Two Examples:
1. CNDA – A large instance with many users
2. XNAT Central – Small instance with light
usage
CNDA – System Specifications
• Hosted on our VMware vSphere 5 cluster
– 3.46 GHz Core i7 hex core CPUs
– 10 GbE
– BlueArc 15k SAS pool NFS backing
• CNDA Virtual Machine
– 8 vCPU
– 20 GB ram fully reserved
– 20 GB vmdk
• Separate VM for PostgreSQL
CNDA - System Specifications
NAS File Storage
• 34 TB BlueArc NL-SAS + 34 TB BlueArc on DR site
• 45 TB ZFS backup
Sun Grid Engine Processing Cluster
• 10 - 4 core i7 2.4 GHz systems w/ 8 GB ram
• 6 – 8 vCPU virtual machines w/ 16 GB ram
Supporting VMs
• Shadow for database related processing
• Files system shadow for cron triggered reporting
• Development VMs
XNAT Central – System Specifications
XNAT Central Virtual Machine
• 4 vCPU
• 4 GB ram
Separate PostgreSQL virtual machine
NAS File Storage
• 2 TB BlueArc + 2 TB BlueArc DR
• 3 TB ZFS Backup
Java, Tomcat & PostgreSQL Essentials
Java & Tomcat:
• Memory settings
– Xmx
• Maximum memory allocation
– Xmn
• Minimum allocation
• Set to 1/5 Xmx
– Xms
• Initial heap size
• Set to 1/3 Xmx
– XX:MaxPermSize
• Permanent memory allocation for classes & libraries
• Set to 256m for XNAT
Java & Tomcat
Additional configuration:
• Debugging w/ Eclipse
-Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8000
• Running on port 80/443 with modjk
server.xml:
<Host name="localhost" appBase="/var/lib/tomcat6/webapps/empty"
unpackWARs="true" autoDeploy="true"
xmlValidation="false" xmlNamespaceAware="false">
<Context path="" docBase="/var/lib/tomcat6/webapps/xnatcentral">
<Resource name="UserTransaction" auth="Container“
type="javax.transaction.UserTransaction"
factory="org.objectweb.jotm.UserTransactionFactory“
jotm.timeout="60"/>
<Manager pathname=""/>
</Context>
DICOM Port
Java on Linux does not support listening on
ports < 1024 unless running as root
IP Tables makes an easy work around:
iptables –t nat –I PREROUTING –p tcp \
--dport 104 \
-j DNAT –-to-destination <host ip>:8104
PostgreSQL Tuning
Performance parameters:
• max_connections = 100
• shared_buffers = xxxMB
– ¼ memory available
• work_mem = xxxMB
– Typically 50 to 80 MB
• maintenance_work_mem = xxxMB
– Typically 128 to 250 MB
• effective_cache_size = xxxMB
– ½ memory available
Scaling Vertically & Horizontally
• Reasons to Scale Vertically
– More users
– Higher activity level
– Real-time processing
• Reasons to Scale Horizontally
– Pipeline processing
– Scheduled jobs
Automated Demonstration
Development, Test and Production Cycles
• VMware vSphere
• Puppet
• ZFS
ZFS File System
• Developed by Sun Microsystems
• Highly scalable file system
– Scales in performance with SSD read and write caches
– Scales to Exabyte file system sizes
•
•
•
•
Snapshots, compression, deduplication
Writeable clones of snapshots
Extremely efficient backups with ZFS send/receive
Multiple OS support
–
–
–
–
OpenIndiana: maintained fork of OpenSolaris
FreeBSD
ZFS on Linux
Oracle Solaris
Network Health Monitoring
Preferred tools
• vCenter
• Nagios
• Pingdom
• Spiceworks
Useful links
•
•
•
•
•
•
•
•
•
Our ZFS Backup Server
http://xnat.org/blog/category/xnat-hardware/zfs-storage/
Build Java RPMs for RHEL/Centos
http://www.city-fan.org/tips/SunJava6OnFedora
Scale PostgreSQL horizontally
https://github.com/greg2ndQuadrant/repmgr
Puppet
http://www.puppetlabs.com
Apache Redirect HTTP to HTTPS using mod_rewrite
http://www.sslshopper.com/apache-redirect-http-to-https.html
OpenIndiana
http://www.openindiana.org
ZFS on Linux
http://zfsonlinux.org
Spiceworks
http://www.spiceworks.com
VMware Perl Scripts
http://www.virtuallyghetto.com