Transcript ppt
Selecting and Implementing An Embedded Database System Presented by Jeff Webb March 2005 Article written by Michael Olson IEEE Software, 2000 Overview Strategy – Focus on application requirements Embedded DB products vary Key Some do less than what you need Some will do more Choose the tool that most closely matches your needs Overview After choosing OS, HW platform and DB you must design for reliability Design for performance up front Evaluate performance once the application is built Evaluate Database Services Available High end RDM systems provide: Concurrency Transaction processing Disaster recovery Although these features may be needed, enterprise systems are seldom a good choice due to: platform differences packaging differences Evaluate Database Services Available Several embedded DB systems use the same techniques as enterprise systems but in smaller packages. Often, full blown disaster recovery is not needed Many embedded databases are configurable allowing inclusion or exclusion of services Consider Services Required Now… consider which services you need. For example: Locking? Will run faster without locking Recovery Lack from failures matter? of, or disabling will increase performance Operating Systems For Embedded systems Hundreds of OS for variety of processor hardware DB developers have an enormous job porting their software to the variety of platforms Few OS dominate the market MS Win CE Neutrino VxWorks Wind Rivers Databases For Embedded systems Classification Client Server relational systems Client Server OO systems Embedded libraries Classification Client + Server relational systems Programmers already know SQL - Extra run-time cost of client server communications Classification Client Server OO systems Appear to be a good choice however Most are designed for Unix systems and their deeply engrained assumptions about memory management and interprocess communications are difficult to port to embedded OS. Classification Embedded Created libraries specifically for embedded systems Provide simple language API that does not require SQL + Faster execution, increased reliability - Require developers to master nonstandard programming interfaces Platform Support Consider combination of: OS Processor Storage system Exotic processor boards Rare combinations can be difficult to fit Often embedded OS vendors maintain lists of partner companies whose products run on their systems Performance Considerations High concurrency? Size of database Multiple control threads? Can not rely on standard benchmark measuring systems Evaluation of actual performance is a must Designing the Application Design for performance Consider: Speed Predictability & Reliability Designing the Application Speed Predictability & Reliability Data representation Access patterns Configuration Designing for Speed Data representation/Access patterns/Configuration Most embedded DB tools operate on a fixed set of data types Every fetch and store may require translation A few systems, mostly library types allow storage in program-native format Designing for Speed Data representation/ Consider Access patterns/Configuration the queries that the application will need to make Data should be laid out accordingly Can keys be used that will allow related records to be physically stored together? B+tree storage typically performs faster than hash table algorithms Designing for Speed Data representation/Access patterns/ Configuration Must understand the chosen systems configuration options Memory use for secondary cache Write data to disk or store in memory? Locking system granularity Entire locking system on/off Vendors often choose the wrong defaults Designing the Application Design for performance Consider: Speed Predictability & Reliability Predictability & Reliability May need to run with no humans present Not easy Fanatically check return values and error indicators Resource exhaustion More common in embedded systems Track and release resources yourself Predictability & Reliability Are transactions required so that changes won’t be lost after a crash? The recovery system must be callable from the application program on start up? Performance tuning Common causes for poor performance – 2 or more threads contending for same data Disk-to-memory transfers Deadlocks Contention Contention/Disk-to-memory transfers/ Deadlocks When several threads are waiting for the same resources Use record level locking if possible Use smaller pages if possible to make page level locks more like record level locks Touch the Hot data last and hold it for short periods of time Disk-to-memory transfers/ Contention/ Deadlocks latency – mechanical Flash RAM More memory for buffer cache Indexes Disk Contention/Disk-to-memory transfers/ Deadlocks T1 waits for a lock on O2 T1 T2 Obj O2 locked Obj O1 locked T2 waits for a lock on O1 Deadlocks Turn off locking if it is not needed and if the application permits Break large transactions into several smaller transactions Write transactions so that they all acquire the same resources in the same order Consider Optimistic Concurrency Control Price Helps make final decision Some are available at no cost Licensing methods vary Per developer Per application using it Per deployment platform Per user is less common in embedded systems During development / deployment Questions