Transcript UDMIS.info - University of Dayton

MIS 385/MBA 664
Systems Implementation with DBMS/
Database Management
Dave Salisbury
[email protected] (email)
http://www.davesalisbury.com/ (web site)
UDMIS.info
Physical Database Design
The purpose of the physical design process
is to translate the logical description of the
data into technical specifications for storing
and retrieving data
 Goal: create a design that will provide
adequate performance and insure database
integrity, security, and recoverability
 Decisions made in this phase have a major
impact on data accessibility, response times,
security, and user friendliness.

UDMIS.info
Physical Design Process

Inputs









Decisions
Normalized relations
Volume estimates
Attribute definitions
Response time
expectations
Data security needs
Backup/recovery needs
Integrity expectations
DBMS technology used
Leads to
UDMIS.info





Attribute data types
Physical record
descriptions (doesn’t
always match logical
design)
File organizations
Indexes and database
architectures
Query optimization
Determining volume and usage

Data volume statistics represent the
size of the business


Usage is estimated from the timing of
events, transaction volumes, and
reporting and query activity.

UDMIS.info
calculated assuming business growth over
a period of several years
Less precise than volume statistics
Figure 6.1 - Composite usage map
(Pine Valley Furniture Company)
Usage statistics for PVFC
UDMIS.info
Figure 6.1 - Composite usage map
(Pine Valley Furniture Company)
Data Volumes
Data volumes
UDMIS.info
Figure 6.1 - Composite usage map
(Pine Valley Furniture Company)
Access Frequencies
Access
Frequencies (per
hour)
UDMIS.info
Figure 6.1 - Composite usage map
(Pine Valley Furniture Company)
One usage analysis
Usage analysis:
140 purchased parts
accessed per hour 
80 quotations accessed
from these 140 purchased
part accesses 
70 suppliers accessed from
these 80 quotation accesses
UDMIS.info
Figure 6.1 - Composite usage map
(Pine Valley Furniture Company)
Another usage analysis
Usage analysis:
75 suppliers accessed per
hour 
40 quotations accessed
from these 75 supplier
accesses 
40 purchased parts
accessed from these 40
quotation accesses
UDMIS.info
Physical Design Decisions

Specify the data type for each attribute from
the logical data model

minimize storage space and maximize integrity
Specify physical records by grouping
attributes from the logical data model
 Specify the file organization technique to
use for physical storage of data records
 Specify indexes to optimize data retrieval
 Specify query optimization strategies

UDMIS.info
Designing Fields
Field: smallest unit of data in database
 Field design

Choosing data type
 Coding, compression, encryption
 Controlling data integrity

UDMIS.info
Choosing Data Types
CHAR – fixed-length character
 VARCHAR2 – variable-length character
(memo)
 LONG – large number
 NUMBER – positive/negative number
 INTEGER – positive/negative numbers up to
38 digits
 DATE – actual date – stores c/y/m/d/h/m/s
 BLOB – binary large object (good for
graphics, sound clips, etc.)

UDMIS.info
Data Format

Data type selection goals
minimize storage
 represent all possible values


eliminate illegal values
improve integrity
 support manipulation


UDMIS.info
Note: these have different relative
importance
Data coding
e.g., C(OAK), B(MAPLE) , etc
 Implement by creating a look-up table
 There is a trade-off in that you must create
and store a second table and you must
access this table to look up the code value
 Consider using when a field has a limited
number of possible values, each of which
occupies a relatively large amount of space,
and the number of records is large and/or
the number of record accesses is small

UDMIS.info
Example code-look-up table at PVFC
Code saves space, but
costs an additional
lookup to obtain
actual value.
UDMIS.info
Data Format decisions (integrity)

Data integrity
controls




UDMIS.info
Default value
Range control
Null value control
Referential integrity

Missing data




substitute an
estimate
report missing data
sensitivity testing
Triggers can be used
to perform these
operations
For example...

Suppose you were designing the age
field in a student record at your
university. What decisions would you
make about:
data type
 integrity (range, default, null)
 How might your decision vary by other
characteristics about the student such as
degree sought?

UDMIS.info
Physical Records
Physical Record: A group of fields
stored in adjacent memory locations
and retrieved together as a unit
 Page: The amount of data read or
written in one I/O operation
 Blocking Factor: The number of
physical records per page

UDMIS.info
Denormalization


Transforming normalized relations into
unnormalized physical record specifications
Benefits:


Costs (due to data duplication)



Wasted storage space
Data integrity/consistency threats
Common denormalization opportunities



UDMIS.info
Can improve performance (speed) be reducing number of
table lookups (i.e reduce number of necessary join queries)
One-to-one relationship
Many-to-many relationship with attributes
Reference data (1:N relationship where 1-side has data not
used in any other relationship)
Denormalization: two entities with one-toone relationship
UDMIS.info
Denormalization: many-to-many
relationship with nonkey attributes
Extra table
access
required
Null description possible
UDMIS.info
Denormalization: reference data
Extra table
access
required
Data duplication
UDMIS.info
Consider the following normalized
relations
STORE(Store_Id, Region, Manager_Id,
Square_Feet)
 EMPLOYEE(Emp_Id, Store_Id, Name,
Address)
 DEPARTMENT(Dept#, Store_ID,
Manager_Id, Sales_Goal)
 SCHEDULE(Dept#, Emp_Id, Date, hours)

What opportunities might exist for denormalization?
UDMIS.info
Partitioning

Horizontal Partitioning: Distributing the rows of a
table into several separate files



Vertical Partitioning: Distributing the columns of a
table into several separate files




UDMIS.info
Useful for situations where different users need access to
different rows
Three types: Key Range Partitioning, Hash Partitioning, or
Composite Partitioning
Useful for situations where different users need access to
different columns
The primary key must be repeated in each file
Combinations of Horizontal and Vertical
Partitions often correspond with User Schemas
(user views)
Partitioning

Advantages of Partitioning:






Disadvantages of Partitioning:


UDMIS.info
Records used together are grouped together
Each partition can be optimized for performance
Security, recovery
Partitions stored on different disks: contention
Take advantage of parallel processing capability
Slow retrievals across partitions
Complexity
Data Replication
Purposely storing the same data in multiple
locations of the database
 Improves performance by allowing multiple
users to access the same data at the same
time with minimum contention
 Sacrifices data integrity due to data
duplication
 Best for data that is not updated often

UDMIS.info
Designing Physical Files

Physical File:


Constructs to link two pieces of data:



How the files are arranged on the disk.
Access Method:

UDMIS.info
Sequential storage.
Pointers.
File Organization:


A named portion of secondary memory allocated
for the purpose of storing physical records
How the data can be retrieved based on the file
organization.
Sequential file organization
Records of the
file are stored
in sequence
by the primary
key field
values.
1
2
If sorted –
every insert or
delete requires
resort
If not sorted
Average time to find
desired record = n/2.
UDMIS.info
n
Sequential Retrieval
Consider a file of 10,000 records each
occupying 1 page
 Queries that require processing all records
will require 10,000 accesses
 e.g., Find all items of type 'E'
 Many disk accesses are wasted if few
records meet the condition
 However, very effective if most or all records
will be accessed (e.g., payroll)

UDMIS.info
Indexed File Organizations




Index – a separate table that contains organization
of records for quick retrieval – like an index in a
book.
Primary keys are automatically indexed
Oracle has a CREATE INDEX operation, and MS
ACCESS allows indexes to be created for most field
types
Indexing approaches:




UDMIS.info
B-tree index
Bitmap index
Hash Index
Join Index
Fig. 6-7b – B-tree index
Tree Search
Leaves of the tree
are all at same
level 
consistent access
time
uses a tree search
Average time to find desired
record = depth of the tree
UDMIS.info
Hashing
A technique for reducing disk accesses
for direct access
 Avoids an index
 Number of accesses per record can be
close to one
 The hash field is converted to a hash
address by a hash function

UDMIS.info
Hashed File Organization
Hashing Algorithm: Converts a primary
key value into a record address
 Division-remainder method is common
hashing algorithm

UDMIS.info
Hashing
Hash algorithm
Usually uses divisionremainder to
determine record
position. Records with
same position are
grouped in lists.
UDMIS.info
Hashing
Hash address = remainder after dividing SSN by 10000 (in this case)
UDMIS.info
Shortcomings of Hashing

Different hash fields may convert to the
same hash address


these are called Synonyms
store the colliding record in an overflow area
Long synonym chains degrade performance
 There can be only one hash field per record
 The file can no longer be processed
sequentially
 More collisions between synonyms leads to
reduced access speed

UDMIS.info
Bitmap index organization
Bitmap saves on space requirements
Rows - possible values of the attribute
Columns - table rows
Bit indicates whether the attribute of a row has the values
UDMIS.info
Join Index – speeds up join operations
UDMIS.info
Comparing File Organizations
UDMIS.info
Clustering Files
In some relational DBMSs, related records
from different tables can be stored together
in the same disk area
 Useful for improving performance of join
operations
 Primary key records of the main table are
stored adjacent to associated foreign key
records of the dependent table
 e.g. Oracle has a CREATE CLUSTER
command

UDMIS.info
Indexing
An index is a table file that is used to determine the location of rows in
another file that satisfy some condition
UDMIS.info
Querying with an Index
Read the index into memory
 Search the index to find records
meeting the condition
 Access only those records containing
required data
 Disk accesses are substantially
reduced when the query involves few
records

UDMIS.info
Maintaining an Index

Adding a record requires at least two disk
accesses:



Trade-off:



UDMIS.info
Update the file
Update the index
Faster queries
Slower maintenance (additions, deletions, and
updates of records)
Thus, more static databases benefit more overall
Rules for Using Indexes
Use on larger tables
 Index the primary key of each table
 Index search fields (fields frequently in
WHERE clause)
 Fields in SQL ORDER BY and GROUP
BY commands
 When there are >100 values but not
when there are <30 values

UDMIS.info
Rules for Using Indexes
DBMS may have limit on number of indexes
per table and number of bytes per indexed
field(s)
 Null values will not be referenced from an
index
 Use indexes heavily for non-volatile
databases; limit the use of indexes for
volatile databases


UDMIS.info
Why? Because modifications (e.g. inserts,
deletes) require updates to occur in index files
Rules for Adding Derived Columns
Use when aggregate values are
regularly retrieved.
 Use when aggregate values are costly
to calculate.
 Permit updating only of source data.
 Create triggers to cascade changes
from source data.

UDMIS.info
Another rule of thumb to increase
performance

UDMIS.info
Consider contriving a shorter field or
selecting another candidate key to
substitute for a long, multi-field
primary key (and all associated foreign
keys)
RAID
Redundant Arrays of Inexpensive Disks
 Exploits economies of scale of disk
manufacturing for the computer
market
 Can give greater security
 Increases fault tolerance of systems
 Not a replacement for regular backup

UDMIS.info
RAID
The operating system sees a set of physical
drives as one logical drive
 Data are distributed across physical drives
 All levels, except 0, have data redundancy
or error-correction features
 Parity codes or redundant data are used for
data recovery

UDMIS.info
Mirroring

Write


Read




Read required page from another drive
Tradeoffs



UDMIS.info
Alternate pages are read simultaneously from each drive
Pages put together in memory
Access time is reduced by approximately the number of
disks in the array
Read error


Identical copies of file are written to each drive in array
Provides data security
Reduces access time
Uses more disk space
Mirroring
Complete Data Set
Complete Data Set
No parity
UDMIS.info
Striping


Three drive model
Write




Read


Lost bits are reconstructed from third drive’s parity data
Tradeoffs



UDMIS.info
Portions from each drive are put together in memory
Read error


Half of file to first drive
Half of file to second drive
Parity bit to third drive
Provides data security
Uses less storage space than mirroring
Not as fast as mirroring
Striping
One-Half Data Set
One-Half Data Set
Parity Codes
UDMIS.info
RAID with four disks and striping
Here, pages 1-4 can
be read/written
simultaneously
UDMIS.info
Raid Types

Raid 0







Maximized parallelism
No redundancy
No error correction
no fault-tolerance
Raid 1


Redundant data – fault tolerant
Most common form






No redundancy
One record spans across data
disks
Error correction in multiple
disks– reconstruct damaged
data



Error correction in one disk
Multiple records per stripe
Parallelism, but slow updates
due to error correction
contention
Raid 5



UDMIS.info
Error correction in one disk
Record spans multiple data
disks (more than RAID2)
Not good for multi-user
environments,
Raid 4

Raid 2

Raid 3
Rotating parity array
Error correction takes place in
same disks as data storage
Parallelism, better performance
than Raid4
Database Architectures
Legacy
Systems
Current
Technology
Data
Warehouses
UDMIS.info
Query Optimization
Parallel Query Processing
 Override Automatic Query Optimization
 Data Block Size -- Performance tradeoffs:






UDMIS.info
Block contention
Random vs. sequential row access speed
Row size
Overhead
Balancing I/O Across Disk Controllers
Query Optimizer Factors

Type of Query


Highly selective.
All or most of the records of a file.
Unique fields
 Size of files
 Indexes
 Join Method



UDMIS.info
Nested-Loop
Merge-Scan (Both files must be ordered or
indexed on the join columns.)
Query Optimization
Wise use of indexes
 Compatible data types
 Simple queries
 Avoid query nesting
 Temporary tables for query groups
 Select only needed columns
 No sort without index

UDMIS.info