Transcript Chapter 21: New Applications

Chapter 21: New Applications
 Decision-Support Systems
 Data Analysis
 Data Mining
 Data Warehousing
 Spatial and Geographic Databases
 Multimedia Databases
 Mobility and Personal Databases
 Information-Retrieval Systems
 Distributed Information Systems
 The World Wide Web
1
Database System Concepts 3rd Edition
21.1
©Silberschatz, Korth and Sudarshan
Decision Support System
 Decision-Support systems are used to make business
decisions often based on data collected by on-line
transaction-processing systems.
 Examples of business decisions:
 what items to stock?
 What insurance premium to change?
 Who to send advertisements to?
 Examples of data used for making decisions

Retail sales transaction details

Customer profiles (income, age, sex, etc.)
2
Database System Concepts 3rd Edition
21.2
©Silberschatz, Korth and Sudarshan
Decision-Support Systems: Overview
 Data analysis tasks are simplified by SQL
extensions.
 Statistical analysis packages (e.g., : S++) can be
interfaced with databases
 Statistical analysis is a large field will not study it here
 Data mining seeks to discover knowledge
automatically in the form of statistical rules and
patterns from Large databases.
 A data warehouse archives information gathered
from multiple sources, and stores it under a unified
schema, at a single site.
3
Database System Concepts 3rd Edition
21.3
©Silberschatz, Korth and Sudarshan
Data Analysis
 Aggregate functions summarize large volumes of data
 A histogram partitions the values taken by an attribute into
ranges, and computes an aggregate over the values in
each range; cumbersome to use standard SQL to
construct a histogram. Extension proposed by Red Brick:
 select percentile, avg (balance)
 from account
 group by N_tile (balance, 10) as percentile
4
Database System Concepts 3rd Edition
21.4
©Silberschatz, Korth and Sudarshan
Data Analysis (Cont.)
Small
Medium
Large
Total
Light
Dark
8
20
35
10
310
5
53
35
Total
28
45
15
88
 Cross-tabulation of number by size and color of sample
relation sales with the schema Sales(color, size, number).
5
Database System Concepts 3rd Edition
21.5
©Silberschatz, Korth and Sudarshan
Data Analysis (Cont.)
Color
Size
Number
Light
Light
Light
Light
Dark
Dark
Dark
Dark
all
all
all
all
Small
Medium
Large
all
Small
Medium
Large
all
Small
Medium
Large
all
8
35
10
53
20
10
5
35
28
45
15
88
 Can represent subtotals in relational form by using the value all
 E.g. : obtain (Light, all, 53) and (Dark, all, 35) by aggregating
individual tuples with different values for size for each color.
6
Database System Concepts 3rd Edition
21.6
©Silberschatz, Korth and Sudarshan
Data Analysis (Cont.)
 Rollup: Moving from finer-granularity data to a coarser
granularity by means of aggregation.
 Drill down: Moving from coarser-granularity data finer-
granularity data.
 Proposed extensions to SQL, such as the cube
operation help to support generation of summary data
 The following query generates the previous table.
select color, size, sum (number)
from sales
groupby color, size with cube
7
Database System Concepts 3rd Edition
21.7
©Silberschatz, Korth and Sudarshan
Data Analysis (Cont.)
 Figure shows the combinations of dimensions size, color, price
 In general computing cube operation with n groupby columns gives
2nd different groupby combinations.
8
Database System Concepts 3rd Edition
21.8
©Silberschatz, Korth and Sudarshan
Data Mining
 Like knowledge discovery in artificial intelligence data mining
discovers statistical rules and patterns it differs from machine
learning in that it deals with large volumes of data stored
primarily on disk.
 Knowledge discovered from a database can be represented by
a set of rules.
 e.g.,: “Young women with annual incomes greater than $50,000
are most likely to buy sports cars”
 Discover rules using one of two models:
1. The user is involved directly in the process of knowledge
discovery.
2. The system is responsible for automatically discovering
knowledge from the database by detecting patterns and
correlation's in the data.
9
Database System Concepts 3rd Edition
21.9
©Silberschatz, Korth and Sudarshan
Knowledge Representation Using Rules
 General form of rules  X antecedent  consequent
 X is a list of one or more variables with associated ranges.
 The rule  transactions T, buys (T, bread)  buys(T, milk)
states: if there is a tuple (t1, bread) in the relation buys,
there must also be a tuple (t1, milk) in the relation buys.
 Population: Cross-product of the ranges of the variables in
the rule.
 In the above example, the set of all transactions.
 Support: Measure of what fraction of the population
satisfies both the antecedent and the consequent of the
rule.
 e.g., 10% of transactions buy bread and milk.
 Confidence : Measure of how often the consequent is true
when the antecedent is true.
 e.g., 80% of transactions that buy bread also buy milk
10
Database System Concepts 3rd Edition
21.10
©Silberschatz, Korth and Sudarshan
Some Classes of Data-Mining Problems
 Classification : Finding rules that partition the given data
into disjoint groups (classes) that are relevant for making a
decision
 (e.g.,: which of several factors help classify a person’s credit
worthiness).
 Associations: Useful to determine associations between
different items
 (e.g.,: someone who buys bread is quite likely also to buy
milk).
 Sequence correlations: determine correlations between
related sequence data.
 (e.g., : when bond rates go up stock prices go down within
two days.)
11
Database System Concepts 3rd Edition
21.11
©Silberschatz, Korth and Sudarshan
User-Guided Data Mining
 In user-guided data mining, primary responsibility for
discovering rules is with the user.
 User may runs tests on the database to verify or refute a
hypothesis. Confidence and support for rules expressing
a hypothesis are derived from the database.
 An iterative process of forming and refining rules is used.
Example: Test the hypothesis “People who hold master’s
degrees are the most likely to have an excellent credit
rating.” If confidence of rule is low, may refine it into the
rule:
 people P, P.degree = Masters and C.income  75, 000
 C.credit = excellent
 Data-visualization though graphical representations like
maps, charts, and color-coding, helps detect patterns in
data
12
Database System Concepts 3rd Edition
21.12
©Silberschatz, Korth and Sudarshan
Classification Rules
 Classification rules help assign new objects to a set of classes.
E.g., given a new automobile insurance applicant, should he or
she be classified as low risk, medium risk or high risk?
 Classification rules for above example could use a variety of
knowledge, such as educational level of applicant, salary of
applicant, age of applicant, etc.
 Classification rules can be compactly shown as a Classification
tree.
13
Database System Concepts 3rd Edition
21.13
©Silberschatz, Korth and Sudarshan
Part of Credit Risk Classification Tree
14
Database System Concepts 3rd Edition
21.14
©Silberschatz, Korth and Sudarshan
Discovery of Classification Rules
 Training set: a data sample in which the grouping for each
tuple is already known.
 Top down generation of classification tree.
 Each internal node of the tree partitions the data into groups
based on the attribute.
 The data at a node is not partitioned further if either all (or most)
of the items at the node belong to the same class, or all
attributes have been considered. Such a node is a leaf node.
 Otherwise the data at the node is partitioned further by picking
an attribute for partitioning data at the node.
15
Database System Concepts 3rd Edition
21.15
©Silberschatz, Korth and Sudarshan
Discovery of Classification Rules (Cont.)
 Consider credit risk example: Suppose degree is chosen
to partition the data at the root. Since degree has a small
number of possible values, one child is created for each
value.
 At each child node of the root, further classification is
done tuple if required. Here, partitions are defined by
income. Since income is a continuous attribute, some
number of intervals are chosen, and one child created for
each interval.
 Different classification algorithms use different ways of
choosing which attribute to partition on at each node, and
what the intervals, if any, are.
 In general, different branches of the tree could grow to
different levels. Different nodes at the same level may use
difficult partitioning attributes.
16
Database System Concepts 3rd Edition
21.16
©Silberschatz, Korth and Sudarshan
Discovery of Association Rules
 Example:  transactions T, buys (T, bread)  buys (T, milk)
 In general: notion of transaction , and its intemset, the set of items
contained in the transaction
 General form of rule:
 transactions T, c(T, i1) and . . . and c(T, io)  c(T, i0)
where c(T, ik) denotes that transaction T contains item ik.
Above can be represented as A  b where A = {i1, i2, . . . , in}
and b = io.
 Support of rule = number of transactions whose itemsets contain
A  {b}
 Usually desire rules with strong support, which will involve only
items purchased in a significant percentage of the transactions.
17
Database System Concepts 3rd Edition
21.17
©Silberschatz, Korth and Sudarshan
Discovery of Association Rules (Cont.)
 Consider all possible sets of relevant items.
 For each set find its support (i.e. , how many transactions
purchase all items in the set).
 Use sets with sufficiently high support to generate
association rules.
 From set A generate the rules A - {b} b for each b  A.
 Support of each of the rules is support of A.
 Confidence of a rule is support of A divided by support of
A - {b}.
18
Database System Concepts 3rd Edition
21.18
©Silberschatz, Korth and Sudarshan
Finding Support
 Few sets: Determine level of support via a single pass.
 A count is maintained for each set, initially set to 0.
 When a transaction is fetched, the count is incremented for each
set of items which contained in the itemset of the transaction.
 Sets with a high count at the end of the pass correspond to items
with a high degree of association.
 Many sets: If memory not enough to hold all counts for all sets
Use multiple passes, considering only some sets in each
pass.
 Optimization: Once a set is eliminated because it occurs in too
small a fraction of the transactions, none of its supersets
needs to be considered.
19
Database System Concepts 3rd Edition
21.19
©Silberschatz, Korth and Sudarshan
Data Warehousing
 A data warehouse is a repository of information gathered
from multiple sources.
20
Database System Concepts 3rd Edition
21.20
©Silberschatz, Korth and Sudarshan
Data Warehousing (Cont.)
 Provides a single consolidated interface to data
 Data stored for an extended period, providing access to
historical data
 Data/updates are periodically downloaded form online
transaction processing (OLTP) systems.
 Typically, download happens each night.
 Data may not be completely up-to-date, but is recent enough for
analysis.
 Running large queries at the warehouse ensures that OLTP
systems are not affected by the decision-support workload.
21
Database System Concepts 3rd Edition
21.21
©Silberschatz, Korth and Sudarshan
Issues in Building a Warehouse
 When and how to gather data.
 Source driven: data source initiates data transfer
 Destination driven: warehouse initiates data transfer
 What schema to use.
 Schema integration
 Cleaning and conversion of incoming data
 What data to summarize.
 Raw data may be too large to store on-line
 Aggregate values (totals/subtotals) often suffice
 Queries on raw data can often be transformed by query optimizer to
use aggregate values
 How to propagate updates.
 Date at warehouse is a view on source data
 Efficient view maintenance techniques required
22
Database System Concepts 3rd Edition
21.22
©Silberschatz, Korth and Sudarshan
Spatial and Geographic Databases
 Spatial databases store information related to spatial
locations, and support efficient storage, indexing and querying
of spatial data.
 Special purpose index structures are important for accessing
spatial data, and for processing spatial join queries.
 Design databases (or CAD databases) store design
information about how objects are constructed E.g.: designs of
buildings, aircraft, layouts of integrated-circuits
 Geographic databases store geographic information (e.g.,
maps): often called geographic information systems or GIS.
23
Database System Concepts 3rd Edition
21.23
©Silberschatz, Korth and Sudarshan
Represented of Geometric Information
 Various geometric constructs can be represented in a
database in a normalized fashion.
 Represent a line segment by the coordinates of its endpoints.
 Approximate a curve by partitioning it into a sequence of
segments; represents each segment as a separate tuple that
also carries with it the identifier of the curve (2D features such
as roads).
 Closed polygons: list its vertices in order, starting vertex is the
same as the ending vertex.
Alternative: triangulation — divide polygon into triangles;
give the polygon an identifier with each of its triangles.
24
Database System Concepts 3rd Edition
21.24
©Silberschatz, Korth and Sudarshan
Representation of Geometric Constructs (Cont.)
25
Database System Concepts 3rd Edition
21.25
©Silberschatz, Korth and Sudarshan
Representation of Geometric Information (Cont.)
 Representation of points and line segment in 3-D similar to 2-D,
except that points have an extra z component
 Represent arbitrary polyhedra by dividing them into tetrahedrons,
like triangulating polygons.
 Alternative: List their faces, each of which is a polygon, along with
an indication of which side of the face is inside the polyhedron.
26
Database System Concepts 3rd Edition
21.26
©Silberschatz, Korth and Sudarshan
Design Databases
 Represent design components as objects (generally
geometric objects); the connections between the objects
indicate how the design is structured.
 Simple two-dimensional objects: points, lines, triangles,
rectangles, polygons.
 Complex two-dimensional objects: formed from simple
objects via union, intersection, and difference operations.
 Complex three-dimensional objects: formed from simpler
objects such as spheres, cylinders, and cuboids, by union,
intersection, and difference operations.
 Wireframe models represent three-dimensional surfaces
as a set of simpler objects.
27
Database System Concepts 3rd Edition
21.27
©Silberschatz, Korth and Sudarshan
Representation of Geometric Constructs
(a) Difference of cylinders
(b) Union of cylinders
 Design databases also store non-spatial information about
objects (e.g., construction material, color, etc.)
 Spatial integrity constraints are important.
 E.g., pipes should not intersect, wires should not be too close to
each other, etc.
28
Database System Concepts 3rd Edition
21.28
©Silberschatz, Korth and Sudarshan
Geographic Data
 Raster data consist of bit maps or pixel maps, in two or more
dimensions.
 Example 2-D raster image: satellite image of cloud cover, where
each pixel stores the cloud visibility in a particular area.
 Additional dimensions might include the temperature at different
altitudes at different regions, or measurements taken at different
points in time.
 Design databases generally do not store raster data.
29
Database System Concepts 3rd Edition
21.29
©Silberschatz, Korth and Sudarshan
Geographic Data (Cont.)
 Vector data are constructed from basic geometric objects:
points, line segments, triangles, and other polygons in two
dimensions, and cylinders, speheres, cuboids, and other
polyhedrons in three dimensions.
 Vector format often used to represent map data.
 Roads can be considered as two-dimensional and represented
by lines and curves.
 Some features, such as rivers, may be represented either as
complex curves or as complex polygons, depending on whether
their width is relevant.
 Features such as regions and lakes can be depicted as
polygons.
30
Database System Concepts 3rd Edition
21.30
©Silberschatz, Korth and Sudarshan
Applications of Geographic Data
 Examples of geographic data
 map data for vehicle navigation
 distribution network information for power, telephones, water
supply, and sewage
 Vehicle navigation systems store information about roads
and services for the use of drivers:
 Spatial data: e.g, road/restaurant/gas-station coordinates
 Non-spatial data: e.g., one-way streets, speed limits, traffic
congestion
 Global Positioning System or GPS unit - utilizes
information broadcast from GPS satellites to find the
current location of user with an accuracy of tens of
meters.
 increasingly used in vehicle navigation systems as well as
utility maintenance applications.
31
Database System Concepts 3rd Edition
21.31
©Silberschatz, Korth and Sudarshan
Spatial Queries
 Nearness queries request objects that lie near a specified
location.
 Nearest neighbor queries, given a point or an object, find
the nearest object that satisfies given conditions.
 Region queries deal with spatial regions. e.g., ask for
objects that lie partially or fully inside a specified region.
 Queries that compute intersections or unions of regions.
 Spatial join of two spatial relations with the location playing
the role of join attribute.
32
Database System Concepts 3rd Edition
21.32
©Silberschatz, Korth and Sudarshan
Spatial Queries (Cont.)
 Spatial data is typically queried using a graphical query
language; results are also displayed in a graphical
manner.
 Graphical interface constitutes the front-end
 Extensions of SQL with abstract data types, such as
lines, polygons and bit maps, have been proposed to
interface with back-end.
 allows relational databases to store and retrieve spatial
information
 queries can mix spatial and nonspatial conditions
 extensions also include and allowing spatial conditions
(contains or overlaps).
33
Database System Concepts 3rd Edition
21.33
©Silberschatz, Korth and Sudarshan
Indexing of Spatial Data
 k-d - early structure used for indexing in multiple dimensions.
 Each level of a k-d tree partitions the space into two.
 choose one dimension for partitioning at the root level of the tree.
 choose another dimensions for partitioning in nodes at the next level
and so on, cycling through the dimensions.
 In each node, approximately half of the points stored in the sub-
tree fall on one side and half on the other.
 Partitioning stops when a node has less than a given maximum
number of points.
 The k-d-B tree extends the k-d tree to allow multiple child nodes
for each internal node; well-suited for secondary storage.
34
Database System Concepts 3rd Edition
21.34
©Silberschatz, Korth and Sudarshan
Division of Space by a k-d Tree
 Each line in the figure (other than the outside box)
corresponds to a node in the k-d tree; the maximum
number of points in a leaf node has been at 1.
 The numbering of the lines in the figure indicates the
level of the tree at which the corresponding node
appears.
35
Database System Concepts 3rd Edition
21.35
©Silberschatz, Korth and Sudarshan
Division of Space by Quadtrees
 Alternative to k-d trees.
 Each node is associated with a rectangular region of space;
the top node is associated with the entire target space.
 Each non-leaf nodes divides its region into four equal sized
quadrants, and correspondingly each such node has four child
nodes corresponding to the four quadrants.
 Leaf nodes have between zero and some fixed maximum
number of points (set to 1 in example figure above).
Database System Concepts 3rd Edition
21.36
36
©Silberschatz, Korth and Sudarshan
Quadtrees (Cont.)
 PR quadtree: stores points; space is divided based on regions,
rather than on the actual set of points stored.
 Region quadtrees store array (raster) information.
 A node is a leaf node is all the array values in the region that it
covers are the same. Otherwise, it is subdivided further into four
children of equal area, and is therefore an internal node.
 Each node corresponds to a sub-array of values.
 The sub-arrays corresponding to leaves either contain just a single
array element, or have multiple array elements, all of which have
the same value.
 Extensions of k-d trees and quadtrees handle indexing of line
segments and polygons.
37
Database System Concepts 3rd Edition
21.37
©Silberschatz, Korth and Sudarshan
R-Trees
 R-trees are a N-dimensional extension of B-trees, useful
for indexing sets of rectangles and other polygons.
 Supported in many modern database systems, along with
variants like R+ -trees and R*-trees.
 Basic idea: generalize the notion of a one-dimensional
interval associated with each B+ -tree node to an
N-dimensional interval, that is, an N-dimensional
rectangle.
 Will consider only the case N = 2; generalization for
N > 2 is straightforward
38
Database System Concepts 3rd Edition
21.38
©Silberschatz, Korth and Sudarshan
R Trees (Cont.)

A rectangular bounding box is associated with each tree
node.
 Bounding box of a leaf node is a minimum sized rectangle that
contains all the rectangles/polygons associated with the leaf
node.
 The bounding box associated with a non-leaf node contains the
bounding box associated with all its children.
 Bounding box of a node serves as its key in its parent node (if
any)
 Bounding boxes of children of a node are allowed to overlap
 A polygon is stored only in one node, and the bounding box
of the node must contain the polygon
 The storage efficiency or R-trees is better than that of k-d
trees or quadtrees since a polygon is stored only once.
39
Database System Concepts 3rd Edition
21.39
©Silberschatz, Korth and Sudarshan
Example R-Tree
 A set of rectangles (solid line) and the bounding boxes (dashed line) of the
nodes of an R-tree for the rectangles. The R-tree is shown on the right.
B
A
0
3
1
C
G
I
D
2
1
A B C
H
2
DE F
3
GH I
E
F
40
Database System Concepts 3rd Edition
21.40
©Silberschatz, Korth and Sudarshan
Operations on R-Trees
 To find data items (rectangles/polygons) intersecting
(overlaps) a given query point/region , do the following,
starting from the root node:
 If the node is a leaf node, output the data items whose keys
intersect the given query point/region.
 Else, for each child of the current node if its bounding box
overlaps the query point/region, recursively search the child

Can be very inefficient in worst case, but works
acceptably in practice.
 Simple extensions of above to handle contained in and
contains
 To insert a data item:
 Find a leaf to store it, and add it to the leaf. Handle overflows
by splits as in B+ -trees
 Adjust bounding boxes starting from the leaf upwards
41
Database System Concepts 3rd Edition
21.41
©Silberschatz, Korth and Sudarshan
Multimedia Databases
 To provide such database functions as indexing and
consistency, it is desirable to store multimedia data in a
database (rather than storing them outside the database,
in a file system).
 The database must handle large object representation.
 Similarity-based retrieval must be provided by special
index structures.
 Must provide guaranteed steady retrieval rates for
continuos-media data.
42
Database System Concepts 3rd Edition
21.42
©Silberschatz, Korth and Sudarshan
Similarity-Based Retrieval
 Pictorial data - Two pictures or images that are slightly different as
represented in the database may be considered the same by a
user.
 (E.g., identify similar designs for registering a new trademark.)
 Audio data- Speech-based user interfaces allow the user to give a
command or identify a data item by speaking.
 E.g., test user input against stored commands.)
 Handwritten data- Identify a handwritten data item or command
stored in the database (requires similarity testing).
43
Database System Concepts 3rd Edition
21.43
©Silberschatz, Korth and Sudarshan
Continuos-Media Data
 Most important types are video and audio data.
 Characterized by high data volumes and real-time
information-delivery requirements.
 Data must be delivered sufficiently fast that there are no
gaps in the audio or video.
 Data must be delivered at a rate that does not cause
overflow of system buffers.
 Synchronization among distinct data streams must be
maintained (video of a person speaking must show lips
moving synchronously with the audio).
44
Database System Concepts 3rd Edition
21.44
©Silberschatz, Korth and Sudarshan
Multimedia Data Formats
 Store and transmit multimedia data in compressed
form;JPEG is the most widely used format for image
data.
 Encoding each frame of a video using JPEG is wasteful
since, successive frames of a video are often nearly the
same.
 MPEG standards use commonalties among a sequence
of frames to achieve a greater degree of compression.
 MPEG-1 stores a minute of 30-frame-per-second video
and audio in approximately 12.5 MB (compares with 75
MB for video using only JPEG); quality comparable to
VHS video tape.
 MPEG-2 designed for digital broadcast systems and
digital video disks; negligible loss of video quality.
 Compresses 1 minute of audio-video to approximately
17 MB.
Database System Concepts 3rd Edition
45
21.45
©Silberschatz, Korth and Sudarshan
Video Servers
 Current video-on-demand servers are based on file systems;
existing database systems do not meet real-time response
requirements.
 Multimedia data are stored on several disks (RAID
configuration), or on tertiary storage for less frequently accessed
data.
 Head-end terminals - used to view multimedia data; PCs or TVs
attached to a small, inexpensive computer called a set-top box.
 Network - Transmission of multimedia data from a server to
multiple head-end terminals requires a high-capacity network,
such as asynchronous-transfer-mode (ATM) network.
46
Database System Concepts 3rd Edition
21.46
©Silberschatz, Korth and Sudarshan
Mobility and Personal Databases
 A mobile computing environment consists of mobile
computers, referred to as mobile hosts, and a wired
network of computers.
 Mobile host may be able to communicate with wired
network through a wireless digital communication network
 Wireless local-area networks (within a building)
 Cellular digital packet networks (wide area)
 A model for mobile communication
 Mobile hosts communicate to the wired network via
computers referred to as mobile support stations.
 Each mobile support station manages those mobile hosts
within its cell.
 When mobile hosts move between cells, there is a handoff of
control from one mobile support station to another.
47
Database System Concepts 3rd Edition
21.47
©Silberschatz, Korth and Sudarshan
Database Issues in Mobile Computing
 New issues for query optimization.
 energy (battery power) is a scarce resource and its usage
must be minimized
 mobile user’s locations may be a parameter of the query.
 Broadcast data can enable any number of clients to
receive the same data at no extra cost; leads to
interesting querying and data caching issues.
 Users may need to be able to perform database updates
even while the mobile computer is disconnected.
 e.g., mobile salesman records sale of products on (local
copy of) database.
 Can result in conflicts detected on reconnection, which may
need to be resolved manually.
48
Database System Concepts 3rd Edition
21.48
©Silberschatz, Korth and Sudarshan
Routing and Query Processing
 Must consider these competing costs:
 User time.
 Communication cost
 Connection time - used to assign monetary charges in
some cellular systems.
 Number of bytes, or packets, transferred - used to compute
charges in digital cellular systems
 Time-of-day based charges - vary based on peak or offpeak periods
 Energy - optimize use of battery power by minimizing
reception and transmission of data. Receiving radio
signals requires much less energy than transmitting
radio signals.
49
Database System Concepts 3rd Edition
21.49
©Silberschatz, Korth and Sudarshan
Broadcast Data
 Mobile support stations can broadcast frequently-requested
data; allows mobile hosts to wait for needed data, rather
than having to consume energy transmitting a request.
 A mobile host may optimize energy costs by determining if
a query can be answered using only cached data; if not
then must either;
 Wait for the data to be broadcast
 Transmit a request for data and must know when the relevant
data will be broadcast.
 Broadcast data may be transmitted according to a fixed
schedule or a changeable schedule.
 For changeable schedule- the broadcast schedule must itself
be broadcast at a well-known radio frequency and at wellknown time intervals.
50
Database System Concepts 3rd Edition
21.50
©Silberschatz, Korth and Sudarshan
Disconnectivity and Consistency
 A mobile host may remain in operation during periods of
disconnection.
 Problems created if the user of the mobile host issues
queries and updates on data that resides or is cached
locally:
 Recoverability: Updates entered on a disconnected machine
may be lost if the mobile host fails. Since the mobile host
represents a single point of failure, stable storage cannot be
simulated well.
 Consistency : Cached data may become out of date, but the
mobile host cannot discover this until it is reconnected.`
51
Database System Concepts 3rd Edition
21.51
©Silberschatz, Korth and Sudarshan
Mobile Updates
 Partitioning via disconnection is the normal mode of
operation in mobile computing.
 For data updated by only the mobile host, simple to
propagate update when mobile host reconnects; in other
cases data may become invalid and updates may
conflict.
 When data are updated by other computers, invalidation
reports inform a reconnected mobile host of out-of-date
cache entries; however, mobile host may miss a report.
 Version-numbering-based schemes guarantee only that if
two hosts independently update the same version of a
document, the clash will be detected eventually, when the
hosts exchange information either directly or through a
common host.
 Automatic reconciliation of inconsistent copies of data is
still an area of research.
52
Database System Concepts 3rd Edition
21.52
©Silberschatz, Korth and Sudarshan
Detecting Inconsistent Updates
 Version vector scheme used to detect inconsistent updates to
documents at different hosts (sites).
 Copies of document d at hosts i and j are inconsistent if
 1. the copy of document d at i contains updates performed by host
k that have not been propagated to host j (k may be the same as i),
and
 2. the copy of d at j contains updates performed by host l that have
not been propagated to host i (l may be the same as j)
 Basic idea: each host i stores, with its copy of each document d,
a version vector - a set of version numbers, with an element
{Vd,I,j} for every other host j
 When a host i updates a document d, it increments the version
number Vd,I,j.
53
Database System Concepts 3rd Edition
21.53
©Silberschatz, Korth and Sudarshan
Detecting Inconsistent Updates (Cont.)
 When two hosts i and j connect to each other they check if
the copies of all documents d that they share are consistent:
1. If the version vectors are the same on both hosts-that is,
for each k, Vd,i,k,= Vd,j,k - then the copies of d are identical.
2. If, for each k, Vd,i,k,  Vd,j,k, and the version vectors are
not identical, then the copy of document d at host i is older
than the one at host j
 That is, the copy of document d at host j was obtained by
one or more modifications of the copy of d at host i. Host i
replaces its copy of d, as well as its copy of the version
vector for d, with the copies from host j.
3. If there is a pair of hosts k and l such that Vd,I,k < Vd,j,k and
Vd,i,l > Vd,j,l then the copies are inconsistent: two or more
updates have been performed on d independently.
54
Database System Concepts 3rd Edition
21.54
©Silberschatz, Korth and Sudarshan
Handling Inconsistent Updates
 Dealing with inconsistent updates is hard in general. Manual
intervention often required to merge the updates.
 Version vector schemes
 were developed to deal with failures in a distributed file system,
where inconsistencies are rare.
 are used to maintain a unified file system between a fixed host
and a mobile computer, where updates at the two hosts have to
be merged periodically. Also used for similar purposes in
groupware systems.
 are used in database systems where mobile users may need to
perform transactions. In this case, a “document” may be a
single record.
 Must ensure that inconsistencies are either very rare, or fall
in special cases that are easy to deal with in most cases
55
Database System Concepts 3rd Edition
21.55
©Silberschatz, Korth and Sudarshan
Information Retrieval Systems
 Information retrieval (IR) systems use a simpler data
model than database systems, but provide more
powerful querying capabilities within the restricted
model.
 Queries attempt to locate documents that are of interest
by specifying, for example, sets of keywords.
 e.g., find documents containing the words “database
systems”
 Information retrieval systems order answers based on
their estimated relevance.
 e.g., user may really only want documents about
database systems, but the system may retrieve all
documents that mention the phrase database systems”.
 Documents may be ordered by, for example, how many
times the phrase appears in the document.
56
Database System Concepts 3rd Edition
21.56
©Silberschatz, Korth and Sudarshan
Queries
 Combinations of keywords
 motorcycle and maintenance
 computer or micro-processor
 computer but not database
 Closeness of keyword s in the and case affects ranking.
Some systems allow user to specify that the keywords must
occur close to each other.
 Synonyms
 To retrieve document title motorcycle repair for the query
motorcycle and maintenance, need to realize that maintenance
and repair are synonyms
 Similarity based retrieval - retrieve documents similar to a
given document. Similarity may be defined based on metrics
such as number of common keywords.
57
Database System Concepts 3rd Edition
21.57
©Silberschatz, Korth and Sudarshan
Differences From Database Systems
 Information retrieval systems, unlike traditional database
systems, handle:
 Unstructured documents
 Searching using keywords and relevance ranking
 Most information retrieval systems do not handle:
 High update rates
 Concurrency control
 Data structured using more complex data models (e.g., relational
or object oriented data models)
 Complex queries written in, e.g., SQL
58
Database System Concepts 3rd Edition
21.58
©Silberschatz, Korth and Sudarshan
Indexing of Documents
 Documents that contain a specified keyword can be
located using an inverted index which maps each
keyword Ki to the set Si of identifiers of documents that
contain Ki .
 IR systems save space by using index structures that
support only approximate retrieval. May result in:
 false drop - some relevant documents may not be
retrieved.
 false positive - some irrelevant documents may be
retrieved.
 For many applications a good index should not permit any
false drops, but may permit a few false positives.
 Relevant performance metrics:
 Precision - what percentage of the retrieved documents are
relevant to the query.
 Recall - what percentage of the documents relevant to the
query were retrieved.
Database System Concepts 3rd Edition
21.59
59
©Silberschatz, Korth and Sudarshan
Indexing of Documents (Cont.)
 and operation: Finds documents that contain all of a set of
keywords K1, K2, ..., Kn.
 Retrieve the corresponding sets of identifiers of documents
S1, S2, ... Sn.
 The intersection, S1 S2 .....  Sn, gives the identifiers of the
desired set of documents.
 or operation: Gives the set of all documents that contain at least
one of the keywords K1, K2, …, Kn
 Found by computing the union, S1 S2 .....  Sn, of the sets.
60
Database System Concepts 3rd Edition
21.60
©Silberschatz, Korth and Sudarshan
Indexing of Documents (Cont.)
 not operation: Finds documents that do not contain a
specified keyword Ki
 Let Si be the set of identifiers of documents that contain the
keyword Ki.
 Given a set of document identifies S, eliminate documents
that contain the specified keyword Ki by taking the difference
S-Si
 A full-text index uses every work in the document as a
keyword.
 Stop words are very commonly occurring words that are
useless as key words, e.g, a, an, the, it etc. These are
eliminated from the index.
61
Database System Concepts 3rd Edition
21.61
©Silberschatz, Korth and Sudarshan
Browsing
 Storing related documents together facilitates browsing,
where a user can see not only requested document but
also related ones.
 Browsing in a library facilitated by classification system
that organizes logically related books together.
62
Database System Concepts 3rd Edition
21.62
©Silberschatz, Korth and Sudarshan
A Classification Tree
63
Database System Concepts 3rd Edition
21.63
©Silberschatz, Korth and Sudarshan
Classification DAG
 Documents can reside in multiple places in a hierarchy
in an information retrieval system, since physical
location is not important.
 Classification hierarchy is thus Directed Acyclic Graph
(DAG).
64
Database System Concepts 3rd Edition
21.64
©Silberschatz, Korth and Sudarshan
65
Database System Concepts 3rd Edition
21.65
©Silberschatz, Korth and Sudarshan
Hypertext
 Hypertext systems take the idea of storing document
identifiers or pointers and provide a facility where the user
can easily switch from one document to another.
 Typically use a point-and-click interface, where a simple
mouse click on the display screen on top of the referred
document retrieves and displays it.
 Hypermedia systems provide not only text, but also other
media such as images, videos and audio clips.
 Distributed hypertext systems permit references to
documents stored at other sites in a distributed system such
as the World Wide Web.
66
Database System Concepts 3rd Edition
21.66
©Silberschatz, Korth and Sudarshan
Distributed Information Systems Overview
 Distributed information systems running on the Internet have
seen explosive growth in recent years.
 Early distributed information systems include the Gopher and
WAIS
 The widely used World Wide Web system (for short, the
Web) supports
 browsing of information using the distributed hypertext
paradigm.
 standardized ways of accessing data and standardized GUIs.
 There are automated tools for locating and indexing
information in such distributed heterogeneous system. Still an
area of ongoing research and development.
67
Database System Concepts 3rd Edition
21.67
©Silberschatz, Korth and Sudarshan
The Gopher and WAIS
 A Gopher system consists of servers and clients.
 Servers organizes data into directories.
 Client initially communicates with a server; the top level directory
of the server hierarchy is displayed as a menu.
 Menu item can be another directory in the hierarchy, a document,
or a link to a directory on another server.
 Allow a seamless connection to remote servers.
 Information retrieval in Gopher is based on browsing and
navigating a directory hierarchy.
 Wide Area Information System (WAIS), retrieves information via
a powerful keyword based indexing mechanism.
 Each WAIS site maintains a site description; describes the kind
of information stored at other sites, and how to access them.
68
Database System Concepts 3rd Edition
21.68
©Silberschatz, Korth and Sudarshan
Sample Gopher Client Display
Internet Gopher Information Client 2.0 pl 8
 University of XYZ Computer Science Department Gopher
1. WELCOME
2. Information About Gopher/
3. About the University of XYZ CS Department
4. University of XYZ Campus Information/
5. Search Gopher Titles at the Univ.. of Minnesota < ?>
6. University of Minnesota Gopher (Gopher Central) /
7. Libraries/
8. Phone Books/
9. Other Gopher and Information Servers/
10. Internet file server (ftp) sites/
69
Database System Concepts 3rd Edition
21.69
©Silberschatz, Korth and Sudarshan
The World Wide Web
 The Web is a distributed information system based on
hypertext.
 Most Web documents are hypertext documents formatted via
the HyperText Markup Language (HTML), which is based on
the Standard Generalized Markup Language (SGML).
 HTML documents contain
 text along with font specifications, and other formatting
instructions
 hypertext links to other documents, which can be associated with
regions of the text.
 When the document is displayed on a browser, the user can
click on a region that has a link associated with it; the
document pointed to by the link is then displayed.
70
Database System Concepts 3rd Edition
21.70
©Silberschatz, Korth and Sudarshan
Universal Resources Locators
 In the Web, functionality of pointers is provides by
Universal Resource Locators (URLs).
 URL example:

http://www.belllabs.com/topic/book/db-book
 The first part indicates how the document is to be accessed;
“http” indicates that the document is to be accessed using
the Hyper Text Transfer Protocol.
 The second part gives the unique name of a machine on the
Internet.
 The rest of the URL is the path name of the file on the
machine.
71
Database System Concepts 3rd Edition
21.71
©Silberschatz, Korth and Sudarshan
Sample HTML Source Text
<title> www Home Page </title>
<img src=“lucent_logo.gif”> <img src=“iitb-logo.gif”>
<hr>
<h3> <a href=
“http://www.bell-labs.com/topic/books/db-book”>
Database System Concepts: Home Page </a>
</h3>
<h3> Information Services </h3>
<d1>
<dd> <a href=:http://www.altavista.digital.com”>
The Altavista Information Gathering Service</a>
<dd> <a href=
“http://www.Informatik.uni-trier.de/~ley/db”>
Database and Logic Programming Bibliography</a>
<dd> <a href=:http://cs.indiana.edu/cstr/search”>
Indiana University’s Unified
CS Tech. Report Service</a>
<dd> <a href=“http://www.ncstrl.org”>
Networked Computer Science Tech. Report Library</a>
</d1>
Database System Concepts 3rd Edition
21.72
72
©Silberschatz, Korth and Sudarshan
Display of HTML Source from Previous Slide
73
Database System Concepts 3rd Edition
21.73
©Silberschatz, Korth and Sudarshan
Web Servers
 A Web server can easily serve as a front end to a variety of
information services.
 The document name in a URL may identify an executable
program, that, when run, generates a HTML document.
 When a HTTP server receives a request for such a
document, it executes the program, and sends back the
HTML document that is generated.
 The Web client can pass extra arguments with the name of
the document.
 To install a new service on the Web, one simply needs to
create and install an executable that provides that service.
 The HTML language supported by the Web provides a
graphical user interface to the information service.
74
Database System Concepts 3rd Edition
21.74
©Silberschatz, Korth and Sudarshan
Display Languages
 Text markup languages such as the Standard Generalized
Markup Language (SGML) fill a void between plain text
and page description/text formatting languages.
 SGM L provides grammar for specifying document formats
based on standard markup annotations.
 HTML provides formatting, hypertext link, and image
display features.
 HTML also provides some input features
 Values can be input via menus of various types `(pop up,
radio buttons, check lists, etc.)
 Values can also be entered via text fields
 HTTP provides methods to send filled in input back to the
server, to be acted upon by an executable at the server
75
Database System Concepts 3rd Edition
21.75
©Silberschatz, Korth and Sudarshan
Java
 Java language allows documents to be active (e.g.,
animation by executing programs at the local site).
 Java programs can be stored at server sites (like HTML
documents), and can be download and executed by any
client site.
 permits flexible interaction with the user.
 Executing programs at the client site speeds up interaction
greatly, compared to every interaction being set to a server
site for processing.
 Java’s security system ensures that the Java code does
not make any system calls directly
 Notifies the user about potentially dangerous actions, such
as file writes, and allows the option to abort the program or
to continue execution.
76
Database System Concepts 3rd Edition
21.76
©Silberschatz, Korth and Sudarshan
Web Interfaces to Databases
 Extremely useful to link transaction processing databases
with the Web.
 Example: Information filled in on an HTML order form can be
executed as a database transaction; the results can be
formatted into HTML and displayed to the user.
 Fixed HTML sources for display to users have limitations:
 Cannot customize fixed Web documents for individual users.
 Problematic to update Web documents, especially if multiple
Web documents replicate data.
 Solution: Generate Web documents dynamically using data
in a database
77
Database System Concepts 3rd Edition
21.77
©Silberschatz, Korth and Sudarshan
Web Interfaces to Database (Cont.)
 Dynamic generation of Web documents:
 Can tailor the display based on user information stored in the
database.
 Displayed information is up-to-date, unlike the static Web
pages.
 Web interfaces to databases simplify the task of connecting
a database to the Web
 Define a HTML document in a macro language with embedded
SQL queries.
 Variables defined in HTML forms can be used directly in the
embedded SQL queries.
 When the document is requested, a macro processor executes
the SQL queries, and generates the actual HTML document
that is sent to the user.
78
Database System Concepts 3rd Edition
21.78
©Silberschatz, Korth and Sudarshan
Locating of Information on the Web
 The Archie system automatically follows Gopher links to
locate information, and creates a centralized index of
information found various sites.
 Web indexing systems (Web crawlers) follow the
hypertext links in documents to find other documents,
and build an index on the documents.
 The indices are often full-text indices, and are stored
locally at the indexing system.
 These systems run a background process to
 find new sites.
 obtain updated information from known sites.
 discard defunct sites.
79
Database System Concepts 3rd Edition
21.79
©Silberschatz, Korth and Sudarshan
Locating of Information on the Web (Cont.)
 Web indexing systems permit documents to be located
even though they are not registered with any central
authority.
 Drawback: poor precision of recall
 Full text indexing retrieves unrelated documents that just
happen to mention the requested keyword
 HTML extensions now allow documents to be tagged with
keywords to be used by search engines; unfortunately,
many documents do not provide such keywords.
 The extremely large number of documents on the Web
often leads to far more results than a human can handle.
 Alternative approach: a cataloging system for the Web,
such as that provide by Yahoo
 Combinations of catalogs and indexing are quite useful
Provided by, e.g., Yahoo (www.yahoo.com).
80
Database System Concepts 3rd Edition
21.80
©Silberschatz, Korth and Sudarshan