Transcript Foundational Software Systems

Foundational Software
Systems
A Brief Overview
Copyright © 1998-2016 Curt Hill
Introduction
• Certain classes of software need to
be understood by all technical
people
• Operating Systems
• Servers
• Databases and Data Warehouses
• Enterprise Resource Planning
• Customer Relationship Management
Copyright © 1998-2016 Curt Hill
OS
• Foundation of the foundation
• Every general purpose computer
has one
– Many special purpose computers as
well
• Applications are why we buy the
computer, but the operating system
is required to run the applications
Copyright © 1998-2016 Curt Hill
What does an OS do?
•
•
•
•
•
Execute application programs
Provide services to these programs
Establish and manage a file system
Utility functions
Many functions that are used by
most programs will eventually be
incorporated into the OS
Copyright © 1998-2016 Curt Hill
Execute application
programs
• Interpret commands
– Interpret typed commands or mouse
actions
• Load and execute programs
• Provide services to the running
programs
– The exact description of these services
distinguishes different OSs
Copyright © 1998-2016 Curt Hill
Establish and manage a file
system
• Determine how things are stored on
disk
– Do we have a directory or just search?
– Are things stored in a contiguous
fashion or scattered?
– How are things named?
– How are things accessed?
• Interrupt processing
• Utility functions
– Prepare a disk for use
– Copy/move/delete files
– Show contents of a file or disk
Copyright © 1998-2016 Curt Hill
Utility functions
• Protection
– Files from deletion by someone other
than owner
– Memory of one program from another
• Hardware manipulation
– Mount disks and tapes
• Miscellaneous
– Error detection
– Multitasking
– Accounting
Copyright © 1998-2016 Curt Hill
OS Layers
• Most OSs are onion shaped
– For historical and maintenance reasons
– Historically the innermost was all that
was available
• Each layer then communicates with
adjacent layers through well defined
paths
– Well defined in terms of this particular
OS
• Layers: Kernel, Service, Command
Copyright © 1998-2016 Curt Hill
Familiar OSs
• Windows
– Several versions from personal to
server
• UNIX
– Bell Labs and BSD
• Mac OS
– OS 10 is a BSD UNIX with specialized
Mac GUI as command layer
• LINUX
– UNIX like alternative to UNIX
Copyright © 1998-2016 Curt Hill
The strange history of UNIX
• Started at Bell Labs in early 1970s
• Most influential operating system in
history
• Bell Labs gave it away
– They did not think it was marketable
– It was not in their business model
• Later they decided this was a
mistake
– They attempted to assert control over it
Copyright © 1998-2016 Curt Hill
Rounding them up
• Some organizations decided to pay
Bell Labs royalties for their version
of UNIX
– Sun – Solaris
– IBM – AIX
– HP – HPUX
• Others rebelled
– Linus Torvalds
– UC Berkeley sued
Copyright © 1998-2016 Curt Hill
BSD and LINUX
• UC Berkeley sued saying that you
cannot repossess what was given
away
– They won
– Now UNIXs are either Berkeley
Software Distribution (BSD) or Bell
Labs
• Torvalds rewrote the UNIX kernel
– LINUX was born
– Always open source
Copyright © 1998-2016 Curt Hill
LINUX Again
• LINUX is open source so there are
many distributions of it:
–
–
–
–
–
Debian
Ubuntu
Fedora
SUSE
Red Hat Enterprise
• There are very many others
Copyright © 1998-2016 Curt Hill
Applications and the OS
• Each OS has its own interface to
programs
– Even if running on the same hardware
• Thus an application is dependent on
the OS
– Can only run on one particular family of
OS
• Popular programs, such as
Microsoft Office Suite have multiple
versions for each OS they support
Copyright © 1998-2016 Curt Hill
Lifetime
• Like all software and OS has a
limited lifetime
• A commercial OS will have a period
where it will be actively supported
– When it exceeds that it becomes more
vulnerable to malware
• Usually we replace the hardware
first
• Using an obsolete OS invites
problems
Copyright © 1998-2016 Curt Hill
Servers
• Client server computing is the basis
of the Internet
– Among other things
• Requires two computers
– A client
– A server
• Communication medium
• The illusion is that each client
computer thinks it is has exclusive
access to the server
Copyright © 1998-2016 Curt Hill
The Paradigm
•
•
•
•
•
Server provides one service
Waits for requests for that service
Client initiates a request
Server replies to the request
Information may flow in both
directions
• Many clients may make requests
with just the one server
Copyright © 1998-2016 Curt Hill
Why Client - Server?
• Specialization
– The client handles the user interface
using local computational ability
– The server manages the data and
communication
• Sharing
– One server may support many clients
– Services provided are effectively
shared
– Data may be maintained in only one
location Copyright © 1998-2016 Curt Hill
Clients
• An application program that runs on
the local computer
• Able to use the local computer for
doing work
• When invoked by a user, it initiates
communication with a server
• Handles all the user interface issues
• Only used for a single session
Copyright © 1998-2016 Curt Hill
Example
• The ubiquitous client is the web
browser
• Several functions:
• Communicate with server
• Display HTML/XML
• Interpret client-side languages,
such as JavaScript
• The trend is for the web browser to
be the universal interface to every
kind of server on the web
Copyright © 1998-2016 Curt Hill
Servers
• Specialized program that typically
provides only one service
• May handle multiple client requests
simultaneously (or pseudo
simultaneously)
• Interacts with users in many sessions
– Both simultaneously and over long time
periods
• Waits for a request and then produces the
reply
Copyright © 1998-2016 Curt Hill
Example
• The web server
• Waits for requests using Hyper Text
Transmission Protocol
• Produces results in HTML/XML
• May handle many clients
simultaneously
Copyright © 1998-2016 Curt Hill
Connection Diagrammed
Server
Request
Request
Client
Reply Reply Request
Client
Copyright © 1998-2016 Curt Hill
Reply
Client
Interaction Diagrammed
Time
Client
Processing
Request
Wait
Reply
Server
Wait
Wait
• The server is not waiting if it is
processing other requests
• The client may also perform other
processing while waiting for the
reply
Copyright © 1998-2016 Curt Hill
Multiple tiers
• The classic client-server is a two tier
system
• The client is one tier and the server
another
• Three or more tiers are also
possible
• The middle server functions as a
– Server to the client
– Client to other servers
– Known as middleware
• Often used to distribute load
Copyright © 1998-2016 Curt Hill
A Three Tier Approach
Server
Server
Server
Server
Application Server
Client
Client
Client
Copyright © 1998-2016 Curt Hill
Client
Examples of Multi-Tier
• Authentication server
– Maintains a connection to verify user
– Funnels all requests to appropriate
server
• Shopping cart
– Records purchases
– The product lookup/purchase handled
by other servers
• Distributed Database Management
System
Copyright © 1998-2016 Curt Hill
Horizontal Distribution
• Multi-Tier is a vertical distribution
• A Horizontal distribution may also be
effective
• A single server owns the web
address: www.x.com
• It then distributes all further
requests to a rotation of identical
web servers
–
–
–
–
www1.x.com
www2.x.com
…
wwwN.x.com
Copyright © 1998-2016 Curt Hill
Horizontal Distribution
www.x
First
Request
www1.x www2.x
… wwwN.x
Subsequent
Request
Client Client Client Client Client Client
Copyright © 1998-2016 Curt Hill
Database
• The web server may be the most
common server type, but the
database server is the workhorse
• It is the foundation for many other
important classes of programs
Copyright © 1998-2016 Curt Hill
Historical Perspective
• Actions we might want on any file of fixed
records:
–
–
–
–
Create
Update (add, remove, change records)
Sort
Generate any of several reports
• Each of these would be a program for an
overworked programming staff
– Typically a COBOL program
Copyright © 1998-2016 Curt Hill
Database
• The key
– All these programs are about the same
– All that changes is the underlying file
• The solution
– Describe the file in a general way
• This is the metadata – data that describes
data
– Generate a program that handles the
file based on the description
Copyright © 1998-2016 Curt Hill
How to describe a file
• A file is a collection of records
• Each record is a collection of fields
– Typically only one type of record in a
file
• Each field is described by a:
– Name
– Type
• For example numeric, string, boolean etc.
– Length
• Booleans have a predefined length, others
require specification
Copyright © 1998-2016 Curt Hill
Database Advantages 1
• Data independence
– Application program no longer need
some or all of the files
– Do not know or care how data is stored,
aka abstraction
– Simplifies application development
• Efficient access
– The DBMS employs sophisticated
access techniques seldom used by
normal programmers
Copyright © 1998-2016 Curt Hill
Database Advantages 2
• Integrity constraints
– The DBMS may check data in a way
seldom done in normal file processing
– Eg. Account validity
• Security
– A DBMS may enforce requirements on
who can access the data and in what
way
Copyright © 1998-2016 Curt Hill
Database Advantages 3
• Administration
– Minimize redundancy
– Manage sharing of the data
– Optimize for the enterprise, not a small
group
– Easier to backup the data
• Concurrent access
– Manages the simultaneous update
problem
Copyright © 1998-2016 Curt Hill
Functions of a Database
•
•
•
•
•
•
•
•
•
Data storage and retrieval
User-accessible catalog
Transaction support
Concurrency support
Recovery services
Authorization services
Support for data communication
Integrity services
Data independence services
Copyright © 1998-2016 Curt Hill
Data storage and retrieval
• While hiding the file structure, users
may:
– Add
– Delete
– Update in place
• Most relational databases support a
common language
– Structured Query Language
Copyright © 1998-2016 Curt Hill
SQL
• A language where a question is
asked of the database and it
provides an answer:
– Pronounced Sequel or spelled out
– Usually state what is wanted rather
than how to get it
• The Data Manipulation Language
usually includes both query and
update facilities
Copyright © 1998-2016 Curt Hill
Transaction Support
• The unit of update is a transaction
– All or nothing
– No partial updates are allowed
• The DBMS must guarantee the
integrity of a transaction, even in the
face of:
– Program errors
– Incorrect input
– System crashes
Copyright © 1998-2016 Curt Hill
Transactions: ACID
• Atomic
– Transaction perceived to be indivisible
• Consistent
– Transforms database from one consistent
state to another
• Isolated
– Understandable without regards to any
other agents
• Durable
– Once committed permanence is
guaranteed even with system crashes
Copyright © 1998-2016 Curt Hill
Types of database
• A large organization often have two
distinct types of databases
• A transactional or operational
database
• An analytical database or data
warehouse
• NoSQL
• These have separate uses
Copyright © 1998-2016 Curt Hill
Database uses
• The database is about transactions
– Maintaining current knowledge
• The data warehouse is about
decision support
– Spotting trends
• NoSQL is much harder to classify,
any of these
–
–
–
–
Massive data
Unique organizations
Distributed
Among many others
Copyright © 1998-2016 Curt Hill
Characteristics
• Operational database have:
–
–
–
–
Strict performance requirements
Predictable workloads
Small units of work
High utilization
• Data warehouses are contrary in
every respect
Copyright © 1998-2016 Curt Hill
Warehouses
• A data warehouse
– Subject oriented, integrated, nonvolatile, time-variant collection of data
in support of management decsions
• Data warehouses support
– OLAP (OnLine Analytical Processing)
– DSS/EIS (Decision Support Systems or
Executive Information Systems)
– Data mining
Copyright © 1998-2016 Curt Hill
What is Data Mining?
• Exploratory data analysis based on a data
warehouse
– Knowledge Discovery in Databases (KDD)
• Data Mining extracts previously unknown
and potentially useful information
– Rules, constraints, correlations, patterns,
signatures and irregularities
• The goal is to automate the methods for
finding these in the data
Copyright © 1998-2016 Curt Hill
Goals of Data Mining
• Prediction of future behaviors
– Seasonal or non-seasonal trends
– How will consumers respond to
discounts?
– Allows the enterprise to be ready
• Identification of item, event or
activity
– Intruders may be identified by the files
they access or programs they use
Copyright © 1998-2016 Curt Hill
Goals Again
• Classification of categories of users
or products
– Shoppers may be categorized as:
•
•
•
•
Discount seeking
Rush
Regular
Attached to certain brand names
– The store may be made more friendly to
such
• Optimize the use of time, space,
materials and
money
Copyright © 1998-2016 Curt Hill
Knowledge Discovery
• There are several types of
discoverable knowledge:
• Association Rules
• Classification hierarchies
• Sequential patterns
• Time series patterns
• Clustering
Copyright © 1998-2016 Curt Hill
Databases
• The two types of databases are the
transactional and analytical
– The day to day transactional feeds the
analytical
• We do not want knowledge
discovery, data mining and the like to
occur on the transactional database
– Could seriously impede performance
• Either might be used in the next two
applications
– Transactional is more likely
Copyright © 1998-2016 Curt Hill
• Data
Some Definitions
– Small measurable pieces of information
– Foundation of knowledge
• Information
– Organized data
– Putting data into meaningful patterns
• Knowledge
– A higher aggregation of information
and data that enables interpretation
– Ability to use information
– Sets of rules and relationships
Copyright © 1998-2016 Curt Hill
Example
• Any numbers or text is data
– Such as: 324 3.83
• It becomes information when we
know its context
– Such as these are the number of items
in stock and how much we paid per unit
• Knowledge is when we take this
information and combine it with
processes and rules
– Computing our price for selling
something made out of these
Copyright © 1998-2016 Curt Hill
Data Mart
• A data warehouse restricted to a
single subject
– Such as a group or line of business
within a larger organization
• A dependent data mart obtains its
data from the central organizational
data warehouse
• An independent data mart obtains
data from other sources
– No overlap with the data warehouse
Copyright © 1998-2016 Curt Hill
Knowledge Warehouse
• An architectural integration of
knowledge management, decision
support, artificial intelligence and
data warehousing
• Built on top of a data warehouse
• Must also encode the rules and
processes of the business
– These rules used to only be in the
minds of employees
– Next captured on documents
Copyright © 1998-2016 Curt Hill
ERP
• Enterprise Resource Planning
• The key idea behind database
technology is that the process of
manipulating files is basically the
same
– The only thing that changes is the file
format
• The key to ERP is that many
business processes are very similar
– Payroll is just payroll
• Took off because of the Euro and
Y2K threatCopyright © 1998-2016 Curt Hill
ERP Areas
•
•
•
•
•
•
•
•
Financial Accounting
Management Accounting
Human Resources
Manufacturing
Supply Chain Management
Project Management
Customer Relationship Management
These were all separate programs at
one time
Copyright © 1998-2016 Curt Hill
Configuration
• Most systems need to be customized
by programmers modifying the code
• Most ERPs may be configured to
customize to a particular situation
• This configuration is usually much
easier than code modification
• The program typically handles best
practices, but the configuration can
take into account the idiosyncrasies
of this business
Copyright © 1998-2016 Curt Hill
Simple Example
• The form of a product number
drastically varies between
companies
– Usually groups of letters and digits
• Configuring the system to store and
validate the type used by a
particular company needs to be a
simple configuration
Copyright © 1998-2016 Curt Hill
CRM
• Customer Relationship Management
• The notion of a call center
• A past, present or future customer
calls
• A representative, with no previous
knowledge of this customer, has to
deal with them
• CRM puts all the information
necessary to help in front of the
representative, regardless of the
purpose of the call
Copyright © 1998-2016 Curt Hill
Types of calls
• Sales
– Availability, delivery, costs and
discounts
• Support after the sale
– Warranty, replacements, returns
• Information and marketing
– Request for information
• Appointments for sales force
• Analysis of customer or products
Copyright © 1998-2016 Curt Hill
Implementation
• The leading CRMs, like ERPs are
highly configurable
• Also like ERPs they are backed by a
transactional database
• Both generate large quantities of
SQL on the fly
• ERPs are starting to absorb CRMs
Copyright © 1998-2016 Curt Hill
Finally
• The type or size of businesses that
would be architected will likely have
all of these
• They will also have serious
customizations to these products
• Many other specialized applications
also exist
– EG. EA tools
Copyright © 1998-2016 Curt Hill