Transcript Why art is needed for software engineering

Why Art is Needed
for Software Development
Le développement de logiciel,
est-ce un art ?
Gregor v. Bochmann
School of Information Technology and Engineering (SITE)
University of Ottawa
September 26, 2002
My personal background
• MSc in physics and mathematics; PhD in theoretical physics;
experience in computer programming; changed to computer
science in 1971 (after PhD), professor at University of Montreal
for 25 years
• People important for me
– My mother was high-school teacher for music and visual arts
(my father, biologist, died in 1942 during the war)
– The grandfathers of my father were both well-known artists
– My high-school physics and mathematics teachers
– My piano teacher
• Hobbies
– Travelling, hiking, observing nature
– Playing music: chamber music, orchestra
– Setting up an inventory of the paintings of my great-grandfather
(see http://beethoven.site.uottawa.ca/MalerBochmann/ )
About this talk
• Theme:
Art
artisan, artist


Software Development
software engineer
Historical development
– artisanal economy
– first industrial revolution: energy and automation
– second industrial revolution: information processing
Artisans, artists, software engineers
– similarities and differences: the importance of inventions
The activities of a software engineer
Automation of software engineering
– research objectives
– intrinsic limitations (the importance of inventions)
Some comments on my research contributions
Conclusions
Art

Software Development
artisan, artist

software engineer
What do these people do ?
– build something useful
– Artisan (shoe maker makes shoes)
– Engineer (mechanical engineer designs a bridge)
– learn the rules of the profession
– Shoe maker: how to make shoes
– Engineer: physical laws concerning statics and dynamic mechanics
– Artist (composer): rules of harmony
– produce something beautiful
– elegant shoes
– nice looking bridge
– beautiful music
– Artists: convey a sentimental or social message
– Mozart’s piano concert or Zola’s book
Economy based on artisans
Example of a shoe maker
– builds something useful
– notes the client’s requirements: size, style of shoes, desired
color, etc.
– follows the rules of the profession
– selects the material to be used for different parts of the shoe
– knows how to fabricate these parts
– knows the best order in which to assemble them (this may be
adapted to the particular circumstances – inventions)
– produces something beautiful
– adds on some ornamentation – inventions (as requested by
the client and/or according to his own invention and
temperament)
Typically, each object is individually designed
and fabricated
First industrial revolution
Characteristics
•
•
•
Energy replaces man or horse power
Chemical engineering: better materials
Automated manufacturing (e.g. assembly lines for cars,
automatic weaving, etc.)
People’s activities:
1. Design product to be manufactured (inventions)
2. Design and build machines for manufacturing (inventions)
3. Assist the machines on the assembly line
Note: This is boring ! -- Why ?
Typically, many identical objects are manufactured
for each design
Second industrial revolution (IT)
Relative production cost decreases
first unit
–
–
–
–
–
–
additional unit
House
Plane
Vinyl music album
MP3 music album
Income tax program
New e-commerce appl.
Software and digital media: no copying costs
Consequences and open questions
• Only activity (1) remains (to design the product
-- need
for inventions)
• Free copying (e.g. Napster) vs. Copyright
– or ”Open Source Software” ??
– Providing useful software components without remuneration
• Globalization and uniformization …
– or sub-cultures ??
• Do we need more software engineers ?
– No: software for all useful applications will soon be available
– Yes: the power of computer and communication hardware increases
exponentially over time (Morse’s law); therefore many new
applications will be possible in the future and they must be developed
Back to:
art – science – engineering
useful – rules – beautiful – message
• Simplicity – Beauty (or Triviality ?)
–
–
–
–
Nature
Mathematics
Paintings: too simple becomes trivial
Science: to find simple explanations for nature’s behavior; this is
never trivial
– Engineering: simple, elegant designs are easier to verify, build
• Rules vs. Freedom
– The free space allows the expression of the artist’s message
– This is the space for inventions
» Bach’s “Inventions“
» Communication through human or computer language
» Engineering: design choices
– Filling the free space with random choices (e.g. automatic art)
Shoe Maker vs. Softw. Engineer
– builds something useful
• notes the client’s requirements: size, style of shoes, desired color, etc.
• notes the requirements of the system to be built
– follows the rules of the profession
•
•
•
•
•
•
selects the material to be used for different parts of the shoe
selects software components to be re-used
knows how to fabricate these parts
knows techniques for software design, implemention and testing
knows the best order in which to assemble them
follows a suitable SE process
– produces something beautiful
• adds on some ornamentation (as requested by the client and/or according to
his own invention and temperament)
• tries to built the system in a simple manner (as far as possible)
• a program can be elegant, beautiful ! ?
• where requirements leave freedom, designs a nice user interface and user
options
SE : an artisanal profession ?
. . . as indicated by above comparison
• Most software has too many bugs !!
• Much software is created
– without a well-defined development process
– without the use of automated tools
• Sometimes, older disciplines look down to SE and computer science, which is a
relatively young discipline
– Each car manufacturer has a well-defined and optimized manufacturing process
– There is automation not only in the manufacturing process, but also during the design
(e.g. simulation tools, static analysis packages, etc.)
• Much of my research: towards automation of SE
• Can the SE process be completely automated ?
Activities
An activity (in general)
processing
input
output
IT activities
Algorithm: “process or rules for calculation” (Concise Oxford Dict.)
Program: algorithm coded in a language executable by a computer, or
automatically translatable by a compiler into executable code
Example: The activity of sorting
Fred
Tom
Alice
Suzanne
12 000
7 000
10 000
5 000
input
sorting
algorithm
output
Alice
Fred
Suzanne
Tom
10 000
12 000
5 000
7 000
Requirements (properties of output, depending on input):
(1) output list is sorted by names (increasing order)
(2) the set of output lines should be equal to the set of input lines
Example of a sorting algorithm
start
create empty
output list
is input list
empty ?
yes
no
extract an element
from the input list;
call it E
insert E into
the output list
at the appropriate place
end
Activities of a programmer-analyst
Requirements
Requirements
input
input
Find (invent)
appropriate algorithm
and describe it
Find appropriate
decomposition, reusing
if possible existing modules
output
output
Algorithm
A
B
C
Requirements
input
Algorithm or
composition
Algorithm
(in executable
specification language)
input
Verification
(does the algor. or composition
output
satisfy requirements ?)
Translation
from high-level language
to executable code
output
“correct” or
“in error”
executable
code
Some comments
Requirements
input
Find (invent)
appropriate algorithm
and describe it
Algorithm
output
input
if possible existing modules
reusable components
distributed applications,
communication protocols
Try to establish
Find appropriate
decomposition, reusing
Requirements
Can these activities be
automated?
A
B
output
C
Requirements
input
output
Algorithm or
composition
Algorithm
(in executable
specification language)
“correct” or
“in error”
Verification
(does the algor. or composition
satisfy the required properties ?)
input
Automatic translation
from high-level language
to executable code
Compiler
executable
code
output
Can one find algorithms for
automating the verification
process ? (see next slides)
Completely automated: Compilers exist
for programming languages and
certain executable specification
languages
The “verification” activity
Requirements
input
Algorithm or
composition including
algorithms
Verification
(does the algorithms satisfy
the required properties ?)
output
“correct” or
“in error”
Can the solution of this problem be automated ?
Can one find an algorithm that solves this problem ?
Two approaches: proving – testing
- try to make a proof of the correctness
- execute the given algorithms with various inputs (test inputs)
and check whether the obtained outputs satisfy the requirements
Verification: … towards a proof
• Different levels of formalization
– How are the requirements and the algorithm defined ?
• Intuition, informal description / documentation, formal
specification
• Undecidable problems
– Important result of computer science: Certain problems
are undecidable (it is known that no algorithm exists that
would solve them !!)
• The verification problem is undecidable
• Simplified cases of the verification problem are decidable
– e.g. a proof (prepared by human) of the correctness of some algorithm may be
verified by a proof checking algorithm
Verification: … through testing
Approach: To execute the program (or to simulate the execution of an
algorithm) with particular inputs and to verify that the obtained outputs
satisfy the requirements
Difficulty: If one does not test for all possible input sequences
(exhaustive testing) there is no assurance that all errors are found.
Exhaustive testing is normally not possible.
Example: Test of a telephone switch supporting N lines (the case of M
simultaneous connect requests to various destinations):
I = (N)M (number of cases)
M=N = 4: I = 44 = 256
M=N=10: I = 1010
M=N=20: I = 2020 = 1026
Note: Many practical problems have an exponential complexity, that is, the
execution time depends exponentially on the size of the input data (here
the size of the switch)
Formal methods and related tools
Specification languages
human creativity
Find (invent)
appropriate algorithm
and describe it
Requirements
input
Algorithm
output
Find appropriate
decomposition, reusing
Requirements
A
B
if possible existing modules
input
output
C
based on predicate calculus: e.g. Z, for assertions
about inputs and outputs, invariant assertions
(for requirements)
based on state machines (e.g. SDL) or process
algebras (e.g. LOTOS); executable
specification languages (for algorithms)
Algorithms for the derivation of protocol
specifications
Equation solving (given A and C, find B)
Tools for verification
Requirements
“correct” or
“in error”
Verification
input
Algorithm or
executable program
(does the algorithm satisfy
the required properties ?)
output
Semi-automatic theorem provers
Automated tools for exhaustive exploration of
reduced finite state models (model checking)
Simulated execution environments for composed
systems
Tools for test development with guaranteed
coverage criteria (non-exhaustive)
My own contributions (in red)
Development of state machine
languages, including
standardization of the Estelle
language (similar to SDL and
UML state charts): definition,
execution environments, testing,
application to communication
protocol standards
Seminal work on protocol derivation
and equation solving
Methods and tools for test
development (in the context of an
industrial research chair at the
University of Montreal with
Hewlett-Packard, 1989-97)
Now: Also applications in the area of
distributed multimedia systems
(teleconferencing, IP telephony)
and photonic networks
Specification languages
based on predicate calculus: e.g. Z, for assertions
about inputs and outputs, invariant assertions
(for requirements)
based on state machines (e.g. SDL and Estelle) or
process algebras (e.g. LOTOS); executable
specification languages (for algorithms)
Algorithms for the derivation of protocol
specifications
Equation solving (given A and C, find B)
Tools for verification
Semi-automatic theorem provers
Automated tools for exhaustive exploration of
reduced finite state models
Simulated execution environments for composed
systems
Tools for test development with guaranteed
coverage criteria (non-exhaustive)
Conclusions
• Nature of software development
– To build according to requirements (somethings useful)
– To follow rules of best practices; note that most activities cannot be
completely automated since the problems are either undecidable or of
exponential complexity
– To invent new solutions – freedom of choice – creativity
– de la programmation considérée comme un des beaux-arts (Pierre Lévy, 1992)
• Need for software engineers with
– good communication skills to understand and to explain the problem and
its solution
– creativity to find appropriate system architectures, components and algorithms (
t( this is an
Art )t )
– sense of rigor to verify that the constructed software satisfies the requirements
and does not contain errors ( an
• Ongoing research tries to further automate the SE process, but it is
not clear how far this process may go, given the inherent
limitations