Transcript Concurrent Functional Programming with Erlang and OTP (Open

Concurrent Functional Programming with
Erlang and OTP (Open Telecom Platform)
Bjarne Däcker
<[email protected]>
Computer Science Laboratory
Ericsson Utvecklings AB
Acknowledgements
Thomas Arts <[email protected]>
Hans Nilsson <[email protected]>
Torbjörn Keisu <[email protected]>
Ulf Wiger <[email protected]>
2001-03-19 ETX/D/XPS-01:002 Uen 1
The setting
 1995 PC Week Study of software projects:
– 16% successful
– 53% operational (but less than successful)
– 31% cancelled
 Butler Group 1997 on large software projects
– 5 out of 6 large projects fail
– In >$300M companies, 9 out of 10 large projects fail
 How to approach this?
– Use high-level modeling tools & generate code?
– Raise the level of programming language?
– Fight all causes for project failure!
2001-03-19 ETX/D/XPS-01:002 Uen 2
Telecom industry
 Switches, routers,
base-stations
 Networks
 Mobile telephones
2001-03-19 ETX/D/XPS-01:002 Uen 3
Requirements on a Programming Technology
for Telecommunication Switching Systems
 Massive concurrency
 Soft realtime
 Distribution
 Interaction with hardware
 Very large software systems
 Complex functionality
 Continuous operation for many years
 Software maintenance without stopping the system
 Stringent quality and reliability requirements
 Fault tolerance errors
2001-03-19 ETX/D/XPS-01:002 Uen 4
History of Erlang
1984:
Ericsson
Computer
Science Lab
formed
1984-86:
Experiments
programming
POTS with
several languages
No language well suited
for telecom systems
development
1998:
Open Source
Erlang
1991:
First fast
implementation
1987:
Early Erlang
Prototype projects
1996:
Open Telecom Platform
(research on verification...)
1993:
Distributed
Erlang
1995:
Several
new projects
2001-03-19 ETX/D/XPS-01:002 Uen 5
The Ancestry of Erlang
Concurrent systems
programming languages
like Ada, Modula or Chill
Functional programming
languages like ML or
Miranda
Concurrent functional
programming language
Erlang
2001-03-19 ETX/D/XPS-01:002 Uen 6
Erlang Highlights
 Declarative
 Concurrency
 Soft real-time
 Robustness
 Distribution
 Hot code loading
 External interfaces
 Portability
Functional programming language
High abstraction level
Pattern matching
Concise readable programs
2001-03-19 ETX/D/XPS-01:002 Uen 7
Erlang Example
Basics - Factorial function
Definition
1
Implementation
n=0
n! =
n*(n-1)!
n 1
-module(ex1).
-export([factorial/1]).
factorial(0) ->
1;
factorial(N) when N >= 1 ->
N * factorial(N-1).
Eshell V5.0.1 (abort with ^G)
1> c(ex1).
{ok,ex1}
2> ex1:factorial(6).
720
2001-03-19 ETX/D/XPS-01:002 Uen 8
Erlang Example
A few very high-level constructs - QuickSort
-module(ex2).
-export([qsort/1]).
qsort([Head|Tail]) ->
First = qsort([X || X <- Tail, X =< Head]),
Last = qsort([Y || Y <- Tail, Y > Head]),
First ++ [Head|Last];
qsort([]) ->
[].
Eshell V5.0.1 (abort with ^G)
1> c(ex2).
{ok,ex2}
2> ex2:qsort([7,5,3,8,1]).
[1,3,5,7,8]
"all objects Y
taken from the list Tail,
where Y > Head"
2001-03-19 ETX/D/XPS-01:002 Uen 9
Erlang Highlights
 Declarative
 Concurrency
 Soft real-time
 Robustness
 Distribution
 Hot code loading
 External interfaces
 Portability
Either transparent or
explicit concurrency
Light-weight processes
Highly scalable
2001-03-19 ETX/D/XPS-01:002 Uen 10
Erlang Example
Creating a new process using spawn
-module(ex3).
-export([activity/3]).
activity(Name,Pos,Size) ->
…………
Pid = spawn(ex3,activity,[Joe,75,1024])
activity(Joe,75,1024)
2001-03-19 ETX/D/XPS-01:002 Uen 11
Erlang Example
Processes communicate by asynchronous
message passing
Pid ! {data,12,13}
receive
{start} -> ………
{stop} -> ………
{data,X,Y} -> ………
end
2001-03-19 ETX/D/XPS-01:002 Uen 12
Erlang Example
Concurrency - Finite State Machine
ringing_a(A, B) ->
Selective receive
receive
{A, on_hook} ->
back_to_idle(A, B);
{B, answered} ->
Asynchronous send
A ! {stop_tone, ring},
switch ! {connect, A, B},
conversation_a(A, B)
after 30000 ->
Optional timeout
back_to_idle(A, B)
end.
back_to_idle(A, B) ->
A ! {stop_tone, ring},
B ! terminate,
idle(A).
2001-03-19 ETX/D/XPS-01:002 Uen 13
Erlang Highlights
 Declarative
 Concurrency
 Soft real-time
 Robustness
 Distribution
 Hot code loading
 External interfaces
 Portability
Response times in the
order of milliseconds
Per-process garbage collection
2001-03-19 ETX/D/XPS-01:002 Uen 14
Erlang Highlights
 Declarative
 Concurrency
 Soft real-time
 Robustness
 Distribution
 Hot code loading
 External interfaces
 Portability
Simple and consistent
error recovery
Supervision hierarchies
"Program for the correct case"
2001-03-19 ETX/D/XPS-01:002 Uen 15
Erlang Example
Cooperating processes may be linked together
using
spawn_link(…,…,…)
or
link(Pid)
2001-03-19 ETX/D/XPS-01:002 Uen 16
Erlang Example
When a process terminates, an exit signal is sent to all linked processes
… and the termination is propagated
2001-03-19 ETX/D/XPS-01:002 Uen 17
Erlang Example
Exit signals can be trapped and received as messages
receive
{‘EXIT’,Pid,...} -> ...
end
2001-03-19 ETX/D/XPS-01:002 Uen 18
Erlang Example
Robust systems can be built by layering
“Supervisors”
“Workers”
2001-03-19 ETX/D/XPS-01:002 Uen 19
Erlang Highlights
 Declarative
 Concurrency
 Soft real-time
 Robustness
 Distribution
 Hot code loading
 External interfaces
 Portability
Explicit or transparent distribution
Network-aware runtime system
2001-03-19 ETX/D/XPS-01:002 Uen 20
Transparent Distribution
B ! Msg
C ! Msg
Erlang Run-Time System
Erlang Run-Time System
network
2001-03-19 ETX/D/XPS-01:002 Uen 21
Simple RPC
{rex, Node} ! {self(), {apply, M, F, A}},
receive
{rex, Node, What} -> What
end
loop() ->
receive
{From, {apply, M, F, A}} ->
Answer = apply(M, F, A),
From ! {rex, node(), Answer}
loop();
_Other -> loop()
end.
2001-03-19 ETX/D/XPS-01:002 Uen 22
Erlang Highlights
 Declarative
 Concurrency
 Soft real-time
 Robustness
 Distribution
 Hot code loading
 External interfaces
 Portability
Easily change code in a
running system
Enables non-stop operation
Simplifies testing
2001-03-19 ETX/D/XPS-01:002 Uen 23
Erlang Example
Version 1
Version 2
2001-03-19 ETX/D/XPS-01:002 Uen 24
Erlang Highlights
 Declarative
 Concurrency
 Soft real-time
 Robustness
 Distribution
 Hot code loading
 External interfaces
 Portability
"Ports" to the outside world
behave as Erlang processes
2001-03-19 ETX/D/XPS-01:002 Uen 25
Erlang Example
Port
External
process
Port ! {self(), {command, [1,2,3]}}
2001-03-19 ETX/D/XPS-01:002 Uen 26
Erlang Example
Port
External
process
receive
{Port, {data, Info}} ->
end
2001-03-19 ETX/D/XPS-01:002 Uen 27
Erlang Highlights
 Declarative
 Concurrency
 Soft real-time
 Robustness
 Distribution
 Hot code loading
 External interfaces
 Portability
Erlang runs on any UNIX,
Windows, VxWorks, ...
Supports heterogeneous
networks
2001-03-19 ETX/D/XPS-01:002 Uen 28
Systems Overview
Applications written
in Erlang
Applications
written in C,
C++ or Java
OTP Components
Standard Libraries
Erlang Run-Time System
Hardware and Operating System
2001-03-19 ETX/D/XPS-01:002 Uen 29
Erlang/OTP
 Open Telecom Platform
 Middleware for Erlang development
 Designed for fault tolerance and portability
 Behaviors: A formalization of design patterns
 Components
–
–
–
–
–
–
–
Error handling, reporting and logging
Mnesia, distributed real-time database management system
CORBA
IDL Compiler, Java & C Interface Support
HTTP Server
SNMP Agent
...
2001-03-19 ETX/D/XPS-01:002 Uen 30
OTP Behaviors
 "A formalization of design patterns"
– A behavior is a framework
+ generic code to solve a common problem
– Each behavior has built-in support for
debugging and software upgrade
– Makes it easier to reason about the behavior of a program
 Examples of OTP behaviors
–
–
–
–
–
application
supervisor
gen_server
gen_event
gen_fsm
defines how an application is implemented
used to write fault-tolerant supervision trees
for writing client-server applications
for writing event handlers
for finite state machine programming
2001-03-19 ETX/D/XPS-01:002 Uen 31
The standard textbook
Joe Armstrong, Robert Virding, Claes Wikström and Mike Williams,
Concurrent Programming in Erlang, Prentice Hall, 1996,
2nd edition, ISBN 0-13-508301-X
2001-03-19 ETX/D/XPS-01:002 Uen 32
Interesting web pages
Open source Erlang
www.erlang.org
Commercial Erlang
www.erlang.se
French Erlang site
www.erlang-fr.org
2001-03-19 ETX/D/XPS-01:002 Uen 33
Courses/year (10-15 pupils/course)
45
40
35
30
25
20
15
10
2000
1999
1998
1997
1996
1995
1994
1993
1992
1991
1990
0
1989
5
2001-03-19 ETX/D/XPS-01:002 Uen 34
jul-01
mar-01
nov-00
jul-00
mar-00
nov-99
jul-99
mar-99
200000
180000
160000
140000
120000
100000
80000
60000
40000
20000
0
nov-98
Requests/month to www.erlang.org
2001-03-19 ETX/D/XPS-01:002 Uen 35
jul-01
mar-01
nov-00
jul-00
mar-00
nov-99
jul-99
mar-99
2500
2000
1500
1000
500
0
nov-98
Downloads/month from www.erlang.org
2001-03-19 ETX/D/XPS-01:002 Uen 36
 AXD 301: A Telephony-Class,
scalable (10-160 GBps) ATM switch
designed from scratch in less than 3 years
 AXD 301 Success factors:
–
–
–
–
Highly pragmatic, holistic approach
Competent organisation
Efficient process
Excellent technology (e.g. Erlang/OTP)
 More than just technology...
– Consider all factors together from the start
– Erlang was a perfect match for our approach
2001-03-19 ETX/D/XPS-01:002 Uen 37
AXD 301 in the marketplace
 Central component in
Ericsson's ENGINE offering
ENGINE:
Migrating today's vertical networks
into a single multi-service backbone
 Several major operators
–
–
–
–
–
–
British Telecom
Vodaphone
Worldcom
Telia
Diveo
...
2001-03-19 ETX/D/XPS-01:002 Uen 38
Briefly about the term Carrier-Class
 To us, "Carrier-Class", "Telephony-Class" and
"Telecom Profile" are synonymous
 The quality we've come to expect
from public telephony networks
 The trend towards multimedia services
requires Carrier-Class in more systems
 More than just duplication of hardware:
–
–
–
–
–
Fault-tolerant software
In-service hardware expansion
In-service software upgrade
Load tolerance
Flexibility (frequent changes + long service life)
There's no such thing
as "almost Carrier-Class"!
 Target: 99,999% ("five nines") availability,
including planned outages
2001-03-19 ETX/D/XPS-01:002 Uen 39
Telecom-Class System Architecture
simple wire-speed logic
Line
ATM
Termination Termination
ATM
ATB
CE
ATB
FR
ATB
complex soft-real-time logic
Control
Processors
Switch
Core
CP
IO
CP
IO
CP
IO
Optional
Processors
Server
Device
L3F
Device Processor
on Each Board
Mandatory
Mated
Processor
Pair
ATB
CP
IO
Clock &
Synchronization
2001-03-19 ETX/D/XPS-01:002 Uen 40
Programming languages (control system)
 Erlang: ca 1 million lines of code
– Nearly all the complex control logic
– Operation & Maintenance
– Web server and runtime HTML/JavaScript generation
 C/C++: ca 500k lines of code
– Third party software
– Low-level protocol drivers
– Device drivers
 Java: ca 13k lines of code
– Operator GUI applets
2001-03-19 ETX/D/XPS-01:002 Uen 41
Experiences from AXD 301 SW Design
 Using Erlang in Complex Systems
–
–
–
–
–
Fits very well with the incremental design method
High programmer satisfaction
Outstanding support for robustness and concurrency
Very few side-effects  easier to add/change single components
Small directed teams can achieve impressive results
 Productivity estimates
– Similar line/hour programmer productivity
– 4-10 fewer lines of source code (compared to C/C++, Java, PLEX)
 4-10x higher programmer productivity
– Similar number of faults per 1000 lines of source code
 4-10x higher quality
2001-03-19 ETX/D/XPS-01:002 Uen 42
 Functional Requirements
– Use cases (in telecom “traffic cases”)
– User interfaces
– ...
 Non-functional Requirements
–
–
–
–
–
Code updating without interrupting the service
Distribution over several processors
Automatic handover upon error
Limited restart time
...
 The non-functional requirements are often much trickier to
handle and require technology bottom-up rather than analysis
top-down.
2001-03-19 ETX/D/XPS-01:002 Uen 43
Efficient Program Development
Requirements
 Interaction with the real
environment
Ideas
Prototyping
 Powerful and appropriate
abstraction mechanisms
 Efficient implementation
 Useful tools
Productification
2001-03-19 ETX/D/XPS-01:002 Uen 44
A Simple Erlang-XML Document
XML
<?xml version=“1.0”?>
<home.page title=“My Home Page”>
<title>
Welcome to My Home Page
</title>
<text>
<para>
Sorry, this home page is still under
construction. Please come back soon!
</para>
</text>
</home.page>
Erlang
{‘home.page’, [{title, “My Home Page”}],
[{title, “Welcome to My Home Page”},
{text,
[{para,
“Sorry, this home page is still under ”
“construction. Please come back soon!”}
]}
]}.
Almost equivalent
2001-03-19 ETX/D/XPS-01:002 Uen 45
Erlang Summary
 Declarative
– Compact code
 Concurrency
– Light-weight processes
– Message passing
 Soft real-time
 Robustness
www.erlang.org
www.erlang.se
– Process supervision
– Error trapping
 Distribution
 Hot code loading
 External interfaces
– To hardware and other languages
 Portability
2001-03-19 ETX/D/XPS-01:002 Uen 46
The Research Continues ...
 HiPE - High Performance Erlang (Uppsala, Astec)
 ETOS - Erlang to Scheme (Montréal)
 Erlang Verification (SICS, Astec)
 Type System (Glasgow)
 “Safe” Erlang (Canberra)
 Specification Techniques (Aachen)
 Erlang Processor (Ericsson CADLab)
 ...
2001-03-19 ETX/D/XPS-01:002 Uen 47
Erlang User Conference 2000
Hurray !!!
2001-03-19 ETX/D/XPS-01:002 Uen 48
Welcome to the next EUC
September 27, 2001
Älvsjö. Stockholm
www.erlang.se
Combined with IFL - International
Workshop on the Implementation
of Functional Languages
September 24-26, 2001
Älvsjö. Stockholm
2001-03-19 ETX/D/XPS-01:002 Uen 49