Transcript Erlang

Erlang/OTP
A framework for building real-world applications
Erlang/OTP
• Open Telecom Platform
• When building a large system
– certain designs reappear frequently
• differing only in what they do
• generic part called “behaviour”
– things {processes, hardware} WILL fail
• might as well accept it and deal with it
– distributed systems easily become large
• monitoring + keeping track are the dominating
problems
• especially when multiple components have to
cooperate
OTP view of a distributed system
S* = Supervisors
P* = Processes
S1
P1
P2
S2
P3
P4
This is called a supervisiontree
But wait, where’s the hardware?
From Erlang Pt 1:
• Messagepassing between Processes is transparent
• Starting processes as well
• Pid = spawn(nodename@hostname, Function)
HostA
HostB
Node1
Node2
Node2
Node1
Behaviours in OTP
• Repetetive patterns
– generic server (gen_server)
• models the client/server architecture
– generic event handler (gen_event)
• models one-way event (does not expect reply)
– generic finite state machine (gen_fsm)
• state + event = new state
– all states and events are user defined!
– supervisor
• supervises a set of processes which typically
are gen_* implementations themselves
• Only need to fill in the callbacks!
Behaviours built on top of:
• many processes is a Good Thing
• transparent messagepassing
• no difference AT ALL between local/remote processes
• pattern-matching
• delegating messages/events to appropriate handler
• side-effect-freeness
• enables hot codeswap without interrupted service or state (momentarily)
• registered processes:
register(ProcessId, someatom).
someatom ! {message, Argument}.
The supervisor
• Responsible for managing processes
• Hierarchy is important and maintained
• Assures its configured children are running
• Restarts them if terminated unexpectedly
• Depending on “child-specification”
• {Module, Function, Args}
• restartpolicy, eg:
• one-for-one: one process dies, the same one gets restarted
• one-for-all: one process dies, the whole set gets killed and
restarted
• rest-for-one: one process dies, it and the processes following it
will be restarted
• transient: will not be restarted upon normal exit
• maximum restartfrequency, to prevent infinite tight restartloops because
of a bug/failure. if threshold passed, whole supervisiontree will be
terminated.
Example: gen_server
-module(my_server).
-behaviour(gen_server). % compiler issues warnings/errors about missing callbacks!
% Define your client API
test_server(Argument) ->
gen_server:call(myserver_instance, {test, Argument}).
% Start the server, typically from a supervisors’ ChildSpec
start_link(Args) ->
gen_server:start_link({local, myserver_instance), my_server, Args, _Options).
% The callbacks
init(Args) -> {ok, stateFromArgs(Args)}.
handle_call({test, Argument}, From, State) -> {reply, {ok, Argument}, State}.
code_change(OldVsn, State, Extra) -> NewState = modify(State), {ok, NewState}.
terminate(Reason, State) -> cleanup(State), ok.
Example: gen_server (using it)
Terminal 1
Terminal 2
$> erl -sname server@localhost
$> erl -sname client@localhost
(server@localhost)1> my_server:start_link().
{ok,<0.43.0>}
(client@localhost)1> my_server:test_server(“aap”).
handle_call({test, "aap")
(server@localhost)2>
“aap”
(client@localhost)2>
Erlang and the outside world
* Linking to external programs *strongly* discouraged
* Instead: "ports"
- launch remote program
- communicate over streams (e.g., stdin/stdout)
- binary or line-based protocols possible
* Used to talk to gnuplot and ssh managers
Why do we need ssh managers?
* Long-running command (ssh <user@host> tail -f <file>) never stops
* Ports can be closed (usually on eof)
* Doesn't stop the associated Unix process!
* Need wrapper that can stop program
Implications
* Communication with external programs is low-tech
* Very unlike scripting languages (Python et al.)
* Additional library support for C programs, but
* Pushes you towards an Erlang-centric architecture
In practice: log monitoring/plotting
Motivations
* Python had threading issues and
* We needed an Erlang demonstrator application
Decision:
* Rewrite in Erlang
* Erlang spawns "ssh tail -f"
Overview: log scraping
* Script at stations copies logs to mk5s
* Erlang/Python process copies logs to JIVE
* Erlang process scans logs for selected events
* Events stored to database
* Erlang/gnuplot process makes plots (.png)
Overview: realtime monitoring
* Schedules converted to database
* PHP queries db (via URL+json) for active stations
* Erlang maintains (Mnesia) db of webcam images
* Erlang maps URLs to plot images & webcam pics
* PHP queries (Erlang-coded) URLs for webcam and status
graphs
Key features
* Erlang-to-erlang distributed programming
- DMS's code makes graphs; HV's code will serve them
- Communicate via Erlang calls/messages across nodes
* Reliability
- Webcam code written to OTP standards - recovers from everything
- Log scraper ditto
Key features (2)
* Mnesia
- The Erlang database: Erlang’s native point of state
* RDBMS interface
- Log grovelling uses Erlang/MySQL driver
* Erlang <-> Unix process communication
- Log groveller runs python subprocess (with wrapper)
- Erlang uses gnuplot (no wrapper required)
Key features (3): Mnesia
• The transparently distributed (replicated) DB of erlang
• client doesn’t know which node actually executes query/commit
• as long as there’s a node alive the system will stay up, rejoining nodes sync
• It stores tuples with named fields (the columns); records
• The columntype is always the same: “ANY”
• Can store any Erlang term in a column
• e.g. binary jpeg data; no intelligence required, it just works!
• Contents of a “column” may vary between rows
• some may consider this an asset, some won’t
• It is NOT a RDBMS
• see above note
• first “column” =:= PrimaryKey, store/retrieve rows based on this
• Two table types
• set: PrimaryKey => record mapping is 1:1
• bag: PrimaryKey => record mapping is 1:n
• Access to the db MUST be done in transactions (which is a good thing)