Transcript SLC Database Service

SLC Aware IOC
Database Service
Design
Debbie Rogind
Nov 11, 2004
Topics
 DBS Use Cases
 Design Considerations
 DBS Initialization
 Data Structures
 Download Sequence

ST0 Conversion
 Receive/Process Setpoints and
other messages from Alpha
 Process/Send SLC IOC udpates
to Alpha
 DB Input/Output Utilities
DB Service Use Cases
 DBEX - “Look like” a tcp micro
 Same message formats, traffic, use proxy
 Req and Accept database (“IPL”)
 Accept updated setpoints from Alpha
 Provide updated data from IOC
 Supertype 3 (ST3) readbacks
 Supertype 2 (ST2)setpoints (new)


Accept and reply to diagnostic inquiries
Accept and Process DBEX up/down
messages
 Console Users

Utilities to edit, dump, get type of various slc
data
 Application code



Utilities to find, get, put, update slc data
Coded “data flow dependancies”
??? What else
Design Considerations
 Port as much existing micro code as




possible
Messages from VMS control system are
little-endian, VMS format, and packed
The VMS supertype header structure is
packed – no “pad” exists between 4 byte
boundaries
Support ASCII representations for
primary, secondary names and units
Slc IOC needs supplementary data
definition:



primary ASCII to #
secondary (secn) ASCII to # mappings for
given primary
datum width (for I or Z)
Topics
 DBS Use Cases
 Design Considerations
 DBS Initialization
 Data Structures
 Download Sequence

ST0 Conversion
 Receive/Process Setpoints and
other messages from Alpha
 Process/Send SLC IOC udpates
to Alpha
 DB Input/Output Utilities
DB Thread Initialization
 slcExec starts Phase 1 message
Q’ed threads dbSend, dbHdlr
 slcExec starts Phase 2 dbRecv
after all Phase 1 actives are T
 All db threads



Create cmlog handle
Init resources
 Message queue
 Sockets
 Memory heap
 Timers, semaphores
Set active = T
 Wait (at message Q or socket)
 dbHdlr thread initializes shared
db thread globals
 slcExec sends “DB_DOWNLOAD”
to dbSend
Topics
 DBS Use Cases
 Design Considerations
 DBS Initialization
 Data Structures
 Download Sequence

ST0 Conversion
 Receive/Process Setpoints and
other messages from Alpha
 Process/Send SLC IOC udpates
to Alpha
 DB I/0 Utilities
DB Service Globals
Globals for db threads, initialized by dbHdlr:
Name
Type
Represents
dbnodes_p[5]
dbsuptype_tu
Pointers to superblocks 0-3
dbRecv mallocs, writes pointers & data
while other threads are blocked at db
download event.
After download event pointers read-only (ptr
to ST0 & data deleted; rest never change)
dict_p
void *
Pointer to dictionary; new char-based
gphash table created for housing supertype
0 data (secondary data & location) and valid
ASCII names.
gphash has its own mutex protection for
add/delete/find dictionary utilities.
dbex_up
dbVersion
epicsBoolean
char *
Status of dbex up (T) or down (F)
Db major / minor version w dbex_up msg
dbRWMutex
epicsMutexId
Activiated for reads or writes from/to
supertype 1-4 data. dbRecv (for Alpha
VMS updates), and dbl* I/O utilities access
dbhilo_updates
[N_JOBS]
Dbhilo_job_ts *
Each “job service” array contains pairs of
offsets to upper and lower bounds of
modified ST2/3 data. Acc’ed by utility rte
dbhilo_update called from dblput.
dbhiloMutex
[N_JOBS}
epicsMutexId
Activiated for reads or writes from/to
dbhilo_updates[ ]. Acced by dbSend, and
private utility dbhilo_update called by dblput
dbAckEvent
[N_JOBS}
epicsEventId
Job service threads wait at this event until
their updated data has been acknowledged
by Alpha
• downloadEvent, dbExists flag is created by slcExec
• dbHdlrThread signals downloadEvent, sets dbExists flag after
db download / ST 0 conversion
Supertype 0 Byte Stream
to/from Alpha
fwd_hdr_ts
proxy header
dbsuptypehdr_ts
supertype header
links – next UPS, unit,
collided nodes
len - ups + nsub’s
catn - prim #
unit - unit #
nsub - # of secns
hash length
array listhead ptrs
array node counts
UPS block 0
secn block [0 – nsub]:
supn | subn
fmt | slen
sptr =
offset in supn
…
….
UPS block X
secn block [0 – nsub]
Supertype 1-3 Byte
Stream
fwd_hdr_ts
proxy header
dbsuptypehdr_ts
supertype header
data
data
data
data
data
data
….
Supertype Header & Data
Name
Data type
Represents
ip_port_u
len
user
cmd
ip_port_tu
int4u
user_field_ts
int1u
crc
int1u
Lower half of the ip address, and the
port number
Message bytecount - this fwdheader
User defined; chunk count for large
buffers;
Fwd_server command, e.g.
8 bit crc over header; currently set to
0x55
fwd_hdr_ts
dbsuptypehdr_ts
network
netlsn
id
<<<<<
len
micr
blkbeg
blkend
datw[ ]
Note
or
int2u
int2u
int2u
potential byte
int4u
Int4u
int4u
int4u
Array of int2u
dbsuptype_tu = union
Type of data
alignment mistmatch here >>>>>
Size of superblock
Microname
Copy to location in superblk
Copy from location in superblk
type-specific data; packed; max size
is NETBUFLEN minus
sizeof(dbsuptypehdr_ts)/2
Buffer/Q Structures
 Rcvbuf, sndbuf get cast to dbsndrcvbuf_ts
dbRecv converts to nativeSB -VMStohSupHdr()
Name
Data type
dbsndrcvbuf_ts
proxy_hdr
supblk
nativeSB
Represents
Receive and send buffers
fwd_hdr_ts
dbsuptype_tu
dbsuptypehdr_ts
Proxy header
Superblock header + data
Native superblock header
structure added for slc ioc
DB Thread Queue message structure
Name
Data type
dbmsgmail_ts
msgheader
nativeSB
dbdiag
Represents
Q messages
msgheader_ts
dbsuptypehdr_ts
dbdiagmsg_ts
Msg header
Native superblk hdr
Diagnostic data
 dbSend uses nativeSB
Converts prior to sending htoVMSSupHdr()
Secondary Data
 The secondary data is discussed
in later slides (after ST0
Conversion and the “new”
dictionary is discussed)
Topics
 DBS Use Cases
 Design Considerations
 DBS Initialization
 Data Structures
 Download Sequence

ST0 Conversion
 Receive/Process Setpoints and
other messages from Alpha
 Process/Send SLC IOC udpates
to Alpha
 DB I/0 Utilities
DB Service Download Block
Diagram
DBEX
Proxy
CTL socket
DBEX socket
3 Supertype Data
7 Ack
0, 6 Wait at socket
dbSend
dbmicromail
dbsuperalloc
dbmsgcpy
dbRecv
2 Download Request
4
DB_ACK_DBEX
(set db RWMutex)
5 Write supertype data
(reset dbRWMutex)
8 DB_CONVERT
1
dbHdlr
DB_DOWNLOAD
Repeat steps 3-7 as required by DBEX
dbRWMutex
dbnodes_p[1]
dbnodes_p[0]
ST 1
dbnodes_p[2]
ST 0
ST 2
dbnodes_p[3]
ST 3
SLC DB
dbRecvThread – Download
(“IPL”) Sequence
 Rcvbuf receives Supertype 0-3
messages from DBEX at dbex socket
 VMStohSupHdr(rcvbuf.nativeSB,
rcvbuf.supblk)
 Sends “DB_DBEX_ACK” to dbSend Q
to ack DBEX (if req’ed)
 Validates data and proper sequence of
download (dbmicromail)
 Allocates super block memory
(dbsuperalloc) from heap; populates
dbnode_p[0]
 Copies rcvbuf (dbmsgcopy) to alloc’ed
memory; drops proxyhdr
 Lock / unlock dbRWMutex around
memcpy
 Send “DB_CONVERT” to dbHdlr
 Upon recv’ing last ST 3 block
dbSendThread – Download
(“IPL”) Sequence
 Receives “DB_DOWNLOAD”
message from slcExec

Sends “ack” to DBEX socket





Formats sndbuf.nativeSB
htoVMSSupHdr(sndbuf.supblk,
msg.nativeSB)
Pre-fixes proxy_hdr
Sends
(dbdownloadme)
 Receives “DB_DBEX_ACK”
messages from dbRecv for each
ST 0-3 block received

Sends “ack” to DBEX socket




uses msg.nativeSB.id;
htoVMSSupHdr(sndbuf.supblk,
msg.nativeSB)
Pre-fixes proxy_hdr
Sends
Topics
 DBS Use Cases
 Design Considerations
 DBS Initialization
 Data Structures
 Download Sequence

ST0 Conversion
 Receive/Process Setpoints and
other messages from Alpha
 Process/Send SLC IOC udpates
to Alpha
 DB I/0 Utilities
dbHdlrThread
 dbHdlr is responsible for
 Initializing the DB Globals upon init
 Converting the ST0 data to dictionary
format
 Releasing DB resources upon exit
 Resetting global values upon exit



dbExists
downloadEvent
Thread specific – see General
Thread Shutdown
 Receives DB_CONVERT message to
initiate the ST0 Conversion process
 After conversion
 Sets dbExists = T
 Signals downloadEvent
 Receives TEST_STOP message
 Refer to General Thread Shutdown
dbHdlr Block Diagram
DBEX socket
PRIMARY.MAP
dbRecv
dbSend
NFS
8 DB_CONVERT
SLC IOC
dict_p
dbHdlr
dbRWMutex
dbnodes_p[1]
dbnodes_p[0]
ST 1
dbnodes_p[2]
Dictionary
ST 2
ST 0
dbnodes_p[3]
SLC DB
ST 3
Signal downloadEvent
dbExists = epicsTrue
Dictionary
Int
tablesize
Int
nShift
ELLIST **
list_pp
epicsMutexId lock
dict_p
“primary unit secondary” “QUAD 701 BDES”
Count 2
slcSubblk_ts *
Int1u
Int1u
Int1u
Int1u
Int4u
Int1u
“primary unit secondary”
Count 1
Count 1
secn sec #
supn ST #
len
word length
format (I,A,R,Z,S,T)
sptr word offset
width datum width
(1,2,4,8)
slcSubblk_ts
dbnodes_p[supn]
“primary” (not in ST 0)
primary = “QUAD”
Count =
20
201
301
ST 1
401
701
...
...
“BDES”
GPHENTRY *
ST2
ST3
...
Secondaries
...
...
Units
gphash
Secondary Data type
Name
Data type
slcSubblk_ts
secn
supn
len
format
sptr
width
Represents
SLC Secondary data type
int1u
int1u
int1u
int1u
int4u
int1u
Secondary number
Supertype number
Word length (V=variable)
(I,A,R,Z,S,T)
Word offset into supn block
1,2,4,8 bytes; data width
Gphash find - gphFind (“name”) - returns GPHENTRY*:
Name
Data type
GPHENTRY
node
name
pvtid
userPvt
Represents
SLC Secondary data type
ELLNODE
Const char *
Void *
Void *
Link node
“primary unit secondary”
Pointer to slcSubblk_ts
Not used
dbHdlrThread ST0 Conversion Sequence
 Receives “DB_CONVERT” message from dbRecv
 Replace ST0 num-oriented hash table with string
based hash table dictionary
 Step 1:
 Walk thru all structures in ST0 to swap words
and longs (if arch. is not little endian)
 Step 2:
 Read PRIMARY.MAP file and make entries
into dictionary, each combining
 Secondary data from ST0
 File data – data width, ASCII names
 Hash on “primary unit secondary”
 Hash on “primary”
 Links ALL* units
 For primary name error checking

Could also hash on “primary secondary”
 For faster secondary name error checking
 Structure is optimized for run-time
operations
dbHdlrThread ST0 Conversion Sequence
 Correlates file with ST0 data
PRIMARY.MAP:
Prim catn Sec
QUAD
1
BMON
QUAD
1
KTIM
secn fmt
width #
135
R
4
1
31
T
8
1
…
 For each unique primary name in file
Make “primary” entry
call dbunitsST0 (ld_p, catn)
if (status OK and ld_p.len>0)
for each unit# in list
dbgetUpsST0(uptr, catn, unit)
link unit in “primary” (for ALL*)
for every secondary in file
dbgetSecST0(sptr,uptr,secn)
store “primary unit sec” with
combined ST0 & file data;
+ link sec in unit entry (for ALL*)
dbHdlrThread ST0 Conversion Sequence


Delete ST0
Signal downloadEvent, sets dbExists
flag
 Comments
 Translated dblistu.a38 to C
 Uses memory manager (freeList) to
manage new slcSubblk_ts w combined
data
Optimized for:
dblists in thread’s init:
dblist(dblist_p, prim1, unit1, sec1)
dblist(dblist_p, prim2, unit2, secn) etc.

And “on the fly” access, such as
operator
Topics
 DBS Use Cases
 Design Considerations
 DBS Initialization
 Data Structures
 Download Sequence

ST0 Conversion
 Receive/Process Setpoints and
other messages from Alpha
 Process/Send SLC IOC udpates
to Alpha
 DB I/0 Utilities
Setpoints from Alpha
dbRecvThread – Alpha
Updates to slc ioc
 Receives Supertype 2 (setpoint)
update messages from DBEX




Sends “DB_DBEX_ACK” msg to dbSend, if
req’ed.
Lock dbRWMutex
Copies super block recv’ed (dbmsgcopy) to
slc ioc ST2 block
Unlock dbRWMutex
 Other DBEX messages
 Receives diagnostic req from DBEX

Sends “DB_DIAG_RPY” to dbSend Q to
ack DBEX
 Receives “DXDN”, “DXUP” messages
from DBEX



Resets/sets dbex_up flag
Stores database version in global
dbVersion
If DBEX becoming available, send
“DB_UPDATE_ALL” message to dbSend Q
to update all jobs (TBD)
dbSendThread –Alpha
Updates to slc ioc
 Receives DB_DBEX_ACK messages
from dbRecv for Alpha ST2 updates

Sends “ack” to DBEX socket (same as
“IPL” sequence)
 Receives DB_DIAG_RPY
message from dbRecv








uses msg.nativeSB.id
VMStoh(sndbuf.diag, msg.diag)
Process diagnostic data
htoVMSSupHdr(sndbuf.supblk,
msg.nativeSB)
htoVMS(sndbuf.diag, msg.diag)
Pre-fixes proxy_hdr
Sends to dbex socket
(dbmdiag_sendrpy)
 Receives TEST_STOP message
 Refer to General Thread Shutdown
Topics
 DBS Use Cases
 Design Considerations
 DBS Initialization
 Data Structures
 Download Sequence

ST0 Conversion
 Receive/Process Setpoints and
other messages from Alpha
 Process/Send SLC IOC
udpates to Alpha

dblput
 DB I/0 Utilities
Writing Data from slc ioc
(dblput)
Topics
•
•
•
•
•
DBS Use Cases
Design Considerations
DBS Initialization
Data Structures
Download Sequence
– ST0 Conversion
• Receive/Process Setpoints and
other messages from Alpha
• Process/Send SLC IOC
udpates to Alpha
– Dblput
– SLC IOC DB Updates to Alpha
• DB I/0 Utilities
slc ioc DB Updates to
Alpha
dbSendThread – slc ioc
updates to Alpha
 Receives DB_UPDATE
messages from various “job
threads” to send update





Lock dbhilo_mutex[job] – prevent
further updates for this job
Compress job’s list of pending
updates, if possible (microdbsendc)
Lock dbRWMutex

Read dbhilo_update[job] sptr pair

memcpy data from ST block indicated by
sptr pair into sndbuf
(microdbsendb)
Unlock dbRWMutex
Increment msg seq # (1-256), write
in proxyhdr.user_byte


store seq #, “job id” locally
Send sndbuf (already in VMS
format) to dbex socket
dbSendThread – CTL socket
After sending data to dbex socket, loop waiting at
CTL socket for Ack from DBEX
 Further updates for this job are blocked;
other job updates q’ed
 Timeout after ~15 sec (less if !dbex_up)


Wait 1 sec at ctl socket
Check for slcExec “stop flag”
CTL loop exit conditions:
1)Successful Ack (valid msg seq #)
 Clear dbhilo_update[job]
 If (! dbex_up), set dbex_up flag
2) Invalid msg seq #
 If (! dbex_up), set dbex_up flag ; no clear
3) Timeout – no clear
4) Socket error
 Set “lost connection flag” ; no clear
 Loop waiting for connection

Check for slcExec “stop flag”
Unlock dbhilo_mutex[job]
Signal ackEvent[job]
 Upon dbex_up / connection up

Send updates for all jobs
Topics
 DBS Use Cases
 Design Considerations
 DBS Initialization
 Data Structures
 Download Sequence

ST0 Conversion
 Receive/Process Setpoints and
other messages from Alpha
 Process/Send SLC IOC udpates
to Alpha
 DB Input/Output Utilities
DB I/O Utilities
 Threads use dballoc(), dbfree(), and db
macros for list management

dballoc() / dbfree() use mem management
 Re-entrant
 Return 4byte unsigned VMS cond code
 Except dbexists returns epicsBoolean
 All device args (prim,unit, secn) are 4
chars; blank padded, null-terminated;
prim/secn are left justified; unit is right-j?
 dblist (dblist_p, “prim”,”unit”,”secn”)

Unit can = “ALL*”
dblunits (dbdata_p, “prim”)
dblget (dbdata_p, dblist)
dblput (dbdata_p, dblist)
dballoc (dbdata_p, numBytes,
numElements)
 dbfree (dbdata_or_list_p)
 dbunit2string (intUnit, “unit”)
 dbexists (void)




DB I/O Utilities – cont.
 micro_dbsend(job)
 All threads call to send their updates to
Alpha
 Threads wait for ack or timeout
 DB I/O utils wait on downloadEvent
prior to access


Calling threads are held off until after
db is downloaded; timeout in case of
failure
Shell commands check dbExists()
before calling dbl* utils
 Error conditions
 A dblist (dblist_p, prim, unit, secn)
of either non-existent primary, unit, or
secondary returns an error, except if
unit = ALL*
 A dbunits (dblist_p, prim) of non-existent
primary returns an error
DB I/O functionality
 Dblist (dblist_p, prim, unit, secn)
 Calls gphFind (“prim unit secn”)
 Enters ptr to entry into dblist_p
 Dblist (dblist_p, prim, ALL*, secn)
 Calls gphFind (“prim”)
 Traverses linked list of prim/unit;
for each entry, traverses linked
list of secns to find; gets ptr from
matching entry’s data
 Enters ptr to entry in dblist_p
 Dblunits (dbdata_p, prim)
 Calls gphFind (“prim”)
 Traverses linked list of prim/unit;
for each entry, enters (4 char)
unit(s) into dbdata_p
DB I/O functionality
 Dblput (dbdata_p, dblist_p)
 Converts native typed input
dbdata to VMS typed data and
stores to dblist location
 locks / unlocks dbRWMutex
around memcpy of data to ST
block
 Calls dbhilo_update (job, hi, lo)


inserts hi, lo offset pairs into
dbhilo_update[job]
locks / unlocks dbhilo_mutex[job]
 Dblget (dbdata_p, dblist_p)
 sets/resets dbRWMutex around
memcpy from ST block data
 Converts retrieved data to native
 Inserts in dblist
Shell commands
Shells: EPICS iocsh, RTEMS
Cexp, vxWorks
 All previous dbl* utils can be called from ioc
shell after parsing console input.
 dbexists() will be checked prior to calling
underlying dbl* utility - no waiting for user
 dbgettype (“prim”, “secn”)
 dbdump (“prim”, “unit”, “secn”,
“<filename>”)
 dbdumpunits (“prim”, “<filename>”)
 dbedit (“prim”, “unit”, “secn”,…)
 dbdumphash (“<filename>”)
 dbdumpdatabase (“<filename>”)