Transcript Poster - Lab

ge
Miao
Hai
Yiran
Qing
and Garrett S.
1.Electrical and Computer Engineering, University of Pittsburgh, Pittsburgh, PA, USA 15261.
2.2. Air Force Research Laboratory, Information Directorate, 525 Brooks Road, Rome, New York, USA 13441.
1
Chen ,
2
Wu
2
Rose
Convergence Speed Analysis
Memristor-based BSB Recall Circuit
In this work, we propose a hardware realization of the Brain-Statein-a-Box (BSB) neural network model training algorithm. This method
can be implemented as an analog/digital mixed-signal circuit to train
memristor crossbar arrays within BSB circuits. The training effect is
demonstrated through experimentation and the quality as an autoassociative memory is also analyzed and compared with software based
training methods. The impacts of non-ideal device characteristics and
fabrication defects in crossbar arrays are discussed. Our hardware
architecture shows great potential for low power, high speed, small
hardware size computations, and provides inherent security features.
Simulation Results
The training effect is achieved. An clear convergence gap can be
observed between trained pattern and other untrained patterns.
Physical constraints in hardware training
Voltage
L Memristor
Theory
z
h

Voltage
Voltage
Voltage
Voltage
L
TiO
L
z
Pt 2Pt RL
hL h
z
TiO
L
Pt
Pt
TiO
2-x2TiO2
z
RL RL
Pt
h
h
TiO
TiO
2
TiO222-x
TiO
RL2-x
Doping RL
TiO
RHL
R
Pt TiO
front
TiO2-x
2-x
TiO
Doping
Doping
2-x
R
R
H
H
Pt
Pt
front
front
Pt
Doping
(a)
(b)
Doping RH
R
H
Pt
Ptfront
R
H
TiO2 R
Pt
front
L
(a) (a)
(b) (b)
TiO2-x
(a)
(b)
(b)
(a)
(b)
In 2008, HP Lab demonstrated the first memristive device, in which the
RH was achieved by moving the doping front along a TiO
Pt
memristive
effect
2
thin-film.
Figure (a) and(b)(b) illustrate the conceptual view of the TiO2 thin-film
memristor and the corresponding variable resistor model, which is
equivalent to two serially-connected resistors. Here, RL (RH) is used to
denote the low (high) resistance state (LRS and HRS). The overall
resistance or memristance can be expressed as:
z

M ( )    RH  (1   )  RL
where  is the relative doping front position, which is the ratio of doping
front position over the total thickness of the TiO2 thin-film. When the
electrical excitation through a memristor is greater than the threshold
voltage, the memristance changes (in training). Otherwise, a memristor
behaves like a resistor.
POSTER TEMPLATE BY:
www.PosterPresentations.com
60%
Hardware (linear)
Hardware (nonlinear)
Lillo (1994)
Perfetti (1995)
Park (2010)
40%
20%
0%
1
100%
Embedded BSB Training Circuit
wij    (t j  y j )   i
wij    sign (t j  y j )  sign ( i )
Vin(i)>0
Training starts
M1& M2, ST Initialization
ST = m
YES
Training ends
Vout(i)>0
Vout(i) > Vth_h
Vin
BSB
Recall
Circuit
R/W
Control
60%
40%
Hardware (linear)
Hardware (nonlinear)
Lillo (1994)
Perfetti (1995)
Park (2010)
r1
r2
r3
20%
0%
Prototype patterns γ(k), k =1,...,5
Diff(i)[0]

Vout(i) > Vth_l
Vout
Vout(i) > Vth_h
BSB recall circuit
ST &
Arbiter
Diff = 0
NO
ST = 0
r4
r5
Table 1. Quality of domain of attraction.
Msel Vin
Training signal generation
& program memristor
crossbar
80%
Uniform size of domain of attraction.
Training pattern selection
Error detection, refresh Diff
3
Error correction rate.
Vop , if V  Vop

S ' (v)  V ,
if Vop  V  Vop .

V
,
if
V

V
op

op 

Algorithm modification:
2
Hamming distance (l)
V(t  1)  S ' (G1VA (t )  G1VA (t )  G2 V(t ))
NO
Pt
80%
0
Registers
γ(k), k=1...m
Limited data access: A crossbar array represents the terms in a
weight matrix by analog states of memristors – memristance. Reading
the exact memristance of the entire array is significantly costly in terms
of design complexity as well as performance overhead.
Limited accuracy of signal detection: The output signals of the
memristor crossbar in analog format shall be used to control the
training scheme. The accuracy in detecting these output signals affects
the quality, speed, and cost of the overall training procedure.
Non-ideal device characteristics: For example, the evolution of the
memristance for actual memristive devices tends to be nonlinear.
Therefore, applying the same excitation can result in different
memristance change, depending on the state of memristor.
Process variations and defects: Due to process variations,
memristors in a crossbar array are not always the same. For example,
the upper- and lower-bounds of memristance varies [16][17]. Moreover,
“dead cells” stuck at high-resistance state (HRS) or low-resistance
state (LRS) could exist.
100%
Uniform size of domain of
attraction
Abstract
1
Li ,
Error correction rate
1
Hu ,
Training
Signal
Generate
Error
Detection
±
Vth_l
±
Vth_h

Vout(i) > Vth_l
YES
ST++
Diff
Training complete
(a)
STEP 1: Initializing the crossbar arrays. At the beginning of a training
procedure, all memristance values in M1 and M2 are initialized to
approximate the average of RL and RH. The initialization doesn’t have to
be precisely accurate. Indeed, even when all the memristors are all at
LRS or HRS, the crossbar arrays can still be successfully trained but it
requires more time to reach convergence.
STEP 2: Selecting a prototype pattern  (k) Bn(k=1,…,m). Here, B is
the n-dimension binary space (1, 1). Assume a training set includes m
prototype patterns and each pattern  (k) has the same probability to be
chosen every time. Here, we use a counter ST to record the number of
patterns that have passed training in sequence. When ST > 0, the
corresponding patterns that passed training are excluded from the
selection.
STEP 3: Sending  (k) to BSB recall circuit. We convert  (k) in binary
space (1, 1) to a set of input voltages within the boundary (0.1V, 0.1V).
These input signals are supplied to the two memristor crossbars
simultaneously. The output signals can be obtained at the end of the
BSB recall circuit.
Diff(i)[1]
(b)
Diff(i)[1]
Diff(i)[0]
Diff(i)
XX
11
00
0
11
11
00
11
-1
1
(c)
STEP 4: Error detection. Since an input signal Vin(i) is ±0.1V, the
preset threshold voltages can be used to compare Vout(i) and λ∙
Vin(i),. We employ four threshold voltages, including:

Vth _ h  0.1  
Vth_ l  0.1  

th _ h
V
 0.1  

th _ l
V
 0.1  
Here, θ represents the tolerable difference.
STEP 5: Program memristor crossbar. Note that training the memristor
crossbar array is conducted column by column. To train the jth column,
the polarity and amplitude of the training pulse is determined by Diff(j).
The training pulses are supplied on the rows of memristor crossbar
arrays. The jth column is connected to ground and all the other columns
are floating. For a column, the training pattern is either the current
selected prototype pattern or the opposite pattern.
STEP 6: If training is completed? The counter ST increases by 1 if a
prototype pattern goes through STEP 25 and reports no error without
further tuning M1 and M2. Otherwise, ST is reset to 0 whenever an error
is detected. Repeat STEP 25 until ST reaches m.
Hardware (linear)
Hardware (nonlinear)
Lillo (1994)
Perfetti (1995)
Park (2010)
Best
419
465
164
478
502
Good
6
0
1
34
10
LRS/LRS (G1=30)
HRS/HRS (G1=30)
HRS/HRS (G1=50,digonal)
Negative
465
473
859
512
512
Bad
134
86
0
0
0
LRS/HRS (G1=30)
HRS/HRS (G1=50)
3000
Quality of Domain of
attraction
ge
Training Scheme Analysis for Memristor-Based Neuromorphic Design
2500
2000
1500
1000
500
0
1
2
3
4
5
Dead cell percentage (%)
Acknowledgement
This material is based upon work supported by the National Science
Foundation under Grant No. EECS-1311747, ECCS-1202225, and
CNS-1253424. Received and cleared for public release by AFRL on
May 1, 2012, case number 88ABW-2012-2568. Any opinions,
findings and conclusions or recommendations expressed in this
material are those of the authors and do not necessarily reflect the
views of NSF, AFRL or its contractors.