#### Transcript Ch03b

Part 1. Review of the CNS The Central Nervous System is made up of 100 Billion NEURONS Neurons are Cells Anatomy of a Neuron Myelin is supplied by support cells known as Glia (Glue) White Matter and Gray Matter • White Matter – Axons (highways) • Gray Matter – Cell bodies (cities) Nervous System Cells • Neurons and communication – Neurons are “charged” like batteries and have the ability to send electrical messages over long distances to other cells. • Sensory neurons carry information from sense organs to the central nervous system. • Motor neurons carry information from the CNS to muscles Charging the Neuron Action Potential A fast burst of electrical energy sent down the axon to the synapse, a gap between two neurons. Integrate and Fire!! • Neurons add up information from excitatory and inhibitory sources • If the net activity raises the level of excitation above a certain threshold, the neuron will fire an action potential down its axon • A neuron is either sending a message or it isn’t. – It’s a binary system (more on this later!) Nervous System Cells • Neurons and communication – Sending a message results in the depletion of the battery’s power, so the neurons need to be constantly recharged. Nervous System Cells • Nerve Impulses – Each neuron can either increase or decrease the likelihood that the next cell will continue to transmit. • Excitatory messages increase the probability that the next cell will “fire” - continue the transmission. • Inhibitory messages decrease the likelihood that transmission will continue to travel. Synaptic Transmission • Electrical message is converted to a chemical message – Activates another neuron – Moves a muscle – Triggers hormonal release from a gland Neural Networks • Each neuron receives information from many others and sends information to many others. – The significance (or weight) of a synaptic connection depends on the number and size of synapses shared by two neurons Synaptic Weight • Although both neurons A and B have excitatory synapses with C, Activity in cell A will have a bigger effect on C than the same amount of activity in cell B. A C • Neurotransmitters can be either excitatory or inhibitory • Lock and Key System Neurotransmitters & Behavior • Our understanding of neurotransmitters is the key to treating many brain-related disorders • Parkinson’s Disease – Parkinson’s Disease is a condition in which the individual has trouble executing voluntary movements, and has tremors, rigidity and a depressed mood. – This condition has been linked to a gradual decay in a system of axons that release the neurotransmitter dopamine. Summary • The Brain is an Electrical System – Neurons fire in an all-or-none fashion – Neurons may either increase or decrease another neuron’s chances of firing • Some Terms – Axon, threshold, dendrite, integrate and fire, neurotransmitter, excitation, inhibition, synapse, weight CRUM Theory • Computational-Representational Understanding of the Mind – Information is stored and processed 1. The flow of neural activity = processing 2. The state of synapses = storage History of Connectionism • Logical operations are not new (ancient Greece) • Connectionists seek to develop models based on what we know about the human brain. • A computer is not a very good connectionist model – A computer is a serial processor – Brain is a parallel processor • Connectionists believe that the phenomenon known as “mind” can be explained by interconnected neural units that follow the rules of neurons. Logical Operations • All statements are TRUE or FALSE War and Science The Beginning: Artificial Intelligence • Is the intelligence of the computer the same as that of the human? • Initially, idea eagerly accepted • 1950 Turing test: can a subject interacting with a computer be persuaded that he/she is communicating instead with a human? Alan Turing (1912-1954) • Chinese Room Problem • Deep Blue vs. Kasparov The McCullough and Pitts Neuron • They proposed that simple electrical circuits that behaved like neurons could perform some basic logical functions. – Neurons were either ON or OFF (binary) – Neurons had thresholds • A certain amount of activation was required in order to get them to fire – Connections (synapses) had weights that could vary in intensity and polarity (+ or -) – Computation took place in a succession of time slices (i.e. it took time for a message to travel from one neuron to the next) McCullough and Pitts (1943) • Using Electrical Circuits to Study Neurons Θ = Threshold W = Weight or connection Strength W=2 Θ=2 McCullough and Pitts (1943) • Logical “OR” operation W=2 Θ=2 McCullough and Pitts (1943) • Logical “AND” operation Θ=2 McCullough and Pitts (1943) • Logical “NOT” operation Θ=2 Donald Hebb (1949) Donald Hebb: Canadian Neuropsychologist who mediated between the Gestalt and atomist-localizating points of view. The Organization of Behavior (1949) • Opposed to the behaviorist “Black Box” • It is impossible to describe behavior as interaction directly between sensory and motor processes • Impossible to describe behavior as an interaction between sensory and motor processors, thinking intervenes • Unlike M&P, Hebb proposed a Dynamic Network Properties of a Hebb Synapse According to Hebb, Changes in the brain (learning) were the result of changes at the synapses. This has been supported by hundreds of subsequent studies Rosenblatt Perceptron (1958) • Learned to categorize – The perceptron had to be trained • If it was wrong, connections were adjusted Perceptron Convergence Rule Netinputout = Σinwain aout = 1 if netinputout > Θ = 0 otherwise Input vs. Output ain w w Θ ain If the activity level is too low then threshold should be decreased and connection strength should be increased. If too high, vise versa. ΔΘ = -εδ Δw = εδ a in where δ = (tout-aout) and ε = learning rate constant (between 0 and 1) Minsky and Pappert (1969) • A Critique of Perceptrons – The best perceptron could not solve the exclusive OR problem (XOR) – The sophisticated “alphabet skill” is not so impressive when one realizes that the simple XOR problem cannot be solved – Sunk connectionism for 15 years – See Pinker p. 110 Concepts and Connectionism • What might work and what won’t • Remember Pinker and “mentalese” as we discuss these examples Categories and Terms • Exemplar – Individual members of a category • Rule – A precise definition of the criteria for a category • Prototype – Specifies the properties that are most likely to be true of a category PROTOTYPICALITY: For each category, assign the number “1” to the best exemplar, “2” to the second best, “3” to the third best, “4” to the fourth best, and “5” to the worst member of the category. VEHICLES car elevator sled tractor train CLOTHING jacket mittens necklace pajamas pants FRUIT olive grapefruit orange pear honeydew Typicality Gradients • Sentence verification experiments Prototypes vs. Exemplars • Prototype theory – We have one “ideal” member of a category and we make judgments by comparing a stimulus to the ideal (which may not be an exemplar!). • Exemplar theory – We have lots of exemplars stored and we make judgments by comparing a stimulus to all exemplars and adding up the result. How does Classification Emerge? Rosch provided evidence for the idea that the basic level is “psychologically privileged.” We start at the middle. - Categories at the middle level are most consistent across cultures, easiest to process, and members are more clearly grouped (What else do you notice?) Weights Weights A parallel-distributed-processing (PDP) network. All seven units carry the message for both animals. The pattern of activity determines what you are thinking about. Learning in a PDP network. (a) Initially presenting canary causes a pattern of activation in the output units that is different than the pattern that stands for canary. (b) An error signal is transmitted back through the network to indicate how weights need to be changed to achieve the correct output response. continued on next slide Figure 8.30c (p. 300) (c) After the processes in (a) and (b) are repeated many times, the network has learned to respond correctly to canary. Pros and Cons of PDP • Uses the rules of the nervous system • Uses rules of learning – Try, get feedback, adjust, try again • Exhibits graceful degradation • Can’t explain fast learning • New learning will compromise old knowledge • May explain some kind of learning, but not all. The Excel Worksheet • The worksheet is the area where you enter all your data. • It is divided into vertical columns labeled A, B, C,…, and horizontal rows labeled 1, 2,3,… • A cell is the intersection of a row and a column. For example, cell D4 is found where column D intersects row 4. Entering Data (cont) In Excel there are two types of data – The first type, a Value, is either numeric data or a formula that generates numeric data. – The second type of data is called a Label. A Label is any string of characters (letters or numbers) that is used for descriptive purposes rather than as a numeric value or formula. Formulas • Formulas are entered into cells to do calculations. • A Formula must start with an = sign. The = sign lets Excel know that the formula is not a label (see previous slide). • A Formula can contain many mathematical symbols including addition (+), subtraction (-), multiplication (*), division (/), and exponentiation (**). Formulas (cont) • The order of calculation in a formula is based on standard mathematical operational precedence: from left to right, first what is contained in parentheses, then exponentiation, then multiplication and division, then finally addition and subtraction. (2+1)2 *3 – 5=22 Formulas (cont) • Cell references can be used in formulas as well. This feature shows the power of spreadsheets. • For example, suppose the cell B7 contains the value 8 and the cell C7 has the value 100. We can enter a value in another cell, say D8, to multiply cell B7 by C7. • The cell in D8 would then hold the formula: =B7*C7 which gives a value of 800. • If we change the value in cell B7 to 5, then the value in cell D8 will change to 500 automatically. Functions • Excel provides many functions to carry out mathematical calculations, lookups, etc. There is extensive on-line help that lists all of the functions available and how to use them in Excel. • To access the on-line help, first select the ‘Search for Help on…’ or the Microsoft Excel Help option under the Help menu • You can also click the function wizard (fx) to search Treating Cells as Neurons • The “neuron” of neural modeling is referred to as a unit • We can define cells in an Excel Spreadsheet as units and use other cells to define weights (synaptic strength) that allow values in one cell to influence those in another Modeling Binary Units in Excel 0 0 • One unit is represented by cell B5 and another is represented by cell F5 • The arrow represents an axon – Not necessary, but helpful 0 0 • Cell D6 represents the weight (synaptic strength) between the presynaptic and postsynaptic units. ? • “Activate” the presynaptic unit by entering a number into cell B5 • The activation that will reach the synapse is the product of the activity of the presynaptic unit and the weight of the connection between the two units (1x2)=2 ? • Define the threshold of the postsynaptic cell to determine if it will be active or not (G7) • If the threshold is exceeded, the activity level of the Postsynaptic cell will be set to 1 ? • The following will set the postsynaptic cell to 1 if the product of B5 and D6 is greater than the threshold (G7). • Put this equation in the postsynaptic unit (F5) =if(B5*D6 >= G7,1,0) Exercise 1 • Set up a simple network of binary units – 3 Presynaptic units converging on one postsynaptic unit – Set up cells that represent the 3 presynaptic units, the 3 weight values, the postsynaptic cell and the postsynaptic cell’s threshold so you can quickly change these values and test how these values affect the postsynaptic cell’s output Exercise 2 • Set up a network of five units using two input units, two hidden units, and one output unit as illustrated: – Make all activation and weight values easily changeable – Go slowly! Build and test one pair of units at a time – Try adding inhibition by giving negative values to some of your weights Exercise 3 • Set up a network that computes – AND – OR – NOT • Try XOR – Do you understand Minsky and Pappert’s complaint?