# What is a McCulloch-Pitts neuron?

In their 1943 paper, A logical calculus of the ideas immanent in nervous activity, Warren McCulloch and Walter Pitts proposed a mathematical model of the behavior of neurons. But the notation is dated. Here I show the model in JavaScript. For the impatient, here’s the model that I’ll explain:

``````const makeNeuron = options => inputs => {
for (i of options.inhibitory)
if (inputs[i] === 1)
return 0;
let sum = 0;
for (i of options.excitatory)
sum += inputs[i];
return sum >= options.threshold ? 1 : 0;
}
``````

The McCulloch-Pitts neuron (also called the M-P neuron, or the “Threshold Logic Unit”) is modelled as a pure function with many inputs and one output. All inputs and outputs are either `0` or `1`. Here’s an example neuron in action:

``````> mystery([0,1])
0
> mystery([1,1])
1
``````

McCulloch-Pitts neurons can have many inputs, but this one has just two. Depending on those inputs, it outputs either `0` or `1`. In other words, it’s a logical binary operator. We can find out which operator it is by trying all the inputs:

``````> mystery([1,0])
0
> mystery([0,0])
0
``````

Yes, it’s the `AND` operator: interpreting `1` as true and `0` as false, it only returns true if both inputs are true.

There are many ways we could implement the `AND` neuron in JavaScript, but in the McCulloch-Pitts model, it’s implemented as:

``````const AND = ([x,y]) => x+y >= 2 ? 1 : 0
``````

Instead of using an `&&` operator, we use addition and comparison. The sum of the two inputs can only be `>= 2`.

This implementation probably looks strange. But it has two nice properties. The first nice property is that we can implement other logical operators by changing just one number:

``````const OR = ([x,y]) => x+y >= 1 ? 1 : 0
``````

By changing the `2` to a `1`, we get `OR`. This number is called the activation threshold. We can pull it out as a separate parameter, and implement three binary operators:

``````const makeNeuron =
options =>
([x,y]) =>
x+y >= options.threshold ? 1 : 0

const AND   = makeNeuron({threshold: 2})
const OR    = makeNeuron({threshold: 1})
const ON    = makeNeuron({threshold: 0})
``````

The second nice property of the McCulloch-Pitts model is that it resembles how physical neurons work: the threshold number corresponds to the neuron’s threshold potential measured in volts.

So we can make `AND` and `OR`; what about other operators like `NOR`, or `XOR`? How would you define `NOR` with the above `makeNeuron` function?

It turns out you can’t do it! To cope with this limitation, the McCulloch-Pitts model introduces inhibitory inputs. So far, if any input is switched from `0` to `1`, it increases the chance of the neuron outputting `1`, because it brings it closer to the threshold. These inputs are excitatory. By contrast, inhibitory inputs stop the neuron outputting `1`. If any inhibitory inputs are `1`, the output will be `0`.

Here’s `NOR` with inhibitory inputs:

``````const makeNeuron = options => inputs => {
for (i of options.inhibitory)
if (inputs[i] === 1)
return 0;
let sum = 0;
for (i of options.excitatory)
sum += inputs[i];
return sum >= options.threshold ? 1 : 0;
}

const AND = makeNeuron({threshold: 2, excitatory: [0,1], inhibitory: []});
const OR  = makeNeuron({threshold: 1, excitatory: [0,1], inhibitory: []});
const ON  = makeNeuron({threshold: 0, excitatory: [0,1], inhibitory: []});

// So now we can define ...
const NOR = makeNeuron({threshold: 0, excitatory: [], inhibitory: [0,1]});
``````

This `inhibitory` parameter loosely corresponds to inhibitory postsynaptic potential in the physical model. And the addition of this parameter lets us define `NOR`, which is nice. Now what about `XOR`? Can we define this, too?

Again, it turns out you can’t do it! There is no combination of `threshold` value and `inhibitory` inputs that implements `XOR`. However, if this is a model of the neuron, this fact is not so much a deficiency as an observation about neurons.

### More by Jim

Tagged #programming, #machinelearning. All content copyright James Fisher 2019. This post is not associated with my employer. Found an error? Edit this page.