# What is numerical differentiation?

In school, we learned how to differentiate some functions.
Maybe you remember that the derivative of `x^2`

is `2x`

.
But could you differentiate an *arbitrary JavaScript function*?
And what would that even mean?

Let’s start small:

```
function f(x) {
return 7 * x;
}
```

How might you differentiate `f`

?
Let’s start with the stupidest thing that works!:

```
> f(3)
21
> f(3.01)
21.07
```

What happened here?
We increased the input `x`

by `0.01`

,
and as a result, the output increased by `0.07`

.
The output increase was 7 times more than our change to the input.
In math-speak, we say that the *derivative* of `f(3)`

with respect to `x`

is 7.

We’ve just discovered the simplest, stupidest form of differentiation:

```
function derivative(f) {
return (x) => {
const changeToInput = 0.00000001;
const changeToOutput = f(x + changeToInput) - f(x);
return changeToOutput / changeToInput;
};
}
```

With our magic `derivative`

function,
we can differentiate `x^2`

to get a function equivalent to `2x`

:

```
> function square(x) { return x * x; }
> const derivative_of_square = derivative(square);
> derivative_of_square(-13)
-26
```

But JS functions can have multiple parameters. Here’s one that multiplies its arguments:

```
function mul(a, b) {
return a * b;
}
```

What would it even mean to find the derivative of `mul(2, 3)`

?
Which argument are we tweaking, `a`

or `b`

?
Let’s try it with both:

```
> mul(2, 3)
6
> mul(2.01, 3)
6.03
> mul(2, 3.01)
6.02
```

Above, we see that the derivative for `a`

is `0.03 / 0.01 = 3`

,
and the derivative for `b`

is `0.02 / 0.01 = 2`

.
We can package this up nicely as the array `[3, 2]`

.
In math-speak, the values in this array are called *partial derivatives*,
and the entire array is called the *Jacobian* of the `sum`

function.

We can modify our `derivative`

function to
find the partial derivative for each parameter,
and return the array:

```
function derivative(f) {
return (...args) => {
const changeToInput = 0.00000001;
const derivatives = [];
for (let i = 0; i < args.length; i++) {
const changedArgs = [...args];
changedArgs[i] += changeToInput;
const changeToOutput = f(...changedArgs) - f(...args);
derivatives.push(changeToOutput / changeToInput);
}
return derivatives;
};
}
```

Using this, we can find the derivative of `mul`

at the arguments `(2, 3)`

:

```
> derivative(mul)(2, 3)
[ 3, 2 ]
```

This is the simplest numerical differentiation method.
Its biggest problem is efficiency.
If the function `f`

has a million parameters,
then evaluating `derivative(f)(...)`

calls the function `f`

two million times!
In the next post, we’ll see *automatic differentiation*,
a technique that only calls `f`

once.

This page copyright James Fisher 2024. Content is not associated with my employer.