Derivatives
| Item | Description |
|---|
Derivative: f'(x) = lim h->0 [f(x+h)-f(x)]/h | Instantaneous rate of change — slope of tangent line |
d/dx[x^n] = n*x^(n-1) | Power rule — most common derivative |
d/dx[e^x] = e^x | Exponential is its own derivative |
d/dx[ln(x)] = 1/x | Natural log derivative |
d/dx[sin(x)] = cos(x) | Trig derivative |
d/dx[f(x)*g(x)] = f'(x)*g(x) + f(x)*g'(x) | Product rule |
d/dx[f(x)/g(x)] = (f'*g - f*g') / g^2 | Quotient rule |
d/dx[f(g(x))] = f'(g(x)) * g'(x) | Chain rule — differentiate outer, multiply by derivative of inner |
Partial Derivatives & Gradients
| Item | Description |
|---|
Partial: df/dx_i = derivative holding others constant | For multivariable functions — treat other variables as constants |
Gradient: grad f = [df/dx1, df/dx2, ..., df/dxn] | Vector of all partial derivatives — direction of steepest ascent |
Gradient in ML | The gradient of loss w.r.t. weights tells us how to update each weight |
Hessian = matrix of second partial derivatives | Curvature information — used in Newton's method, slower but more precise |
Jacobian = matrix of all first partials (m x n) | For vector-valued functions — generalizes gradient to multiple outputs |
Integrals
| Item | Description |
|---|
Indefinite: Integral f(x) dx = F(x) + C | Antiderivative — function whose derivative is f(x) |
Definite: Integral_a^b f(x) dx = F(b) - F(a) | Area under curve from a to b — Fundamental Theorem of Calculus |
Integral of x^n = x^(n+1)/(n+1) + C | Power rule for integration (n != -1) |
Integral of 1/x = ln|x| + C | Natural log — special case of power rule failure |
Integration by Parts: Integral u dv = uv - Integral v du | Product rule reversed — useful for products of different function types |
Integral of e^x = e^x + C | Exponential integrates to itself |
ML Applications
| Item | Description |
|---|
Backpropagation = Chain Rule | Gradient flows backward through computation graph using chain rule |
Gradient Descent Update: w_new = w_old - lr * grad | Move parameters opposite gradient direction by learning rate |
MLE via derivative = 0 | Set derivative of log-likelihood to zero to find optimal parameters |
KL Divergence involves integrals | Information theory heavily uses integration |
Pro Tip: The gradient always points in the direction of steepest ascent. Gradient descent moves in the OPPOSITE direction (negative gradient) to find minima.