Math macro command for latex support in markdown Number and Arrays command visualization comment a $a$ A scalar \va $\va$ A vector, additionally $\vzero, \vone, \vmu, \vnu, \vtheta$ for \vzero, \vone, \vmu, \vnu, \vtheta \mA $\mA$ A matrix \tA $\tA$ A tensor \mI_n $\mI_n$ Identity matrix with $n$ rows and $n$ columns \mI $\mI$ Identity matrix with dimensionality implied by context \ve^{(i)} $\ve^{(i)}$ Standard basis vector $[0,\dots,0,1,0,\dots,0]$ with a 1 at position $i$ \text{diag}(\va) $\text{diag}(\va)$ A square, diagonal matrix with diagonal entries given by $\va$ \ra $\ra$ A scalar-valued random variable \rva $\rva$ A vector-valued random variables \rmA $\rmA$ A matrix-valued random varialbes Sets and Graphs Command Visualization Comment \sA $\sA$ A set Note: the command covers sA to sZ but don't no sE since it's expectation \R $\R$ The set of real numbers {0, 1} ${0, 1}$ The set containing 0 and 1 {0, 1, \dots, n} ${0, 1, \dots, n}$ The set of all integers between $0$ and $n$ [a, b] $[a, b]$ The real interval including $a$ and $b$ (a, b] $(a, b]$ The real interval excluding $a$ but including $b$ \sA \backslash \sB $\sA \backslash \sB$ Set subtraction, i.e., the set containing the elements of $\sA$ not in $\sB$ \gG $\gG$ A graph Indexing Command Visualization Comment \eva_i $\eva_i$ Element $i$ of vector $\va$, with indexing starting at 1 \eva_{-i} $\eva_{-i}$ All elements of vector $\va$ except for element $i$ \emA_{i,j} $\emA_{i,j}$ Element $i, j$ of matrix $\mA$ \mA_{i, :} $\mA_{i, :}$ Row $i$ of matrix $\mA$ \mA_{:, i} $\mA_{:, i}$ Column $i$ of matrix $\mA$ \etA_{i, j, k} $\etA_{i, j, k}$ Element $(i, j, k)$ of a 3-D tensor $\tA$ \tA_{:, :, i} $\tA_{:, :, i}$ 2-D slice of a 3-D tensor \erva_i $\erva_i$ Element $i$ of the random vector $\rva$ Linear Algebra Operators Command Visualization Comment \mA^\top $\mA^\top$ Transpose of matrix $\mA$ \mA^+ $\mA^+$ Moore-Penrose pseudoinverse of $\mA$ \mA \odot \mB $\mA \odot \mB$ Element-wise (Hadamard) product of $\mA$ and $\mB$ \mathrm{det}(\mA) $\mathrm{det}(\mA)$ Determinant of $\mA$ \sign(x) $\sign(x)$ Sign of a variable $x$ \Tr \mA $\Tr(\mA)$ Trace of a matrix A Calculus Command Visualization Comment \diff y / \diff x $\diff y / \diff x$ Derivative of $y$ with respect to $x$ \frac{\partial y}{\partial x} $\frac{\partial y}{\partial x}$ Partial derivative of $y$ with respect to $x$ \nabla_\vx y $\nabla_\vx y$ Gradient of $y$ with respect to $\vx$ \nabla_\mX y $\nabla_\mX y$ Matrix derivatives of $y$ with respect to $\mX$ \nabla_\tX y $\nabla_\tX y$ Tensor containing derivatives of $y$ with respect to $\tX$ \frac{\partial f}{\partial \vx} $\frac{\partial f}{\partial \vx}$ Jacobian matrix $\mJ \in \R^{m\times n}$ of $f: \R^n \rightarrow \R^m$ \nabla_\vx^2 f(\vx)\text{ or }\mH(f)(\vx) $\nabla_\vx^2 f(\vx)\text{ or }\mH(f)(\vx)$ The Hessian matrix of $f$ at input point $\vx$ \int f(\vx) d\vx $\int f(\vx) d\vx$ Definite integral over the entire domain of $\vx$ \int_\sS f(\vx) d\vx $\int_\sS f(\vx) d\vx$ Definite integral with respect to $\vx$ over the set $\sS$ Probabilities Command Visualization Comment \ra \bot \rb $\ra \bot \rb$ The random variables $\ra$ and $\rb$ are independent \ra \bot \rb \mid \rc $\ra \bot \rb \mid \rc$ They are conditionally independent given $\rc$ P(\ra) $P(\ra)$ A probability distribution over a discrete variable p(\ra) $p(\ra)$ A probability distribution over a continuous variable, or a variable of unspecified type \ra \sim P $\ra \sim P$ Random variable $\ra$ has distribution $P$ \E_{\rx \sim P} [ f(x) ] \text{ or } \E f(x) $\E_{\rx \sim P} [ f(x) ] \text{ or } \E f(x)$ Expectation of $f(x)$ with respect to $P(\rx)$ \Var(f(x)) $\Var(f(x))$ Variance of $f(x)$ under $P(\rx)$ \Cov(f(x), g(x)) $\Cov(f(x), g(x))$ Covariance of $f(x)$ and $g(x)$ under $P(\rx)$ H(\rx) $H(\rx)$ Shannon entropy of the random variable $\rx$ \KL(P \Vert Q) $\KL(P \Vert Q)$ Kullback-Leibler divergence of $P$ and $Q$ \mathcal{N}(\vx ; \vmu , \mSigma) $\mathcal{N}(\vx ; \vmu , \mSigma)$ Gaussian distribution over $\vx$ with mean $\vmu$ and covariance $\mSigma$ Functions Command Visualization Comment f: \sA \rightarrow \sB $f: \sA \rightarrow \sB$ The function $f$ with domain $\sA$ and range $\sB$ f \circ g $f \circ g$ Composition of the functions $f$ and $g$ f(\vx ; \vtheta) $f(\vx ; \vtheta)$ A function of $\vx$ parametrized by $\vtheta$. Sometimes written as $f(\vx)$ to simplify notation \log x $\log x$ Natural logarithm of $x$ \sigma(x) $\sigma(x)$ Logistic sigmoid, $\displaystyle \frac{1}{1 + \exp(-x)}$ \zeta(x) $\zeta(x)$ Softplus, $\log(1 + \exp(x))$ \Vert \vx \Vert_p $\Vert \vx \Vert_p$ $L^p$ norm of $\vx$ \Vert \vx \Vert $\Vert \vx \Vert$ $L^2$ norm of $\vx$ x^+ $x^+$ Positive part of $x$, i.e., $\max(0,x)$ \bm{1}_\mathrm{condition} $\bm{1}_\mathrm{condition}$ Is 1 if the condition is true, 0 otherwise Custom Commands special Command Visualization Comment \bm{#1} $\bm{x}$ Bold symbol, e.g., $\boldsymbol{x}$ \sign $\sign$ operator, Sign , $\operatorname{sign}$ \Tr $\Tr$ operator Trace, $\operatorname{Tr}$ \E $\E$ Expectation, $\mathbb{E}$ \KL $\KL$ Kullback-Leibler divergence, $D_\mathrm{KL}$ \NormalDist $\NormalDist$ Gaussian distribution, $\mathcal{N}$ \diag $\diag$ Diagonal matrix, $\mathrm{diag}$ \Ls $\Ls$ Loss function, $\mathcal{L}$ \R $\R$ Real number set, $\mathbb{R}$ \emp $\emp$ Empirical distribution, $\tilde{p}$ \lr $\lr$ Learning rate, $\alpha$ \reg $\reg$ Regularization coefficient, $\lambda$ \rect $\rect$ Rectifier activation, $\mathrm{rectifier}$ \softmax $\softmax$ Softmax function, $\mathrm{softmax}$ \sigmoid $\sigmoid$ Sigmoid function, $\sigma$ \softplus $\softplus$ Softplus function, $\zeta$ \Var $\Var$ Variance, $\mathrm{Var}$ \standarderror $\standarderror$ Standard error, $\mathrm{SE}$ \Cov $\Cov$ Covariance, $\mathrm{Cov}$ \tran $\tran$ Transpose operator, $^\top$ \inv $\inv$ Inverse operator, $^{-1}$ \diff $\diff$ Differential operator, $\mathrm{d}$ Reference Ian Goodfellow's ML book: https://github.com/goodfeli/dlbook_notation/blob/master/notation_example.pdf MathJax: https://docs.mathjax.org/en/latest/input/tex/macros.html