I was reading "Data Analysis" by D. S. Sivia and found the following fairly early in.

Suppose you have a posterior probability density function $P(x\mid y)$. One way to approximate it is with a Normal distribution or by writing it with two parameters in the form $P(x\mid y) \approx \bar{x} \pm \sigma$, where $\bar{x}$ is the best estimate for $P(x)$ and $\sigma$ is the standard deviation.

It is clear that the maximum of the posterior is given by $\frac{d P}{d x} \mid_{\bar{x}} = 0$ (and $\frac{d^2 P}{d x^2} \vert_{\bar{x}} < 0)$.

A measure of the reliability of this best estimate can be obtained by computing the Taylor expansion of the log-likelihood, $L = \log P(x\mid y)$:

$L = L(\bar{x}) + \frac{1}{2} \frac{d^2 L}{d x^2} \vert_{\bar{x}} (x-\bar{x})^2 + \ldots$

where the second term is missing because $\frac{d L}{d x} \vert_{\bar{x}} = 0$ since $L$ is a monotonic function of $P$.

Now, the $x^2$ term dominates the Taylor series, and after rearranging we get:

$P(x\mid y) \approx A \exp \left[\frac{1}{2}\frac{d^2 L}{d x^2} \vert_{\bar{x}} (x-\bar{x})^2\right] = \frac{1}{\sigma\sqrt{2\pi}} \exp\left[-\frac{(x-\bar{x})^2}{2\sigma^2}\right]$.

We have obtained the Normal, or Gaussian, distribution. Note that $\sigma = \left(-\frac{d^2 L}{d x^2} \vert_{\bar{x}}\right)^{-\frac{1}{2}}$