Random Variable

Def: map $X:\Omega \to \mathbb{R}$

Probability: $\mathrm{Pr}[X\le x]=\mathrm{Pr}[\{\omega \in \Omega \mid X(\omega )\le x\}]$

Density func: $f_x:\mathbb{R}\to [0,1],x\mapsto \mathrm{Pr}[X=x]$ Distribution func: $F_x:\mathbb{R}\to [0,1],x\mapsto \mathrm{Pr}[X\le x]$

Independence: $\mathrm{Pr}[(X\mid A)\le x]$ $=\mathrm{Pr}[X\le x\mid A]$ $=\frac{\mathrm{Pr}[\{\omega \in A\mid X(\omega )\le x\}]}{\mathrm{Pr}[A]}$

Expected Value

$$\begin{array}{rlrl}\mathbb{E}[X]& :=\sum _{\alpha \in W_x}\alpha \cdot \mathrm{Pr}[X=\alpha ]& & \text{where }{W}_x=\{\alpha \in \mathbb{R}\mid \mathrm{Pr}[X=\alpha ]>0\}\\ & =\sum _{\omega \in \Omega }X(\omega )\cdot \mathrm{Pr}[\omega ]& & (\text{Def. }\mathrm{Pr}[X=x])\\ & =\sum _{i=1}^{\infty }\mathrm{Pr}[X\ge i]& & \end{array}\text{\hspace{1em}(1)(2)(3)}$$

Sum does not converge $⟹$ expected values undefined

Intuition for 3: The first num is counted once, the second twice (by first and second), the third thrice (by first, second, and third), and so on. For the ones outside the def. of $X$, $\mathrm{Pr}=0$.

Indicator variable: $X_A(\omega ):=1\text{ if }\omega \in A,\text{ else }0$. $\mathbb{E}[X_A]=\mathrm{Pr}[A]$

Linearity

$$\begin{array}{rl}\mathbb{E}[X+Y]& =\mathbb{E}[X]+\mathbb{E}[Y]\\ & =\sum _{\omega \in \Omega }(X+Y)(\omega )\cdot \mathrm{Pr}[\omega ]\\ & =\end{array}$$

Independence

Def:

Law of Total Expectation

$A_1,\dots ,A_n$ pairwise disjoint with $A_1\cup \cdots \cup A_n=\Omega$ and $\mathrm{Pr}[A_1],\dots ,\mathrm{Pr}[A_n]>0$:

$$\mathbb{E}[X]=\sum _{i=1}^n\mathbb{E}[X|A_i]\cdot \mathrm{Pr}[A_i].$$

For pairwise disjoint events $A_1,A_2,\dots$ with ${\bigcup }_{i=1}^{\infty }{A}_k=\Omega$ and $\mathrm{Pr}[A_1],\mathrm{Pr}[A_2],\dots >0$ the analogous result holds:

$$\mathbb{E}[X]=\sum _{i=1}^{\infty }\mathbb{E}[X|A_i]\cdot \mathrm{Pr}[A_i].$$

Variance

$$\text{Var}[X]:=\mathbb{E}\left[(X-\mu {)}^2\right],\text{where}\mu =\mathbb{E}[X]$$

Standard deviation: $\sigma =\sqrt{\text{Var}[X]}$

Theorem

$\mathrm{Var}[X]=\mathbb{E}[X^2]-\mathbb{E}[X{]}^2$

Theorem

$\mathrm{Var}[a\cdot X+b]=a^2\cdot \mathrm{Var}[X]$

Intuitively:

• Adding a value does not increase the spread of the elements
• $a^2$ and not $a$ b/c variance is squared difference

Moments

Def.:

• $\mathbb{E}[X^k]:=$ $k$-th moment
• $\mathbb{E}(X-[X]{)}^k]:=$ $k$-th central moment

Common Moments:

• 1st moment: expected value
• 1st central moment: 0 ($\mathbb{E}[X-\mathbb{E}[X]]$ $=\mathbb{E}[X]-\mathbb{E}[X]$ $=0$)
• 2nd central moment: Variance