9. Multivariable Distributions

Joint distribution:

Marginal density: $f_x=\sum _{\beta \in W_Y}\mathrm{Pr}[X=\alpha ,Y=\beta ]$ $=\mathrm{Pr}[X=\alpha ]$

Independence of Random Variables

Def.: $X_1,\dots ,X_n$ are independent $⟺\mathrm{Pr}[X_1={\alpha }_1,\dots ,X_n={\alpha }_n]$ $=\mathrm{Pr}[X_1={\alpha }_1]\cdots \mathrm{Pr}[X_n={\alpha }_n]$ $⟺f_{X_1,\dots ,X_n}({\alpha }_1,\dots ,{\alpha }_n)$ $=f_{X_1}({\alpha }_1)\cdots f_{X_n}({\alpha }_n)$

Consistent w/ def for independence of events via indicator variables.

Lemma

$X_1,\dots ,X_n$ independent $⟹\mathrm{Pr}[X_1\in S_1,\dots ,X_n\in S_n]$ $=\mathrm{Pr}[X_1\in S_1]\cdots \mathrm{Pr}[X_n\in S_n]$

Proof: Rewrite as nested sum of $\mathrm{Pr}[X_1={\alpha }_1,\dots ,X_n={\alpha }_n]$. Use definition. Commute terms.

Lemma

$X_1,\dots ,X_n$ independent and $I\subseteq [n]$ $⟹(X_i{)}_{i\in I}$ are also independent.

Proof: Define $S_i=\{\begin{array}{ll}\{{\alpha }_i\},& \text{if }i\in I\\ W_{X_i},& \text{otherwise}\end{array}$. Then use lemma with sets above.

Theorem

Let $f_1,\dots ,f_n\in \mathbb{R}\to \mathbb{R}$. $X_1,\dots ,X_n$ independent $⟹f_1(X_1),\dots ,f_n(X_n)$ independent

Note: Inverse does not apply (ex. $f(x)=1$) Proof: $\mathrm{Pr}[f_1(X_1)={\alpha }_1,\dots ,f_n(X_n)={\alpha }_n]$ $=\mathrm{Pr}[X_1\in S_1,\dots ,X_n\in S_n]$ ($S=\{x\mid f(x)=\alpha \}$), then lemma above.

Theorem

$X$, $Y$ independent and $Z:=X+Z$. Then:

$$f_Z(\alpha )=\sum _{\beta \in W_X}{f}_X(\beta )\cdot f_Y(\alpha -\beta )$$

Proof: $\mathrm{Pr}[X+Y=\alpha ]$ $=\sum _{\beta \in W_X}\mathrm{Pr}[X=\beta ,Y=\alpha -\beta ]$ $=\sum _{\beta \in W_X}\mathrm{Pr}[X=\beta ]\cdot \mathrm{Pr}[Y=\alpha -\beta ]$ (ind. $X,Y$)

Calculation Rules

• $\mathbb{E}[X+Y]=\mathbb{E}[X]+\mathbb{E}[Y]$ (linearity)
• $\mathbb{E}[X\cdot Y]=\mathbb{E}[X]\cdot \mathbb{E}[Y]$ (only independent, prove by using def. $\mathbb{E}$, def. ind. of random vars)
• $\mathrm{Var}[X+Y]=\mathrm{Var}[X]+\mathrm{Var}[Y]$ (only independent)
• $\mathrm{Var}[X\cdot Y]=\mathrm{Var}[X]\cdot \mathrm{Var}[Y]$(neither dependent nor independent)