2. Non-Linear vs. Linear

Linear models

We have to manually set an extraction function $\varphi$. For example:

$$\varphi :\underset{\mathbf{x}}{\underbrace{\left(\begin{array}{c}\text{size}\\ \text{bedrooms}\\ \text{distance}\end{array}\right)}}\mapsto \underset{\varphi (\mathbf{x})}{\underbrace{\left(\begin{array}{c}\text{size}\\ \text{bedrooms}\\ \text{distance}\\ \text{size}\times \text{bedrooms}\\ \log (\text{distance})\end{array}\right)}}$$

Then, we learn weights $w$ that weigh coordinates of the feature vector ($y=w^T\varphi (x)$, where $\varphi (x)$ is also a vector.)

Always convex problems

Non-Linear Models

In general non-convex