In class I try to convey the emphasis that ML puts on generalization to unseen data. For example, starting from a linear regression problem I ask whether ML is nothing more than glorified curve fitting. This leads to the discussion on overfitting, underfitting, and so on. I believe it would be good to expand this part of the material to discuss the nuances of these phenomena: bias-variance trade-off, double descent, model capacity and the overparametrization regime, etc.
I found this video to do a great job in this matter, showing the progress on the field over time and highlighting the most influential papers. It would be a great starting point to think how to frame the ideas in a pedagogical way.
In class I try to convey the emphasis that ML puts on generalization to unseen data. For example, starting from a linear regression problem I ask whether ML is nothing more than glorified curve fitting. This leads to the discussion on overfitting, underfitting, and so on. I believe it would be good to expand this part of the material to discuss the nuances of these phenomena: bias-variance trade-off, double descent, model capacity and the overparametrization regime, etc.
I found this video to do a great job in this matter, showing the progress on the field over time and highlighting the most influential papers. It would be a great starting point to think how to frame the ideas in a pedagogical way.