A simple thought on machine learning and deep learning

· 未分类
Authors

I have a naive understanding of things so far.

For example, suppose we’re talking about classification problems. For typical SVM, the classification problem turns into looking for a linear boundary between the data (image from wiki):

image-4

But usually the boundary is not linear, so that’s why kernel methods (kernel svm, logistic regressions) are introduced, in hope to map/stretch the nonlinear boundary into linear:

kernel_machine

On the other hand, deep learning approach the problem from a totally different way. Instead of “stretching” the boundary, DN parameterizes and approximates the boundary.

I’m not an expert in kernel SVM or DL, so that’s just a high level understanding of mine. Hopefully I can explain things more clearly in the future posts.

Here’s a good example how DL does the approximation by stacking multiple layers:

https://www.r-bloggers.com/a-primer-on-universal-function-approximation-with-deep-learning-in-torch-and-r/

Theoretically, this idea is backed up by “universal approximation theorem” (Goodfellow’s DL book, 6.4.1 ), that any functional could be approximated by a high dimensional network (although there’s no linear bound of the layer # could be found).

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: