简介
What is machine learning
A computer program is said to learn from experience E with respect to some task T and some performance measure P, if its performance on T, as measured by P, improves with experience E.
T E P
Main type of ML
Supervised learning, with "right answer" given. We know what the correct output should look like.
Regression: predict continuous valued output.
Classification: discrete valued output.
Unsupervised learning, we don't know what is the output.
Clustering
Non-clustering
Model representation
X --> features
Y --> target
Hypothesis: the target function we wanted.
Cost function: Measure the accuracy of hypothesis function.
Linear regression
y^=hθ(x)=θ0+θ1x
Squared error function
📷
The average squared vertical distances of the training data from the predicted line be the least.
Most common used in regression problems. For linear regression with one variable, it’s a curve, while for regression with two variables, it's a surface.
Gradient descent
)📷
Alpha is the learning rate, the term multiplied by alpha is the derivative.
Alpha: choose the "right" value to ensure the gradient descent algorithm converge in a reasonable time.
Simultaneous update of the parameters.
Gradient descent will automatically take smaller steps as we approach a local/global minimum, hence no need to decrease alpha over time.
The optimization problem of linear with one or two variables has only one global extreme value, thus gradient descent always converges to the global minimum.
Problem: what to do if there are local extreme value?
*convex function --> with a bowel shape function
*Batch gradient descent: each step of gradient descent uses all the training example.
Linear algebra review
Matrix(dimension, expression of element)
Vector: an n*1 matrix.
Addition
Scalar multiplication
Multiplication
Identity matrix
Denote: I
📷
📷
Inverse matrix: only square matrix(whose #row is equal to #column) has inverse matrix.
Python: A.I
Matlab: inv(A)
Transpose
Python: A.T
Matlab: A'
Application
Use data matrix and parameter vector to express hypothesis function.
Use data matrix and parameter matrix to express different hypothesis functions, and compare the outcome.