机器学习---吴恩达---Week6_1（机器学习改进方法）

标签：bottom   学习曲线   dataset   star   ati   mini   sel   dev   rem   

应用机器学习

选择合适的λ的方法：

1. Create a list of lambdas (i.e. λ∈{0,0.01,0.02,0.04,0.08,0.16,0.32,0.64,1.28,2.56,5.12,10.24});（使用一定区间范围的λ ）
2. Create a set of models with different degrees or any other variants.（使用不同λ 的模型）
3. Iterate through the λs and for each λ go through all the models to learn some Θ.λ（获得对应模型的参数值）
4. Compute the cross validation error using the learned Θ (computed with λ) on the JCV(Θ) without regularization or λ = 0.（使用得到的参数值和λ 计算验证误差）
5. Select the best combo that produces the lowest error on the cross validation set.（选择最好具有最小验证误差的λ ）
6. Using the best combo Θ and λ, apply it on Jtest(Θ) to see if it has a good generalization of the problem.（使用上述最优模型的参数值与λ 进行测试观察是否能够预测问题）

 Learning Curves（学习曲线）

Experiencing high bias:

Low training set size: causes Jtrain(Θ) to be low and JCV(Θ) to be high.（小训练数据：训练集误差小，验证集误差大）

Large training set size: causes both Jtrain(Θ) and JCV(Θ) to be high with Jtrain(Θ)≈JCV(Θ).（大训练数据：训练集误差增大，验证误差减小，二者同一水平）

If a learning algorithm is suffering from high bias, getting more training data will not (by itself) help much.高偏差情况，增大训练数据不能有效改善线训练结果）

Experiencing high variance:

Low training set size: Jtrain(Θ) will be low and JCV(Θ) will be high.（小训练数据：训练误差小，验证误差大）

Large training set size: Jtrain(Θ) increases with training set size and JCV(Θ) continues to decrease without leveling off. Also, Jtrain(Θ) < JCV(Θ) but the difference between them remains significant.（大训练数据：训练集误差增大，验证误差减小，二者具有一个明显gap）

If a learning algorithm is suffering from high variance, getting more training data is likely to help..高方差情况，增大训练数据可以有效改善线训练结果）

不同问题的针对解决办法

• Getting more training examples: Fixes high variance（增加训练数据---解决高方差）

• Trying smaller sets of features: Fixes high variance（减少训练特征---解决高方差）

• Adding features: Fixes high bias（增加训练特征---解决高偏差）

• Adding polynomial features: Fixes high bias（增加多项式拟合---解决高偏差）

• Decreasing λ: Fixes high bias（增大λ---解决高偏差）

• Increasing λ: Fixes high variance.（减小λ---解决高方差）

Diagnosing Neural Networks（神经网络诊断）

• A neural network with fewer parameters is prone to underfitting. It is also computationally cheaper.（小型神经网络容易欠拟合，计算量小）
• A large neural network with more parameters is prone to overfitting. It is also computationally expensive. In this case you can use regularization (increase λ) to address the overfitting.（大型神经网络容易过拟合，增大λ可以解决此问题，不过计算量大）

Using a single hidden layer is a good starting default. You can train your neural network on a number of hidden layers using your cross validation set. You can then select the one that performs best.（默认使用一个隐藏层是好的开始，可以使用通过不同的验证集选择合适的隐藏层层数）

Model Complexity Effects:（模型复杂度影响）

• Lower-order polynomials (low model complexity) have high bias and low variance. In this case, the model fits poorly consistently.（低阶多项式高偏差低方程，拟合性较差）
• Higher-order polynomials (high model complexity) fit the training data extremely well and the test data extremely poorly. These have low bias on the training data, but very high variance.（高阶多项式拟合性较好，一般具有低偏差高方差的特点）
• In reality, we would want to choose a model somewhere in between, that can generalize well but also fits the data reasonably well.（实际应用应选择拟合性和的同时预测性强的多项式模型）

机器学习---吴恩达---Week6_1（机器学习改进方法）

标签：bottom   学习曲线   dataset   star   ati   mini   sel   dev   rem   

原文地址：https://www.cnblogs.com/zouhq/p/10677620.html