工作小结四

标签：Alexnet   pre   不同   训练   class   kernel   forward   解决   ide   

1. 关于VSCode一直出现“Extension loading。。。”

问题描述：

最近使用VSCode调试python的时候，突然python无法加载

解决方案：

直接方法：直接关闭所有extension，然后重启VScode，再打开extension中的python模块即可

间接方案：依次关闭每个extension，看看对python的影响

2. TypeError: module.__init__() takes at most 2 arguments (3 given)

from torch.utils import data
class DCatDataSet(data.Dataset):#注意这里不是data.dataset

3. ValueError: some of the strides of a given numpy array are negative. This is currently not supported, but will be added in future releases.

image = cv2.imread(os.path.join(self.img_path,self.img_list[index]))[:,:,::-1]
image =  torch.from_numpy(image)

#修改为
image = cv2.imread(os.path.join(self.img_path,self.img_list[index]))[:,:,::-1].copy()
image =  torch.from_numpy(image)

4. pytorch读取修改预训练模型

• 直接覆盖或添加层

假设有下面的模型

import torch.nn as nn
class Test(nn.Module):
    def __init__(self):
        super(Test,self).__init__()
        self.conv_1 = nn.Linear(10, 20, 5),
        self.Relu_1 = nn.Relu() 
        self.Conv_2 = nn.Conv2d(20, 64, 5),
        self.Relu_2 = nn.ReLU()
   #其他函数略去

直接覆盖：

test = Test()
test.conv_1 = nn.Linear(20,20,5)

添加层：

test = Test()
test.conv_1 = nn.Squential(
    nn.Relu(test.Conv_2)
    nn.Conv2D(64,64,3)
) 

直接修改层的weight或bias

test.con_1.weight.detach=.....#test.con_1.weight是一个Tensor，具体操作看个人了
test.con_1.bias.detach=.....#
#--------也可以下面这样操作
for name,i in test.named_parameters():
    if "weight" in name:
        i.data[0,0,0]=1#按需操作

• 任意方式修改网络

#自己定义任意网络
class AlexNet(nn.Module):
    '''
    code from torchvision/models/alexnet.py
    结构参考 <https://arxiv.org/abs/1404.5997>
    '''
    def __init__(self, num_classes=2):
        super(AlexNet, self).__init__()

        self.model_name = 'alexnet'

        self.features = nn.Sequential(
            nn.Conv2d(3, 64, kernel_size=11, stride=4, padding=2),
            nn.ReLU(inplace=True),
            nn.MaxPool2d(kernel_size=3, stride=2),
            nn.Conv2d(64, 192, kernel_size=5, padding=2),
            nn.ReLU(inplace=True),
            nn.MaxPool2d(kernel_size=3, stride=2),
            nn.Conv2d(192, 384, kernel_size=3, padding=1),
            nn.ReLU(inplace=True),
            nn.Conv2d(384, 256, kernel_size=3, padding=1),
            nn.ReLU(inplace=True),
            nn.Conv2d(256, 256, kernel_size=3, padding=1),
            nn.ReLU(inplace=True),
            nn.MaxPool2d(kernel_size=3, stride=2),
        )
        self.classifier = nn.Sequential(
            nn.Dropout(),
            nn.Linear(256 * 6 * 6, 4096),
            nn.ReLU(inplace=True),
            nn.Dropout(),
            nn.Linear(4096, 4096),
            nn.ReLU(inplace=True),
            nn.Linear(4096, num_classes),
        )

    def forward(self, x):
        x = self.features(x)
        x = x.view(x.size(0), 256 * 6 * 6)
        x = self.classifier(x)
        return x

#读取任意一个pytorch模型（前提是你初始化的卷积一样就行，网络和名字不相同都无所谓）
pretrain_model = torchvision.models.densenet121(pretrained=True)

#开始赋值
test = AlexNet()
state_dict_my = test.state_dict()
for key,module in list(state_dict_my.items()):
    prrint(key)#可以修改权重的名字
state_dict_py = pretrain_model.state_dict()
for key,module in list(state_dict_py.items()):
    prrint(key)
#根据自己的需要修改网络参数
state_dict_my[list(state_dict_my.keys())[0]] = state_dict_py[list(state_dict_py.keys())[0]]

• 注意事项

nn.Sequential 定义的层，暂时不知道怎么修改里面的名字和数据，单独修改可以，但是同时修改暂时不会

尽量采用第二种方式去修改和定义网络，第一种局限性太小，也没必要去整修改Sequential，就算可以修改，实用性也很小

5. pytorch设置不同层的学习率

• 相同学习率

#不需要更新的直接设置梯度为False
for name,module in test.named_parameters():
    if "weight" in name:#这里自己设置
        module.requres_grad = False
#----------或者下面这样----------这个方法没测试过
pretrained_state_dict = pretrained_state_model.state_dict()        
pretrained_param_names = list(pretrained_state_dict.keys())
for i, param in enumerate(param_names[:-4]):  # excluding conv6 and conv7 parameters
    state_dict[param].requres_grad = False

• 不同学习率

#这里通过绝对地址进行操作
#当然可以通过keys进行操作
#state_dict_my = list(net.state_dict().keys())
#
#
conv5_params = list(map(id, net.conv5.parameters()))
conv4_params = list(map(id, net.conv4.parameters()))
base_params = filter(lambda p: id(p) not in conv5_params + conv4_params,
                     net.parameters())
params = [{'params': base_params},
          {'params': net.conv5.parameters(), 'lr': lr * 100},
          {'params': net.conv4.parameters(), 'lr': lr * 100}]
optimizer = torch.optim.SGD(params, lr=lr, momentum=0.9)

工作小结四

标签：Alexnet   pre   不同   训练   class   kernel   forward   解决   ide   

原文地址：https://www.cnblogs.com/wjy-lulu/p/12078280.html