Transformers解决RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn

在使用Transformers启动梯度检查点时,即:

model.gradient_checkpointing_enable()

时,会报错RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn。

解决办法:在get_peft_model之前加上:

model.enable_input_require_grads()

你可能感兴趣的:(transformer,pytorch)