huggingface笔记：文本生成Text generation

1 加载LLM模型

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
import os
 
 
model = AutoModelForCausalLM.from_pretrained(
    "gpt2",
    device_map="auto",  # 自动分配到所有可用设备（优先 GPU）
    torch_dtype=torch.bfloat16
)

2 编码输入并生成文本

tokenizer = AutoTokenizer.from_pretrained("gpt2", padding_side="left")

model_inputs = tokenizer(["A list of colors: red, blue"], return_tensors="pt")
model_inputs
'''
{'input_ids': tensor([[  32, 1351,  286, 7577,   25, 2266,   11, 4171]]), 'attention_mask': tensor([[1, 1, 1, 1, 1, 1, 1, 1]])}
'''

2.1 调用 generate() 并使用 batch_decode() 还原文本：'

generated_ids = model.generate(**model_inputs)
generated_ids
'''
tensor([[  32, 1351,  286, 7577,   25, 2266,   11, 4171,   11, 4077,   11, 4171,
           11, 7872,   11, 7872,   11, 4077,   11, 4171,   11, 7872,   11, 4077,
           11, 4171,   11, 7872]])
'''

tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
#A list of colors: red, blue, green, blue, yellow, yellow, green, blue, yellow, green, blue, yellow

3 常用参数

max_new_tokens	最大生成 token 数
do_sample	是否使用采样策略（默认为False） 根据词表中每个 token 的概率随机抽取
num_beams	Beam search 会在每一步保留num_beams个候选序列（称为 beam），最终选择总体概率最高的那一条。
temperature	控制生成随机性（>0.8 适合创意任务，<0.4 更“严谨”） 需配合 do_sample=True
repetition_penalty	>1 可减少重复内容

generated_ids = model.generate(**model_inputs,max_new_tokens=50,do_sample=True,temperature=0.9)
tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
'''
A list of colors: red, blue, white , yellow and purple

There is a separate link in the sidebar of this page to see how this affects the colors of the text. Click the "Color Information" button. Then click "Next" to add this color information to your
'''