Ollama增加上下文大小

1、编辑 Modelfile

from qwen2.5:32b-instruct-q4_K_M
parameter temperature 0.1
PARAMETER top_p 0.8
PARAMETER repeat_penalty 1.1
PARAMETER top_k 20
PARAMETER num_ctx 131072

这里可能会报错：

Error: command must be one of "from", "license", "template", "system", "adapter", "parameter", or "message"

说明： 文件中的内容只允许出现 “from”, “license”, “template”, “system”, “adapter”, “parameter”, or “message” 命令开头的行， 如果出现其他的命令， 会报错。

2、 使用ollama create命令，新建模型

~/.ollama/create_models_file# **ollama create qwen2.5:32b-instruct-q4_K_M_128k -f Modelfile**
gathering model components
using existing layer sha256:eabc98a9bcbfce7fd70f3e07de599f8fda98120fefed5881934161ede8bd1a41
using existing layer sha256:66b9ea09bd5b7099cbb4fc820f31b575c0366fa439b08245566692c6784e281e
using existing layer sha256:eb4402837c7829a690fa845de4d7f3fd842c2adee476d5341da8a46ea9255175
using existing layer sha256:832dd9e00a68dd83b3c3fb9f5588dad7dcf337a0db50f7d9483f310cd292e92e
creating new layer sha256:aa77a7f7797fe1f810a427283393657828b386c4fe146f2d1905ec0835b22f4e
writing manifest
success

3、验证

:~/.ollama/create_models_file# ollama list
NAME                                ID              SIZE     MODIFIED
qwen2.5:32b-instruct-q4_K_M_128k    7d7bde22ddbf    19 GB    9 seconds ago
qwen2.5:32b-instruct-q4_K_M         9f13ba1299af    19 GB    3 hours ago



~/.ollama/create_models_file# ollama run qwen2.5:32b-instruct-q4_K_M_128k
Error: model requires more system memory (108.9 GiB) than is available (40.5 GiB)

可以看到， 随着上下文的增加， 需要的内存大小也同步增加， 需要108G

参考： https://blog.csdn.net/yuand7/article/details/143800780

参考： https://blog.csdn.net/weixin_62627514/article/details/143236375

参考：https://www.reddit.com/r/ollama/comments/1gr0vci/does_qwen_25_coder_pulled_from_ollama_support/?tl=zh-hans

https://www.reddit.com/r/LocalLLaMA/comments/1hl2rmk/where_is_qwen25_with_tool_training_and_128k/?tl=zh-hans