免费使用支持离线部署使用的 txt2video 文本生成视频大模型(Text-to-Video-Synthesis Model)
免费使用支持离线部署使用的 txt2video 文本生成视频大模型(Text-to-Video-Synthesis Model)。
文本生成视频大模型(Text-to-Video-Synthesis Model)是一种基于深度学习技术的人工智能模型，它可以将自然语言文本描述转换为相应的视频。即通过输入文本描述，自动生成符合描述内容的视频。
这些模型使用深度学习技术，并结合计算机视觉和自然语言处理领域的知识，以实现自动生成视频的目的。该技术可以广泛应用于电影、游戏、虚拟现实等领域，具有巨大的商业价值和创新潜力。
这个开源的txt2video模型是一个非常有趣和有用的项目，它为文本到视频生成技术的进一步发展和推广提供了很好的契机。我们可以期待随着这种技术的不断完善和推广，会有更多的创新应用出现。
txt2video文本生成视频大模型的发布具有重要意义，主要表现在以下几个方面。
首先，该模型的发布为文本到视频生成技术的进一步发展和推广提供了很好的契机。随着人工智能技术的不断发展和普及，文本到视频生成技术已经成为了一个热门研究领域，并且在电影、游戏、虚拟现实等领域具有广泛的应用前景。
然而，由于技术门槛较高，很少有开源的文本到视频生成模型供普通开发者或研究者使用。因此，这个开源项目的发布可以让更多的人参与到这个领域的研究和应用中，推动技术的快速发展和应用落地。
其次，该模型的发布为电影、游戏、虚拟现实等领域带来了更多可能性和价值。文本到视频生成技术可以极大地降低电影、游戏、虚拟现实等领域的制作成本和时间，提高制作效率和质量。通过文本描述生成视频，可以避免漫长的拍摄和后期制作过程，同时还可以实现更加灵活和自由的创作方式，从而为这些领域带来更多的可能性和价值。
最后，该模型的发布也对开源社区和AI产业发展具有积极的促进作用。开源社区是一个充满活力和创新的社区，通过开源项目的发布，可以让更多的人参与到共同的研发和创新中来，从而推动技术进步和社区发展。
同时，AI产业也是一个蓬勃发展的行业，在这个行业中，开源项目的共享和合作是非常重要的，可以促进技术的跨界融合和互相学习。因此，该模型的发布也为开源社区和AI产业的发展提供了积极的促进作用。
总之，txt2video文本生成视频大模型的发布为文本到视频生成技术的发展和推广提供了很好的契机，同时也为电影、游戏、虚拟现实等领域带来了更多可能性和价值。通过开源社区和AI产业的共同努力，我们可以期待这种技术的不断完善和应用，为人类的生产和生活带来更多的便利和创新。
代码：
!pip install modelscope==1.4.2
!pip install open_clip_torch
!pip install pytorch-lightning
!pip install modelscope==1.4.2
!pip install open_clip_torch
!pip install pytorch-lightning
account_circle
Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/
Collecting modelscope==1.4.2
Downloading modelscope-1.4.2-py3-none-any.whl (4.2 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.2/4.2 MB 30.3 MB/s eta 0:00:00
Requirement already satisfied: scipy in /usr/local/lib/python3.9/dist-packages (from modelscope==1.4.2) (1.10.1)
Requirement already satisfied: Pillow>=6.2.0 in /usr/local/lib/python3.9/dist-packages (from modelscope==1.4.2) (8.4.0)
Requirement already satisfied: filelock>=3.3.0 in /usr/local/lib/python3.9/dist-packages (from modelscope==1.4.2) (3.10.0)
Requirement already satisfied: tqdm>=4.64.0 in /usr/local/lib/python3.9/dist-packages (from modelscope==1.4.2) (4.65.0)
Collecting jsonplus
Downloading jsonplus-0.8.0-py2.py3-none-any.whl (11 kB)
Collecting oss2
Downloading oss2-2.17.0.tar.gz (259 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 259.5/259.5 KB 17.9 MB/s eta 0:00:00
Preparing metadata (setup.py) ... done
Requirement already satisfied: setuptools in /usr/local/lib/python3.9/dist-packages (from modelscope==1.4.2) (63.4.3)
Collecting addict
Downloading addict-2.4.0-py3-none-any.whl (3.8 kB)
Requirement already satisfied: gast>=0.2.2 in /usr/local/lib/python3.9/dist-packages (from modelscope==1.4.2) (0.4.0)
Collecting einops
Downloading einops-0.6.0-py3-none-any.whl (41 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 41.6/41.6 KB 853.6 kB/s eta 0:00:00
Requirement already satisfied: requests in /usr/local/lib/python3.9/dist-packages (from modelscope==1.4.2) (2.27.1)
Requirement already satisfied: pyyaml in /usr/local/lib/python3.9/dist-packages (from modelscope==1.4.2) (6.0)
Requirement already satisfied: numpy<1.24.0 in /usr/local/lib/python3.9/dist-packages (from modelscope==1.4.2) (1.22.4)
Collecting datasets<=2.8.0,>=2.7.0
Downloading datasets-2.8.0-py3-none-any.whl (452 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 452.9/452.9 KB 1.9 MB/s eta 0:00:00
Requirement already satisfied: attrs in /usr/local/lib/python3.9/dist-packages (from modelscope==1.4.2) (22.2.0)
Collecting pyarrow!=9.0.0,>=6.0.0
Downloading pyarrow-11.0.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.9 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 34.9/34.9 MB 11.8 MB/s eta 0:00:00
Collecting yapf
Downloading yapf-0.32.0-py2.py3-none-any.whl (190 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 190.2/190.2 KB 21.1 MB/s eta 0:00:00
Requirement already satisfied: packaging in /usr/local/lib/python3.9/dist-packages (from datasets<=2.8.0,>=2.7.0->modelscope==1.4.2) (23.0)
Requirement already satisfied: fsspec[http]>=2021.11.1 in /usr/local/lib/python3.9/dist-packages (from datasets<=2.8.0,>=2.7.0->modelscope==1.4.2) (2023.3.0)
Collecting multiprocess
Downloading multiprocess-0.70.14-py39-none-any.whl (132 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 132.9/132.9 KB 20.2 MB/s eta 0:00:00
Collecting aiohttp
Downloading aiohttp-3.8.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.0 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.0/1.0 MB 77.5 MB/s eta 0:00:00
Collecting dill<0.3.7
Downloading dill-0.3.6-py3-none-any.whl (110 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 110.5/110.5 KB 16.3 MB/s eta 0:00:00
Collecting xxhash
Downloading xxhash-3.2.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (212 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 212.2/212.2 KB 16.4 MB/s eta 0:00:00
Collecting responses<0.19
Downloading responses-0.18.0-py3-none-any.whl (38 kB)
Collecting huggingface-hub<1.0.0,>=0.2.0
Downloading huggingface_hub-0.13.3-py3-none-any.whl (199 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 199.8/199.8 KB 25.1 MB/s eta 0:00:00
Requirement already satisfied: pandas in /usr/local/lib/python3.9/dist-packages (from datasets<=2.8.0,>=2.7.0->modelscope==1.4.2) (1.4.4)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.9/dist-packages (from requests->modelscope==1.4.2) (3.4)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.9/dist-packages (from requests->modelscope==1.4.2) (1.26.15)
Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.9/dist-packages (from requests->modelscope==1.4.2) (2.0.12)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.9/dist-packages (from requests->modelscope==1.4.2) (2022.12.7)
Collecting simplejson>=3.3.0
Downloading simplejson-3.18.4-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (136 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 136.8/136.8 KB 20.9 MB/s eta 0:00:00
Requirement already satisfied: sortedcontainers>=1.5.9 in /usr/local/lib/python3.9/dist-packages (from jsonplus->modelscope==1.4.2) (2.4.0)
Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.9/dist-packages (from jsonplus->modelscope==1.4.2) (2.8.2)
Collecting crcmod>=1.7
Downloading crcmod-1.7.tar.gz (89 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 89.7/89.7 KB 13.8 MB/s eta 0:00:00
Preparing metadata (setup.py) ... done
Collecting pycryptodome>=3.4.7
Downloading pycryptodome-3.17-cp35-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.1 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 54.3 MB/s eta 0:00:00
Collecting aliyun-python-sdk-kms>=2.4.1
Downloading aliyun_python_sdk_kms-2.16.0-py2.py3-none-any.whl (67 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 67.4/67.4 KB 10.5 MB/s eta 0:00:00
Collecting aliyun-python-sdk-core>=2.13.12
Downloading aliyun-python-sdk-core-2.13.36.tar.gz (440 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 440.5/440.5 KB 52.2 MB/s eta 0:00:00
Preparing metadata (setup.py) ... done
Requirement already satisfied: six in /usr/local/lib/python3.9/dist-packages (from oss2->modelscope==1.4.2) (1.15.0)
Collecting jmespath<1.0.0,>=0.9.3
Downloading jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting cryptography>=2.6.0
Downloading cryptography-39.0.2-cp36-abi3-manylinux_2_28_x86_64.whl (4.2 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.2/4.2 MB 102.0 MB/s eta 0:00:00
Collecting aiosignal>=1.1.2
Downloading aiosignal-1.3.1-py3-none-any.whl (7.6 kB)
Collecting async-timeout<5.0,>=4.0.0a3
Downloading async_timeout-4.0.2-py3-none-any.whl (5.8 kB)
Collecting yarl<2.0,>=1.0
Downloading yarl-1.8.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (264 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 264.6/264.6 KB 31.8 MB/s eta 0:00:00
Collecting frozenlist>=1.1.1
Downloading frozenlist-1.3.3-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (158 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 158.8/158.8 KB 20.1 MB/s eta 0:00:00
Collecting multidict<7.0,>=4.5
Downloading multidict-6.0.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (114 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 114.2/114.2 KB 16.5 MB/s eta 0:00:00
Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.9/dist-packages (from huggingface-hub<1.0.0,>=0.2.0->datasets<=2.8.0,>=2.7.0->modelscope==1.4.2) (4.5.0)
Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.9/dist-packages (from pandas->datasets<=2.8.0,>=2.7.0->modelscope==1.4.2) (2022.7.1)
Requirement already satisfied: cffi>=1.12 in /usr/local/lib/python3.9/dist-packages (from cryptography>=2.6.0->aliyun-python-sdk-core>=2.13.12->oss2->modelscope==1.4.2) (1.15.1)
Requirement already satisfied: pycparser in /usr/local/lib/python3.9/dist-packages (from cffi>=1.12->cryptography>=2.6.0->aliyun-python-sdk-core>=2.13.12->oss2->modelscope==1.4.2) (2.21)
Building wheels for collected packages: oss2, aliyun-python-sdk-core, crcmod
Building wheel for oss2 (setup.py) ... done
Created wheel for oss2: filename=oss2-2.17.0-py3-none-any.whl size=112391 sha256=3bc33ebc9127bd123e8cea9e15021908095c33b3e5df7a300e0d072120348480
Stored in directory: /root/.cache/pip/wheels/4b/23/be/4682480f462daef0106731b7efd0c513bfca3c846dc20081ab
Building wheel for aliyun-python-sdk-core (setup.py) ... done
Created wheel for aliyun-python-sdk-core: filename=aliyun_python_sdk_core-2.13.36-py3-none-any.whl size=533194 sha256=bf11be3157203b9874078a04074538dfbb48aeb590694c6e681cd58e186e85fa
Stored in directory: /root/.cache/pip/wheels/10/0a/92/f08c7b2c2cee2e47cffbb64fa291e5a30b24e77c726da41e31
Building wheel for crcmod (setup.py) ... done
Created wheel for crcmod: filename=crcmod-1.7-cp39-cp39-linux_x86_64.whl size=36929 sha256=87afc00c3e4291604dafe1da8658348fa21cdd05d5632c7bccea36c85dc713c1
Stored in directory: /root/.cache/pip/wheels/4a/6c/a6/ffdd136310039bf226f2707a9a8e6857be7d70a3fc061f6b36
Successfully built oss2 aliyun-python-sdk-core crcmod
Installing collected packages: yapf, crcmod, addict, xxhash, simplejson, pycryptodome, pyarrow, multidict, jmespath, frozenlist, einops, dill, async-timeout, yarl, responses, multiprocess, jsonplus, huggingface-hub, cryptography, aiosignal, aliyun-python-sdk-core, aiohttp, aliyun-python-sdk-kms, oss2, datasets, modelscope
Attempting uninstall: pyarrow
Found existing installation: pyarrow 9.0.0
Uninstalling pyarrow-9.0.0:
Successfully uninstalled pyarrow-9.0.0
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
pandas-gbq 0.17.9 requires pyarrow<10.0dev,>=3.0.0, but you have pyarrow 11.0.0 which is incompatible.
Successfully installed addict-2.4.0 aiohttp-3.8.4 aiosignal-1.3.1 aliyun-python-sdk-core-2.13.36 aliyun-python-sdk-kms-2.16.0 async-timeout-4.0.2 crcmod-1.7 cryptography-39.0.2 datasets-2.8.0 dill-0.3.6 einops-0.6.0 frozenlist-1.3.3 huggingface-hub-0.13.3 jmespath-0.10.0 jsonplus-0.8.0 modelscope-1.4.2 multidict-6.0.4 multiprocess-0.70.14 oss2-2.17.0 pyarrow-11.0.0 pycryptodome-3.17 responses-0.18.0 simplejson-3.18.4 xxhash-3.2.0 yapf-0.32.0 yarl-1.8.2
Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/
Collecting open_clip_torch
Downloading open_clip_torch-2.16.0-py3-none-any.whl (1.5 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.5/1.5 MB 21.9 MB/s eta 0:00:00
Collecting timm
Downloading timm-0.6.12-py3-none-any.whl (549 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 549.1/549.1 KB 46.8 MB/s eta 0:00:00
Collecting sentencepiece
Downloading sentencepiece-0.1.97-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 58.5 MB/s eta 0:00:00
Requirement already satisfied: huggingface-hub in /usr/local/lib/python3.9/dist-packages (from open_clip_torch) (0.13.3)
Requirement already satisfied: regex in /usr/local/lib/python3.9/dist-packages (from open_clip_torch) (2022.10.31)
Requirement already satisfied: torch>=1.9.0 in /usr/local/lib/python3.9/dist-packages (from open_clip_torch) (1.13.1+cu116)
Collecting ftfy
Downloading ftfy-6.1.1-py3-none-any.whl (53 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 53.1/53.1 KB 8.2 MB/s eta 0:00:00
Requirement already satisfied: torchvision in /usr/local/lib/python3.9/dist-packages (from open_clip_torch) (0.14.1+cu116)
Requirement already satisfied: protobuf<4 in /usr/local/lib/python3.9/dist-packages (from open_clip_torch) (3.19.6)
Requirement already satisfied: tqdm in /usr/local/lib/python3.9/dist-packages (from open_clip_torch) (4.65.0)
Requirement already satisfied: typing-extensions in /usr/local/lib/python3.9/dist-packages (from torch>=1.9.0->open_clip_torch) (4.5.0)
Requirement already satisfied: wcwidth>=0.2.5 in /usr/local/lib/python3.9/dist-packages (from ftfy->open_clip_torch) (0.2.6)
Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.9/dist-packages (from huggingface-hub->open_clip_torch) (6.0)
Requirement already satisfied: packaging>=20.9 in /usr/local/lib/python3.9/dist-packages (from huggingface-hub->open_clip_torch) (23.0)
Requirement already satisfied: requests in /usr/local/lib/python3.9/dist-packages (from huggingface-hub->open_clip_torch) (2.27.1)
Requirement already satisfied: filelock in /usr/local/lib/python3.9/dist-packages (from huggingface-hub->open_clip_torch) (3.10.0)
Requirement already satisfied: numpy in /usr/local/lib/python3.9/dist-packages (from torchvision->open_clip_torch) (1.22.4)
Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /usr/local/lib/python3.9/dist-packages (from torchvision->open_clip_torch) (8.4.0)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.9/dist-packages (from requests->huggingface-hub->open_clip_torch) (3.4)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.9/dist-packages (from requests->huggingface-hub->open_clip_torch) (2022.12.7)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.9/dist-packages (from requests->huggingface-hub->open_clip_torch) (1.26.15)
Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.9/dist-packages (from requests->huggingface-hub->open_clip_torch) (2.0.12)
Installing collected packages: sentencepiece, ftfy, timm, open_clip_torch
Successfully installed ftfy-6.1.1 open_clip_torch-2.16.0 sentencepiece-0.1.97 timm-0.6.12
Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/
Collecting pytorch-lightning
Downloading pytorch_lightning-2.0.0-py3-none-any.whl (715 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 715.6/715.6 KB 12.8 MB/s eta 0:00:00
Requirement already satisfied: PyYAML>=5.4 in /usr/local/lib/python3.9/dist-packages (from pytorch-lightning) (6.0)
Collecting lightning-utilities>=0.7.0
Downloading lightning_utilities-0.8.0-py3-none-any.whl (20 kB)
Requirement already satisfied: typing-extensions>=4.0.0 in /usr/local/lib/python3.9/dist-packages (from pytorch-lightning) (4.5.0)
Collecting torchmetrics>=0.7.0
Downloading torchmetrics-0.11.4-py3-none-any.whl (519 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 519.2/519.2 KB 52.6 MB/s eta 0:00:00
Requirement already satisfied: numpy>=1.17.2 in /usr/local/lib/python3.9/dist-packages (from pytorch-lightning) (1.22.4)
Requirement already satisfied: torch>=1.11.0 in /usr/local/lib/python3.9/dist-packages (from pytorch-lightning) (1.13.1+cu116)
Requirement already satisfied: packaging>=17.1 in /usr/local/lib/python3.9/dist-packages (from pytorch-lightning) (23.0)
Requirement already satisfied: tqdm>=4.57.0 in /usr/local/lib/python3.9/dist-packages (from pytorch-lightning) (4.65.0)
Requirement already satisfied: fsspec[http]>2021.06.0 in /usr/local/lib/python3.9/dist-packages (from pytorch-lightning) (2023.3.0)
Requirement already satisfied: requests in /usr/local/lib/python3.9/dist-packages (from fsspec[http]>2021.06.0->pytorch-lightning) (2.27.1)
Requirement already satisfied: aiohttp!=4.0.0a0,!=4.0.0a1 in /usr/local/lib/python3.9/dist-packages (from fsspec[http]>2021.06.0->pytorch-lightning) (3.8.4)
Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.9/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]>2021.06.0->pytorch-lightning) (1.3.3)
Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.9/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]>2021.06.0->pytorch-lightning) (1.8.2)
Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /usr/local/lib/python3.9/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]>2021.06.0->pytorch-lightning) (4.0.2)
Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.9/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]>2021.06.0->pytorch-lightning) (22.2.0)
Requirement already satisfied: charset-normalizer<4.0,>=2.0 in /usr/local/lib/python3.9/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]>2021.06.0->pytorch-lightning) (2.0.12)
Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.9/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]>2021.06.0->pytorch-lightning) (1.3.1)
Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.9/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]>2021.06.0->pytorch-lightning) (6.0.4)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.9/dist-packages (from requests->fsspec[http]>2021.06.0->pytorch-lightning) (3.4)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.9/dist-packages (from requests->fsspec[http]>2021.06.0->pytorch-lightning) (2022.12.7)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.9/dist-packages (from requests->fsspec[http]>2021.06.0->pytorch-lightning) (1.26.15)
Installing collected packages: lightning-utilities, torchmetrics, pytorch-lightning
Successfully installed lightning-utilities-0.8.0 pytorch-lightning-2.0.0 torchmetrics-0.11.4
[ ]
!nvidia-smi
account_circle
Tue Mar 21 14:35:47 2023
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 525.85.12 Driver Version: 525.85.12 CUDA Version: 12.0 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 Tesla T4 Off | 00000000:00:04.0 Off | 0 |
| N/A 69C P0 31W / 70W | 0MiB / 15360MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| No running processes found |
+-----------------------------------------------------------------------------+
[ ]
from modelscope.pipelines import pipeline
from modelscope.outputs import OutputKeys
p = pipeline('text-to-video-synthesis', 'damo/text-to-video-synthesis')
test_text = {
'text': 'A panda eating bamboo on a rock.',
}
output_video_path = p(test_text,)[OutputKeys.OUTPUT_VIDEO]
print('output_video_path:', output_video_path)
account_circle
2023-03-20 15:24:10,130 - modelscope - INFO - Model revision not specified, use default: master in development mode
2023-03-20 15:24:10,133 - modelscope - INFO - Development mode use revision: master
Downloading: 100%|██████████| 7.58k/7.58k [00:00<00:00, 1.73MB/s]
Downloading: 100%|██████████| 760k/760k [00:02<00:00, 312kB/s]
Downloading: 100%|██████████| 5.26G/5.26G [07:35<00:00, 12.4MB/s]
Downloading: 100%|██████████| 4.86G/4.86G [07:36<00:00, 11.4MB/s]
2023-03-20 15:40:18,525 - modelscope - INFO - initiate model from /root/.cache/modelscope/hub/damo/text-to-video-synthesis
2023-03-20 15:40:18,526 - modelscope - INFO - initiate model from location /root/.cache/modelscope/hub/damo/text-to-video-synthesis.
2023-03-20 15:40:18,530 - modelscope - INFO - initialize model from /root/.cache/modelscope/hub/damo/text-to-video-synthesis
2023-03-20 15:41:56,677 - modelscope - WARNING - No preprocessor field found in cfg.
2023-03-20 15:41:56,685 - modelscope - WARNING - No val key and type key found in preprocessor domain of configuration.json file.
2023-03-20 15:41:56,686 - modelscope - WARNING - Cannot find available config to build preprocessor at mode inference, current config: {'model_dir': '/root/.cache/modelscope/hub/damo/text-to-video-synthesis'}. trying to build by task and model information.
2023-03-20 15:41:56,688 - modelscope - WARNING - No preprocessor key ('latent-text-to-video-synthesis', 'text-to-video-synthesis') found in PREPROCESSOR_MAP, skip building preprocessor.
2023-03-20 15:41:56,696 - modelscope - WARNING - task text-to-video-synthesis input definition is missing
2023-03-20 15:43:01,717 - modelscope - WARNING - task text-to-video-synthesis output keys are missing
output_video_path: /tmp/tmp737yow52.mp4
[ ]
# Download result
from google.colab import files
files.download(output_video_path)
account_circle
The above code will display the save path of the output mp4 video, and the current encoding format can be played normally with VLC player. Some other media players may not view it normally.
文本生成视频大模型项目介绍地址
https://modelscope.cn/models/damo/text-to-video-synthesis/summary
文本生成视频大模型在线体验地址：
https://huggingface.co/spaces/damo-vilab/modelscope-text-to-video-synthesis