mlx-community

/

DeepSeek-OCR-8bit

like 18

Follow

MLX Community 7.29k

Hey,

I'm trying to use the model via mlx-vlm as in the model's card example, and got the TypeError: 'KVCache' object is not subscriptable.

Here is the full traceback:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/opt/homebrew/lib/python3.12/site-packages/mlx_vlm/__main__.py", line 24, in <module>
    submodule.main()
  File "/opt/homebrew/lib/python3.12/site-packages/mlx_vlm/generate.py", line 646, in main
    result = generate(
             ^^^^^^^^^
  File "/opt/homebrew/lib/python3.12/site-packages/mlx_vlm/generate.py", line 532, in generate
    for response in stream_generate(model, processor, prompt, image, audio, **kwargs):
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.12/site-packages/mlx_vlm/generate.py", line 422, in stream_generate
    for n, (token, logprobs) in enumerate(
                                ^^^^^^^^^^
  File "/opt/homebrew/lib/python3.12/site-packages/mlx_vlm/generate.py", line 313, in generate_step
    outputs = model(input_ids, pixel_values, cache=prompt_cache, mask=mask, **kwargs)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.12/site-packages/mlx_vlm/models/deepseekocr/deepseekocr.py", line 323, in __call__
    logits = self.language_model(
             ^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.12/site-packages/mlx_vlm/models/deepseekocr/language.py", line 516, in __call__
    out = self.model(inputs, mask=mask, inputs_embeds=inputs_embeds, cache=cache)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.12/site-packages/mlx_vlm/models/deepseekocr/language.py", line 493, in __call__
    mask = create_attention_mask(h, cache[0])
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.12/site-packages/mlx_lm/models/base.py", line 52, in create_attention_mask
    if cache is not None and cache[0] is not None:
                             ~~~~~^^^
TypeError: 'KVCache' object is not subscriptable

Thanks!