Video metadata have incorrect lenght of video

[Samoed]

🐛 Describe the bug

I tried to reintegrate one of the benchmarks into HF dataset format and got this issue. Sometimes a video could store incorrect information about the number of frames, which could lead to an error while accessing the last frame.

from torchcodec.decoders import VideoDecoder

v = VideoDecoder("https://huggingface.co/datasets/Samoed/torchcodec_issue/resolve/main/ywHBKayhyvQ_19_28.avi")

print(new_v.metadata.num_frames)
# 226

v.get_frame_at(new_v.metadata.num_frames - 2)
# Frame:
#   data (shape): torch.Size([3, 240, 300])
#   pts_seconds: 8.96
#   duration_seconds: 0.04

v.get_frame_at(new_v.metadata.num_frames - 1)
#Traceback (most recent call last):
#  File "<input>", line 1, in <module>
#  File "mteb/.venv/lib/python3.14/site-packages/torchcodec/decoders/_video_decoder.py", line 349, in get_frame_at
#    data, pts_seconds, duration_seconds = core.get_frame_at_index(
#                                          ~~~~~~~~~~~~~~~~~~~~~~~^
#        self._decoder, frame_index=index
#        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
#    )
#    ^
#  File "mteb/.venv/lib/python3.14/site-packages/torch/_ops.py", line 819, in __call__
#    return self._op(*args, **kwargs)
#           ~~~~~~~~^^^^^^^^^^^^^^^^^
# RuntimeError: Requested next frame while there are no more frames left to decode.

Versions

PyTorch version: 2.10.0
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 15.7.3 (arm64)
GCC version: Could not collect
Clang version: 17.0.0 (clang-1700.6.4.2)
CMake version: version 4.3.1
Libc version: N/A

Python version: 3.14.2 (main, Dec  9 2025, 19:29:30) [Clang 21.1.4 ] (64-bit runtime)
Python platform: macOS-15.7.3-arm64-arm-64bit-Mach-O
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
Is XPU available: False
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True
Caching allocator config: N/A

CPU:
Apple M4

accelerate==1.13.0                                                                                                                                                                                                                                                                
aiofiles==24.1.0
aiohappyeyeballs==2.6.1
aiohttp==3.13.3
aiosignal==1.4.0
annotated-doc==0.0.4
annotated-types==0.7.0
anyio==4.13.0
ast-serialize==0.3.0
attrs==26.1.0
audioop-lts==0.2.2
av==17.0.1
bibtexparser==1.4.4
bm25s==0.3.2.post1
brotli==1.2.0
cachetools==7.0.5
certifi==2026.4.22
cffi==2.0.0
cfgv==3.5.0
charset-normalizer==3.4.6
click==8.3.3
contourpy==1.3.3
coverage==7.13.5
cryptography==47.0.0
cycler==0.12.1
datasets==4.8.3
deepmerge==2.0
dill==0.4.1
distlib==0.4.0
dnspython==2.8.0
execnet==2.1.2
fastapi==0.135.1
ffmpy==1.0.0
filelock==3.29.0
fonttools==4.62.1
frozenlist==1.8.0
fsspec==2026.4.0
ghp-import==2.1.0
gitdb==4.0.12
gitpython==3.1.46
gradio==6.0.1
gradio-client==2.0.0
griffelib==2.0.0
groovy==0.1.2
h11==0.16.0
hf-xet==1.4.3
httpcore==1.0.9
httpx==0.28.1
huggingface-hub==1.13.0
identify==2.6.18
idna==3.13
iniconfig==2.3.0
iso639==0.1.4
jinja2==3.1.6
joblib==1.5.3
kiwisolver==1.5.0
librt==0.10.0
markdown==3.10.2
markdown-it-py==4.0.0
markupsafe==3.0.3
matplotlib==3.10.8
mdurl==0.1.2
mergedeep==1.3.4
mkdocs==1.6.1
mkdocs-autorefs==1.4.4
mkdocs-get-deps==0.2.2
mkdocstrings==1.0.3
mkdocstrings-python==2.0.3
mpmath==1.3.0
-e file:///Users/samoed/Desktop/mteb
multidict==6.7.1
multiprocess==0.70.19
mypy==2.0.0
mypy-extensions==1.1.0
networkx==3.6.1
nodeenv==1.10.0
numpy==2.4.4
numpy-typing-compat==20251206.2.2
optype==0.17.0
orjson==3.11.7
packaging==26.2
pandas==2.3.3
pandas-stubs==3.0.0.260204
pathspec==1.1.1
peft==0.19.1
pillow==12.1.1
platformdirs==4.9.4
plotly==5.24.1
pluggy==1.6.0
polars==1.39.3
polars-runtime-32==1.39.3
pre-commit==4.5.1
propcache==0.4.1
psutil==7.2.2
pyarrow==23.0.1
pycparser==3.0
pydantic==2.12.4
pydantic-core==2.41.5
pydub==0.25.1
pygithub==2.9.1
pygments==2.20.0
pyjwt==2.12.1
pymdown-extensions==10.21.2
pymongo==4.16.0
pynacl==1.6.2
pyparsing==3.3.2
pystemmer==3.0.0
pytest==8.3.5
pytest-cov==7.1.0
pytest-cover==3.0.0
pytest-coverage==0.0
pytest-rerunfailures==15.1
pytest-xdist==3.6.1
python-dateutil==2.9.0.post0
python-discovery==1.2.0
python-multipart==0.0.22
pytrec-eval-terrier==0.5.10
pytz==2026.1.post1
pyyaml==6.0.3
pyyaml-env-tag==1.1
regex==2026.4.4
requests==2.32.5
rich==15.0.0
ruff==0.14.0
safehttpx==0.1.7
safetensors==0.7.0
scikit-learn==1.8.0
scipy==1.17.1
scipy-stubs==1.17.1.2
semantic-version==2.10.0
sentence-transformers==5.3.0
setuptools==80.10.2
shellingham==1.5.4
six==1.17.0
smmap==5.0.3
starlette==0.52.1
sympy==1.14.0
tabulate==0.10.0
tenacity==9.1.4
threadpoolctl==3.6.0
tokenizers==0.22.2
tomli==2.4.1
tomlkit==0.13.3
torch==2.10.0
torchaudio==2.10.0
torchcodec==0.10.0
torchvision==0.25.0
tqdm==4.67.3
transformers==5.7.0
typer==0.25.1
typer-slim==0.24.0
types-cachetools==6.2.0.20260317
types-colorama==0.4.15.20250801
types-defusedxml==0.7.0.20250822
types-docutils==0.22.3.20260322
types-gevent==25.9.0.20260322
types-greenlet==3.3.0.20251206
types-networkx==3.6.1.20260321
types-openpyxl==3.1.5.20260322
types-protobuf==6.32.1.20260221
types-psutil==7.2.2.20260130
types-pygments==2.19.0.20251121
types-pysocks==1.7.1.20251001
types-python-dateutil==2.9.0.20260305
types-pywin32==311.0.0.20260317
types-pyyaml==6.0.12.20250915
types-regex==2026.2.28.20260301
types-requests==2.32.4.20260107
types-setuptools==82.0.0.20260210
types-simplejson==3.20.0.20250822
types-tabulate==0.10.0.20260308
types-tensorflow==2.18.0.20260322
types-tqdm==4.67.3.20260303
types-xlrd==2.0.0.20251020
typing-extensions==4.15.0
typing-inspection==0.4.2
typos==1.44.0
tzdata==2025.3
urllib3==2.6.3
uvicorn==0.42.0
virtualenv==21.2.0
watchdog==6.0.0
xxhash==3.6.0
yarl==1.23.0
zensical==0.0.38

[NicolasHug]

Thanks for the report @Samoed . I can reproduce.

Some observations after spending a bit of time on this:

• There are 226 frames/packets in the video. The length and number of frames is correctly reported by torchcodec, the issue isn't related to the number of frames.
• Decoding frame 224 and then 225 works fine
• Decoding frame 225 only fails (225 is a key frame).

There are multiple factors at play for this video:

• The video is poorly encoded: every single packet is missing PTS info. When PTS is missing, we fallback on the pkt_dts field of the AVFrame.
• The thing is, this pkt_dts field is NOT the dts of the packet that created that frame. It's the dts of the packet that triggered that frame to be returned. For videos without B-frames, these are the same and that field is reliable. But for videos with B-frames, which is the case of that video, it means that the DTS will be set to that of a later packet. And that's what happens here for frame 225, the packet for that frame is sent to the decoder, but then we receieve EOF and we flush, and it's that flush packet that triggers the frame to be returned, so its DTS is set to -INF, which causes the frame to be discarded.

I'm going to label this as bug, but I should insist that really, this video is not encoded properly. Ideally we'd address this somehow, because we want to be robust w.r.t. poorly encoded videos, but I can't think of a robust way to make things work at the moment.

Maybe at the very least we should log that the PTS info is missing and that this might lead to decoding issues (once we support logging).

[Samoed]

should insist that really, this video is not encoded properly.

Yes, I suspected this. Thanks for response!