Refactor ContrastiveDataset and ContrastiveDistillationDataset - #579

Open

DemirTonchev wants to merge 24 commits into

huggingface:mainfrom

DemirTonchev:refactor-contrds

Commits on Dec 20, 2024

fixed to work with processing_class instead tokenizer after transformers 4.45.2
DemirTonchev
committed
cb4e803

Commits on Dec 23, 2024

refactor attempt for ContrastiveDataset so that it does not blow RAM with bigger dataset
DemirTonchev
committed
6a69303

Commits on Dec 24, 2024

Commits on Dec 25, 2024

ContrastiveDataset iterator refactor
DemirTonchev
committed
3e3fa5f
trainer fixed to work with ContrastiveDataset iter method
DemirTonchev
committed
2d5e29b
ContrastiveDistillationDataset iter refactor
DemirTonchev
committed
1c905b1

Commits on Dec 26, 2024

typing fix
DemirTonchev
committed
9cafa02

Commits on Jan 10, 2025

Commits on Jan 11, 2025

Commits on Jan 12, 2025

cleanup leftovers attributes from old logic
DemirTonchev
committed
7d9119b

Commits on Apr 9, 2025

Merge branch 'huggingface:main' into refactor-contrds
DemirTonchev
authored
704b4e9