retry model download locally if huggingface throws an http error. (#215)

* rety model download locally if huggingface throws an http error.

* appease the linter

* key error fix

* use non internal lib error

Co-authored-by: Guillaume Klein <guillaumekln@users.noreply.github.com>

---------

Co-authored-by: Guillaume Klein <guillaumekln@users.noreply.github.com>
This commit is contained in:
David Axelrod
2023-05-09 11:20:22 -04:00
committed by GitHub
parent 91f948b0d6
commit 53d247b0bb

View File

@@ -77,11 +77,18 @@ def download_model(
"tokenizer.json",
"vocabulary.txt",
]
kwargs["allow_patterns"] = allow_patterns
kwargs["tqdm_class"] = disabled_tqdm
try:
return huggingface_hub.snapshot_download(
repo_id,
**kwargs,
)
except huggingface_hub.utils.HfHubHTTPError:
kwargs["local_files_only"] = True
return huggingface_hub.snapshot_download(
repo_id,
allow_patterns=allow_patterns,
tqdm_class=disabled_tqdm,
**kwargs,
)