retry model download locally if huggingface throws an http error. (#215)

* rety model download locally if huggingface throws an http error.

* appease the linter

* key error fix

* use non internal lib error

Co-authored-by: Guillaume Klein <guillaumekln@users.noreply.github.com>

---------

Co-authored-by: Guillaume Klein <guillaumekln@users.noreply.github.com>
This commit is contained in:
David Axelrod
2023-05-09 11:20:22 -04:00
committed by GitHub
parent 91f948b0d6
commit 53d247b0bb

View File

@@ -77,13 +77,20 @@ def download_model(
"tokenizer.json", "tokenizer.json",
"vocabulary.txt", "vocabulary.txt",
] ]
kwargs["allow_patterns"] = allow_patterns
kwargs["tqdm_class"] = disabled_tqdm
return huggingface_hub.snapshot_download( try:
repo_id, return huggingface_hub.snapshot_download(
allow_patterns=allow_patterns, repo_id,
tqdm_class=disabled_tqdm, **kwargs,
**kwargs, )
) except huggingface_hub.utils.HfHubHTTPError:
kwargs["local_files_only"] = True
return huggingface_hub.snapshot_download(
repo_id,
**kwargs,
)
def format_timestamp( def format_timestamp(