Issue to load HuggingFace Token Secret

[yfafi]

I have a problem when i deploy the lab. The pod "vllm-inference-service-predictor-xxxx" has a container "download-model" with the following error :
"Traceback (most recent call last):
File "", line 5, in
File "/opt/app-root/lib64/python3.9/site-packages/huggingface_hub/utils/_deprecation.py", line 101, in inner_f
return f(*args, **kwargs)
File "/opt/app-root/lib64/python3.9/site-packages/huggingface_hub/utils/_deprecation.py", line 31, in inner_f
return f(*args, **kwargs)
File "/opt/app-root/lib64/python3.9/site-packages/huggingface_hub/_login.py", line 126, in login
_login(token, add_to_git_credential=add_to_git_credential)
File "/opt/app-root/lib64/python3.9/site-packages/huggingface_hub/_login.py", line 404, in _login
token_info = whoami(token)
File "/opt/app-root/lib64/python3.9/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
return fn(*args, **kwargs)
File "/opt/app-root/lib64/python3.9/site-packages/huggingface_hub/hf_api.py", line 1791, in whoami
raise HTTPError(error_message, request=e.request, response=e.response) from e
requests.exceptions.HTTPError: Invalid user token. The token from HF_TOKEN environment variable is invalid. Note that HF_TOKEN takes precedence over hf auth login."

So i checked the huggingface-secret but the value for the key hftoken is 'None'.
In my file configuration, following the lab tutorial, i confirmed that i've added my hf token in the "values-secret-rag-llm-gitops.yaml"
secrets:

• name: hfmodel
  fields:
  • name: hftoken
    value: XXXXXXXXXXXXXXXXXXXXXXXXXXXX
• name: mssql
  fields:
  • name: sa-pass
    value: null
    onMissingValue: generate
    description: mssql password for sa user
• name: azuresql
  fields:
  • name: user
    value: adminuser
    description: server admin user for azure sql
  • name: password
    value: null
    onMissingValue: generate
    description: server admin password for azure sql
  • name: server
    value: rag-llm-gitops.database.windows.net
    description: server hostname

[dminnear-rh]

Thanks for making this issue. I just created a demo myself and can confirm that the HF token is no longer properly set in the demo environment. I'll take a look and hopefully have it fixed soon!

[dminnear-rh]

@yfafi still need to fix the demo environment, sorry! I didn't want to leave you with nothing though.

You will need to fork/clone the repo, which it seems like you already have, and add your HF token manually.

Create a file in your HOME directory at ~/values-secret-rag-llm-gitops.yaml. Add your HF token there (so it's not part of the git repo itself):

version: "2.0"

secrets:
  - name: hfmodel
    fields:
    - name: hftoken
      value: hf_xxxxxxxxxxxxxxxxxxxxxxxxxxx

You don't need anything else in this file. You can confirm the secret will be set to what you want by running ./pattern.sh common/scripts/display-secrets-info.sh from within the rag-llm-gitops repo. The output should have:

...
"parsed_secrets": {
    "hfmodel": {
        "annotations": {},
        "base64": [],
        "fields": {
            "hftoken": "hf_xxxxxxxxxxxxxxxxxxxxxxxxxxx"
        },
        "generate": [],
        "ini_file": {},
        "labels": {},
        "name": "hfmodel",
        "override": [],
        "paths": {},
        "target_namespaces": [],
        "type": "Opaque",
        "vault_mount": "secret",
        "vault_policies": {},
        "vault_prefixes": [
            "hub"
        ]
    }
},
...

If you see that then you know your secrets file is formatted correctly and in the right place.

Now to actually load the secrets:

1. Make sure you're logged into the demo environment in your terminal
2. Run ./pattern.sh make load-secrets

I'm still hoping to actually fix the demo environment soon, but in the meantime you can use your own HF token with that workaround