You may want to share your model with ML engineers around the world or simply download it. This is the way for you.
Before starting, uploading a model to Hugging face is the same as uploading a project to Github, but unlike general projects, models are large, so you need to install git lfs. Also, you need to install the transformers module because you need transformers-cli to upload to the Hugging face.
1. Upload environment settings
1-1. Install Transformers
Transformers are modules in python and can be installed through the pip command.
pip install transformers
If you are in an anaconda environment, you can install it through the conda command.
conda install -c huggingface transformers
1-2. Installing Git lfs
Connect to Git Large File Storage and click the Download button to download and install the git lfs extension installation file. Since it automatically detects the operating system environment when visiting the site, you can easily install it regardless of Windows, Mac, or Linux.
2. Log in to Hugging face
First, if you do not have an account on Hugging face, Sign up to the Hugging face homepage.
Then launch the command prompt (terminal) and log in to Hugging with transformers-cli. When you type the command and enter your ID and password, the token is saved on your personal computer.
transformers-cli login
If you are not changing your ID, this process only needs to be done once.
3. Upload to Hugging face
3-1. Create a repository to upload your model to the Hugging face.
3-1-A. How to use the command.
transformers-cli repo create your-model-name
# Put your model name in your-model-name.
To create an organization store, you can use the —organization option.
transformers-cli repo create your-model-name --organization your-org-name
# Put your model name in your-model-name.
# Put your organization name in your-org-name.
3-1-B. How to make it on the homepage
First, click the profile icon in the upper right and then click New model.
Select Owner (person or organization), enter the Model name, and click create model to create the repository. Notice, private storage is only available for paid accounts.
A model repository has been created on Huggigface.
3-2. Clone & Commit & Push
From here on, it is almost identical to the Git command.
First, install git lfs from the command window to make it possible to upload large files.
git lfs install
# If you do not do this,
# you will not be able to upload due to the capacity limit.
Clone the model repository using the Git command.
git clone https://huggingface.co/username/your-model-name
# Enter the repository owner for username.
# If it's not an organization repository, it's your Hugging face nickname.
# Put your model name in your-model-name.
Put the model and tokenizer files in the folder created by clone.
(If you have *.mar file, unzip the *.mar file first and proceed)
After moving to the cloned folder in the command window, check the files with the git status command.
cd clone-folder
# Go to the clone-folder.
git status
# Check the status in the folder.
After adding files with the git add command, commit occurs.
git add .
# Add all files in the folder.
git commit -m "message"
# Fires a commit indicating that the operation is complete.
# You can leave a message via -m.
Finally, put the commit into the Hugging face repository via git push. It takes quite a while because the model is large.
git push
4. Make model card
If you go back to the Hugging face repository, you can see that the model has been uploaded. Tags were automatically run, and there was an API to test the model.
Finally, click the Create Model Card button, write an introduction to your model and what you want to say, and click Commit new file to complete
