OpenAI Finetuning On Gorilla With Wandb
ChatGPT-3.5 Fine-tuning - Gorrilla api
Fine-tuning ChatGPT-3.5 on the Gorilla api dataset to try and improve its performance
OpenAI ChatGPT-3.5 fine-tuning docs are here
Warning!
This fine-tuning script will train 7.2 million tokens on OpenAI, check if you're willing to pay that before proceeding :)
Data
Download the Gorrilla huggingface api training data, you can find all the Gorilla training data here
Load the data
Data Parsing
Parse the training data instructions
Format the training samples with the correct format to mirror the Gorilla paper
Print a sample of what will get passed to OpenAI for fine-tuning
Save the training data
OpenAI data verification script
Start Fine-tuning ChatGPT-3.5
Create an OpenAI training file
Create your fine-tuning job
Log the results to Weights & Biases when the model is finished training
(temporarily install openai from a fork until this PR to update the wandb logger is merged in openai: https://github.com/openai/openai-python/pull/590)
Run openai wandb sync to sync your openai results to W&B
Other useful openai commands
List 10 fine-tuning jobs
Retrieve the state of a fine-tune
List up to 10 events from a fine-tuning job