FunctionGemma (270M) LMStudio
unsloth-notebooksunslothnb
Export
To run this, press "Runtime" and press "Run all" on a free Tesla T4 Google Colab instance!
To install Unsloth on your local device, follow our guide. This notebook is licensed LGPL-3.0.
You will learn how to do data prep, how to train, how to run the model, & how to save it
Unsloth + LM Studio
You can now fine-tune models with Unsloth and import them into LM Studio for local inference!
LM Studio is a powerful yet friendly desktop application that makes it easy to discover, download, and run models locally, creating a smooth end-to-end workflow for fine-tuning and deployment.
First, let's install LM Studio from here!
Install Unsloth and other dependencies
[1]
Unsloth
[2]
🦥 Unsloth: Will patch your computer to enable 2x faster free finetuning. 🦥 Unsloth Zoo will now patch everything to make training faster! ==((====))== Unsloth 2025.12.9: Fast Gemma3 patching. Transformers: 4.57.3. \\ /| Tesla T4. Num GPUs = 1. Max memory: 14.741 GB. Platform: Linux. O^O/ \_/ \ Torch: 2.9.0+cu126. CUDA: 7.5. CUDA Toolkit: 12.6. Triton: 3.5.0 \ / Bfloat16 = FALSE. FA [Xformers = 0.0.33.post1. FA2 = False] "-____-" Free license: http://github.com/unslothai/unsloth Unsloth: Fast downloading is enabled - ignore downloading bars which are red colored! Unsloth: Using float16 precision for gemma3 won't work! Using float32. Unsloth: Gemma3 does not support SDPA - switching to fast eager.
model.safetensors: 0%| | 0.00/536M [00:00<?, ?B/s]
generation_config.json: 0%| | 0.00/205 [00:00<?, ?B/s]
tokenizer_config.json: 0.00B [00:00, ?B/s]
tokenizer.model: 0%| | 0.00/4.69M [00:00<?, ?B/s]
tokenizer.json: 0%| | 0.00/33.4M [00:00<?, ?B/s]
added_tokens.json: 0%| | 0.00/63.0 [00:00<?, ?B/s]
special_tokens_map.json: 0%| | 0.00/714 [00:00<?, ?B/s]
chat_template.jinja: 0.00B [00:00, ?B/s]
We now add LoRA adapters so we only need to update a small amount of parameters!
[3]
Unsloth: Making `model.base_model.model.model` require gradients
Formatting is very important in the functiongemma for tool-calling.
For the general conversation, each role (developer, user, model) is wrapped with <start_of_turn>{role} ... <end_of_turn>
[4]
=== Example 1: Basic turns === <bos><start_of_turn>developer You are a helpful assistant.<end_of_turn> <start_of_turn>user Hello, who are you?<end_of_turn>
For tool calling, in the developer turn, <start_function_declaration>declaration:get_weather{...}<end_function_declaration> encodes the full function spec (name, description, parameters) so the model knows what tools it can call and how to format arguments.
[5]
=== Example 2: Tool declarations ===
<bos><start_of_turn>developer
You are a weather assistant.<start_function_declaration>declaration:get_weather{description:<escape>Get the current weather for a given city.<escape>,parameters:{properties:{city:{description:<escape>City name, e.g. 'Tokyo'.<escape>,type:<escape>STRING<escape>}},required:[<escape>city<escape>],type:<escape>OBJECT<escape>}}<end_function_declaration><end_of_turn>
<start_of_turn>user
What is the weather in Tokyo?<end_of_turn>
For tool calling + result of the tool call + LLM answer, the user turn is plain text; the next model turn embeds <start_function_call>call:get_weather{...}<end_function_call> and a matching <start_function_response>response:get_weather{...}<end_function_response>, modeling the whole “call tool → receive result → answer user” loop inside the prompt.
[6]
=== Example 3: User → Model → Tool → Model ===
<bos><start_of_turn>developer
You are a weather assistant.<start_function_declaration>declaration:get_weather{description:<escape>Get the current weather for a given city.<escape>,parameters:{properties:{city:{description:<escape>City name, e.g. 'Tokyo'.<escape>,type:<escape>STRING<escape>}},required:[<escape>city<escape>],type:<escape>OBJECT<escape>}}<end_function_declaration><end_of_turn>
<start_of_turn>user
What is the weather in Tokyo?<end_of_turn>
<start_of_turn>model
<start_function_call>call:get_weather{city:<escape>Tokyo<escape>}<end_function_call><start_function_response>response:get_weather{value:<escape>{"city": "Tokyo", "temp_c": 25, "condition": "sunny"}<escape>}<end_function_response>It is currently 25°C and sunny in Tokyo.<end_of_turn>
Lastly, for our thinking process which we will do in this notebook, since they did not support it in the chat template, we will put it inside the assistant response instead. We will use the tag <think>...</think>. The <think>...</think> blocks inside the model turn are just tagged text that represent internal reasoning; they appear before and after the tool interaction.
[7]
=== Thinking + Tools ===
<bos><start_of_turn>developer
You are a shopping assistant. Use tools when you need detailed Amazon product data such as price and specifications.<start_function_declaration>declaration:get_amazon_product_details{description:<escape>Retrieves comprehensive product information from Amazon, including title, price, description, specifications, and availability.<escape>,parameters:{properties:{asin:{description:<escape>The Amazon ASIN of the product.<escape>,type:<escape>STRING<escape>}},required:[<escape>asin<escape>],type:<escape>OBJECT<escape>}}<end_function_declaration><end_of_turn>
<start_of_turn>user
Is the espresso machine with ASIN B0XYZ12345 any good for home use?<end_of_turn>
<start_of_turn>model
<think>User is asking for an opinion, but I need factual product details first such as price, features, and reviews. I should call the Amazon product details tool with the provided ASIN.</think><start_function_call>call:get_amazon_product_details{asin:<escape>B0XYZ12345<escape>}<end_function_call><start_function_response>response:get_amazon_product_details{value:<escape>{"title": "Home Pro Espresso 3000", "price": 199.99, "pressure_bar": 15, "features": ["steam wand", "single and double shot baskets"], "pros": ["good crema", "compact"], "cons": ["a bit noisy"]}<escape>}<end_function_response><think>Tool response shows a mid-range price and standard 15 bar pressure. Features and pros/cons indicate it’s fine for home espresso but not a high-end machine for enthusiasts.</think>
Based on the product details, the Home Pro Espresso 3000 (ASIN B0XYZ12345) is a solid option for home use. It offers 15-bar pressure, a steam wand, and both single and double shot baskets, which are enough for everyday lattes and cappuccinos. It’s compact and produces good crema, but it can be a bit noisy. If you want a convenient, reasonably priced home machine, it should work well; if you’re very picky about espresso or plan to upgrade grinders and accessories, you might eventually want something more advanced.<end_of_turn>
Data Prep
We now use the built-in functiongemma format for conversation style finetunes. We use TxT360-3efforts dataset. This dataset contains SFT dataset for tool-calling with thinking reasoning, which the functiongemma original model did not have this thinking capability.
Since this dataset contains more than 1 million examples, we will take only the first 50000 using streaming=True, bypassing downloading the entire dataset.
[8]
README.md: 0.00B [00:00, ?B/s]
Resolving data files: 0%| | 0/26 [00:00<?, ?it/s]
Resolving data files: 0%| | 0/26 [00:00<?, ?it/s]
Let's check one example of the dataset.
[9]
'[{"role": "system", "tools": [{"name": "qrcodepro", "description": "Generates a professional QR code with customizable properties and settings.", "parameters": {"type": "object", "properties": {"text": {"description": "The text to encode into the QR code.", "type": "string", "default": "https://www.digicatech.com"}, "validate": {"description": "Whether to validate the input text. Defaults to None.", "type": "string", "default": true}, "setlabel": {"description": "Whether to set a label on the QR code. Defaults to None.", "type": "string", "default": false}, "forecolor": {"description": "The foreground color of the QR code in hexadecimal format without the # prefix. Defaults to \'000000\'.", "type": "string", "default": "000000"}, "type": {"description": "The output file type for the QR code (\'png\', \'svg\', or \'eps\'). Defaults to \'svg\'.", "type": "string", "default": "svg"}, "labeltext": {"description": "The text to use as a label in the QR code. Defaults to None.", "type": "string", "default": ""}, "size": {"description": "The size of the output image in pixels, with a maximum of 4000. Defaults to 150.", "type": "string", "default": "150"}, "labelalign": {"description": "The alignment of the label (\'left\', \'right\', or \'center\'). Defaults to \'center\'.", "type": "string", "default": "center"}, "backcolor": {"description": "The background color of the QR code in hexadecimal format without the # prefix. Defaults to \'FFFFFF\'.", "type": "string", "default": "FFFFFF"}, "level": {"description": "The error correction level of the QR code (\'L\', \'M\', \'Q\', \'H\'). Defaults to \'M\'.", "type": "string", "default": "M"}}}}, {"name": "get_products_in_category", "description": "Fetches products in a specified category from a remote catalog using the RapidAPI service.", "parameters": {"type": "object", "properties": {"category": {"description": "The category from which to fetch products.", "type": "string", "default": ""}, "limit": {"description": "The number of products to retrieve.", "type": "integer", "default": ""}, "skip": {"description": "The number of products to skip before starting to retrieve.", "type": "integer", "default": ""}}}}, {"name": "uuidv3_generator", "description": "Generates a UUIDv3 based on the provided namespace and name using the Toolbench RapidAPI service.", "parameters": {"type": "object", "properties": {"namespace": {"description": "The namespace to use for generating the UUIDv3. Allowed values are \'dns\', \'url\', \'oid\', \'x500\', and \'nil\'.", "type": "string", "default": "nil"}, "name": {"description": "The name to use for generating the UUIDv3.", "type": "string", "default": "test"}}}}, {"name": "get_product", "description": "Fetches product details from the RapidAPI endpoint using the provided product ID and API key.", "parameters": {"type": "object", "properties": {"is_id": {"description": "The product ID to fetch details for.", "type": "integer", "default": ""}}}}]}, {"role": "user", "content": "Generate a QR code for \'My App\' with a size of 300 and a label saying \'Download Now\', and also retrieve details for product 11121."}, {"role": "assistant", "think_fast": "We need to do two tasks: generate QR code with given parameters and retrieve product details for product ID 11121.\\n\\nWe have two functions: qrcodepro and get_product. So need to call both.\\n\\nFirst, generate QR code: text likely is \'My App\'? Actually QR code text to encode is \'My App\'. size 300. set label: setlabel true? There\'s setlabel param (maybe boolean). labeltext param \'Download Now\'. labelalign maybe default center. forecolor default black, etc.\\n\\nWe\'ll call qrcodepro with text=\'My App\', size=\'300\', setlabel=\'true\' (maybe string? but default type seems string). labeltext=\'Download Now\'.\\n\\nSecond, call get_product with is_id=11121.\\n\\nWe need to output function calls then provide final answer summarizing.\\n\\n", "tool_calls": [{"name": "qrcodepro", "arguments": "{\\n \\"text\\": \\"My App\\",\\n \\"size\\": \\"300\\",\\n \\"setlabel\\": \\"true\\",\\n \\"labeltext\\": \\"Download Now\\"\\n}"}]}]' If we look closely, our dataset is in the form of string. Secondly, the format of this dataset is different with the one that is required for functiongemma. Especially the one when we need to pass the tools. We will use our defined function to do this :
[10]
This will separate the conversation and the tools which we will pass to tokenizer.apply_chat_template.
Let's transform all of our dataset
[11]
Map: 0%| | 0/50000 [00:00<?, ? examples/s]
Filter: 0%| | 0/50000 [00:00<?, ? examples/s]
Dataset size after filtering: 23470
Let's see the resulting output
[12]
"<start_of_turn>developer\nYou are a model that can do function calling with the following functions<start_function_declaration>declaration:qrcodepro{description:<escape>Generates a professional QR code with customizable properties and settings.<escape>,parameters:{properties:{backcolor:{description:<escape>The background color of the QR code in hexadecimal format without the # prefix. Defaults to 'FFFFFF'.<escape>,type:<escape>STRING<escape>},forecolor:{description:<escape>The foreground color of the QR code in hexadecimal format without the # prefix. Defaults to '000000'.<escape>,type:<escape>STRING<escape>},labelalign:{description:<escape>The alignment of the label ('left', 'right', or 'center'). Defaults to 'center'.<escape>,type:<escape>STRING<escape>},labeltext:{description:<escape>The text to use as a label in the QR code. Defaults to None.<escape>,type:<escape>STRING<escape>},level:{description:<escape>The error correction level of the QR code ('L', 'M', 'Q', 'H'). Defaults to 'M'.<escape>,type:<escape>STRING<escape>},setlabel:{description:<escape>Whether to set a label on the QR code. Defaults to None.<escape>,type:<escape>STRING<escape>},size:{description:<escape>The size of the output image in pixels, with a maximum of 4000. Defaults to 150.<escape>,type:<escape>STRING<escape>},text:{description:<escape>The text to encode into the QR code.<escape>,type:<escape>STRING<escape>},validate:{description:<escape>Whether to validate the input text. Defaults to None.<escape>,type:<escape>STRING<escape>}},type:<escape>OBJECT<escape>}}<end_function_declaration><start_function_declaration>declaration:get_products_in_category{description:<escape>Fetches products in a specified category from a remote catalog using the RapidAPI service.<escape>,parameters:{properties:{category:{description:<escape>The category from which to fetch products.<escape>,type:<escape>STRING<escape>},limit:{description:<escape>The number of products to retrieve.<escape>,type:<escape>INTEGER<escape>},skip:{description:<escape>The number of products to skip before starting to retrieve.<escape>,type:<escape>INTEGER<escape>}},type:<escape>OBJECT<escape>}}<end_function_declaration><start_function_declaration>declaration:uuidv3_generator{description:<escape>Generates a UUIDv3 based on the provided namespace and name using the Toolbench RapidAPI service.<escape>,parameters:{properties:{name:{description:<escape>The name to use for generating the UUIDv3.<escape>,type:<escape>STRING<escape>},namespace:{description:<escape>The namespace to use for generating the UUIDv3. Allowed values are 'dns', 'url', 'oid', 'x500', and 'nil'.<escape>,type:<escape>STRING<escape>}},type:<escape>OBJECT<escape>}}<end_function_declaration><start_function_declaration>declaration:get_product{description:<escape>Fetches product details from the RapidAPI endpoint using the provided product ID and API key.<escape>,parameters:{properties:{is_id:{description:<escape>The product ID to fetch details for.<escape>,type:<escape>INTEGER<escape>}},type:<escape>OBJECT<escape>}}<end_function_declaration><end_of_turn>\n<start_of_turn>user\nGenerate a QR code for 'My App' with a size of 300 and a label saying 'Download Now', and also retrieve details for product 11121.<end_of_turn>\n<start_of_turn>model\n<think>We need to do two tasks: generate QR code with given parameters and retrieve product details for product ID 11121.\n\nWe have two functions: qrcodepro and get_product. So need to call both.\n\nFirst, generate QR code: text likely is 'My App'? Actually QR code text to encode is 'My App'. size 300. set label: setlabel true? There's setlabel param (maybe boolean). labeltext param 'Download Now'. labelalign maybe default center. forecolor default black, etc.\n\nWe'll call qrcodepro with text='My App', size='300', setlabel='true' (maybe string? but default type seems string). labeltext='Download Now'.\n\nSecond, call get_product with is_id=11121.\n\nWe need to output function calls then provide final answer summarizing.\n\n</think><start_function_call>call:qrcodepro{labeltext:<escape>Download Now<escape>,setlabel:<escape>true<escape>,size:<escape>300<escape>,text:<escape>My App<escape>}<end_function_call><start_function_response>" [13]
Unsloth: Switching to float32 training since model cannot work with float16
Unsloth: Tokenizing ["text"] (num_proc=4): 0%| | 0/23470 [00:00<?, ? examples/s]
The model is already on multiple devices. Skipping the move to device specified in `args`.
🦥 Unsloth: Padding-free auto-enabled, enabling faster training.
We also use Unsloth's train_on_completions method to only train on the assistant outputs and ignore the loss on the user's inputs. This helps increase accuracy of finetunes!
[14]
Map (num_proc=6): 0%| | 0/23470 [00:00<?, ? examples/s]
Let's verify masking the instruction part is done! Let's print the 100th row again.
[15]
"<bos><start_of_turn>developer\nYou are a model that can do function calling with the following functions<start_function_declaration>declaration:live_players_rankings{description:<escape>Retrieves live tennis rankings for a given number of players, including position, points, and information about the last match played in the current active tournament.<escape>,parameters:{properties:{n_player:{description:<escape>The number of players for which to retrieve the live rankings, with a maximum of 500.<escape>,type:<escape>STRING<escape>}},type:<escape>OBJECT<escape>}}<end_function_declaration><start_function_declaration>declaration:matchh2hduel{description:<escape>Retrieves head-to-head duel information for a specific basketball match.<escape>,parameters:{properties:{is_id:{description:<escape>The ID of the basketball match for which to retrieve head-to-head duel information.<escape>,type:<escape>INTEGER<escape>}},type:<escape>OBJECT<escape>}}<end_function_declaration><start_function_declaration>declaration:teamimage{description:<escape>Fetch the image in PNG format for a given team using the MMA RapidAPI.<escape>,parameters:{properties:{is_id:{description:<escape>The ID of the team for which the image is being retrieved.<escape>,type:<escape>INTEGER<escape>}},type:<escape>OBJECT<escape>}}<end_function_declaration><start_function_declaration>declaration:leagueplayersstatistics{description:<escape>Fetches player statistics for a specific Football league and season based on the given parameters.<escape>,parameters:{properties:{accumulation:{description:<escape>Type of accumulation. Choices are 'total', 'perGame', or 'per90'. Default is 'total'.<escape>,type:<escape>STRING<escape>},fields:{description:<escape>Specific fields to include, up to 6. Only used when not specifying a group. Field options include 'successfulDribblesPercentage', 'goals', 'blockedShots', etc.<escape>,type:<escape>STRING<escape>},filters:{description:<escape>Filters to apply, such as player position, match type, appearances, age, preferred foot, team, and nationality. See the noisy docstring for detailed filter options.<escape>,type:<escape>STRING<escape>},group:{description:<escape>Statistic group to retrieve. Choices are 'summary', 'attack', 'defence', 'passing', 'goalkeeper'. Default is 'summary'.<escape>,type:<escape>STRING<escape>},limit:{description:<escape>Number of players to include per page.<escape>,type:<escape>INTEGER<escape>},minapps:{description:<escape>If set to True, only includes players with a minimum number of appearances.<escape>,type:<escape>STRING<escape>},order:{description:<escape>Sorting order of the response properties. Default is '-rating'. Prefix with '-' for descending order, leave as is for ascending.<escape>,type:<escape>STRING<escape>},page:{description:<escape>One-based page number.<escape>,type:<escape>INTEGER<escape>},seasonid:{description:<escape>ID of the season to retrieve player statistics for.<escape>,type:<escape>INTEGER<escape>},tournamentid:{description:<escape>Unique ID of the tournament to retrieve player statistics for.<escape>,type:<escape>INTEGER<escape>}},type:<escape>OBJECT<escape>}}<end_function_declaration><start_function_declaration>declaration:entry_list{description:<escape>Fetches the entry list for a given golf tournament using the specified RapidAPI key.<escape>,parameters:{properties:{tournament_id:{description:<escape>The ID of the tournament for which to fetch the entry list.<escape>,type:<escape>INTEGER<escape>}},type:<escape>OBJECT<escape>}}<end_function_declaration><start_function_declaration>declaration:nhl_box_score{description:<escape>Retrieve NHL game box score data for a specific game using the given game ID.<escape>,parameters:{properties:{is_id:{description:<escape>The ID of the NHL game for which the box score data is to be retrieved.<escape>,type:<escape>STRING<escape>}},type:<escape>OBJECT<escape>}}<end_function_declaration><end_of_turn>\n<start_of_turn>user\nCan you provide the live rankings for the top 10 tennis players in the current tournament, and also fetch the head-to-head duel stats for basketball match ID 12345?<end_of_turn>\n<start_of_turn>model\n<think>We need two pieces: live tennis rankings top 10 players. Use live_players_rankings with n_player=10. Also need head-to-head duel stats for basketball match ID 12345. Use matchh2hduel with is_id=12345.\n\nWe should make two function calls sequentially.</think><start_function_call>call:live_players_rankings{n_player:<escape>10<escape>}<end_function_call><start_function_response>" Now let's print the masked out example - you should see only the answer is present:
[16]
['--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------<think>User wants DNA sequence with ID HQ664368, format FASTA, upstream 250 bases. Use function get_dna_sequence with sequence_id "HQ664368", file_format "fasta", upstream_bases "250".</think><start_function_call>call:get_dna_sequence{file_format:<escape>fasta<escape>,sequence_id:<escape>HQ664368<escape>,upstream_bases:<escape>250<escape>}<end_function_call><start_function_response>'] [17]
GPU = Tesla T4. Max memory = 14.741 GB. 0.832 GB of memory reserved.
Let's train the model! To resume a training run, set trainer.train(resume_from_checkpoint = True)
[18]
==((====))== Unsloth - 2x faster free finetuning | Num GPUs used = 1 \\ /| Num examples = 23,470 | Num Epochs = 1 | Total steps = 500 O^O/ \_/ \ Batch size per device = 4 | Gradient accumulation steps = 2 \ / Data Parallel GPUs = 1 | Total batch size (4 x 2 x 1) = 8 "-____-" Trainable parameters = 30,375,936 of 298,474,112 (10.18% trained)
Unsloth: Will smartly offload gradients to save VRAM!
We will take only the first two messages, which is the system role and the user role while also passing the tools to the prompt.
[19]
<bos><start_of_turn>developer
You are a model that can do function calling with the following functions<start_function_declaration>declaration:qrcodepro{description:<escape>Generates a professional QR code with customizable properties and settings.<escape>,parameters:{properties:{backcolor:{description:<escape>The background color of the QR code in hexadecimal format without the # prefix. Defaults to 'FFFFFF'.<escape>,type:<escape>STRING<escape>},forecolor:{description:<escape>The foreground color of the QR code in hexadecimal format without the # prefix. Defaults to '000000'.<escape>,type:<escape>STRING<escape>},labelalign:{description:<escape>The alignment of the label ('left', 'right', or 'center'). Defaults to 'center'.<escape>,type:<escape>STRING<escape>},labeltext:{description:<escape>The text to use as a label in the QR code. Defaults to None.<escape>,type:<escape>STRING<escape>},level:{description:<escape>The error correction level of the QR code ('L', 'M', 'Q', 'H'). Defaults to 'M'.<escape>,type:<escape>STRING<escape>},setlabel:{description:<escape>Whether to set a label on the QR code. Defaults to None.<escape>,type:<escape>STRING<escape>},size:{description:<escape>The size of the output image in pixels, with a maximum of 4000. Defaults to 150.<escape>,type:<escape>STRING<escape>},text:{description:<escape>The text to encode into the QR code.<escape>,type:<escape>STRING<escape>},validate:{description:<escape>Whether to validate the input text. Defaults to None.<escape>,type:<escape>STRING<escape>}},type:<escape>OBJECT<escape>}}<end_function_declaration><start_function_declaration>declaration:get_products_in_category{description:<escape>Fetches products in a specified category from a remote catalog using the RapidAPI service.<escape>,parameters:{properties:{category:{description:<escape>The category from which to fetch products.<escape>,type:<escape>STRING<escape>},limit:{description:<escape>The number of products to retrieve.<escape>,type:<escape>INTEGER<escape>},skip:{description:<escape>The number of products to skip before starting to retrieve.<escape>,type:<escape>INTEGER<escape>}},type:<escape>OBJECT<escape>}}<end_function_declaration><start_function_declaration>declaration:uuidv3_generator{description:<escape>Generates a UUIDv3 based on the provided namespace and name using the Toolbench RapidAPI service.<escape>,parameters:{properties:{name:{description:<escape>The name to use for generating the UUIDv3.<escape>,type:<escape>STRING<escape>},namespace:{description:<escape>The namespace to use for generating the UUIDv3. Allowed values are 'dns', 'url', 'oid', 'x500', and 'nil'.<escape>,type:<escape>STRING<escape>}},type:<escape>OBJECT<escape>}}<end_function_declaration><start_function_declaration>declaration:get_product{description:<escape>Fetches product details from the RapidAPI endpoint using the provided product ID and API key.<escape>,parameters:{properties:{is_id:{description:<escape>The product ID to fetch details for.<escape>,type:<escape>INTEGER<escape>}},type:<escape>OBJECT<escape>}}<end_function_declaration><end_of_turn>
<start_of_turn>user
Generate a QR code for 'My App' with a size of 300 and a label saying 'Download Now', and also retrieve details for product 11121.<end_of_turn>
<start_of_turn>model
<think>User wants two pieces of info:
1. Create QR code for 'My App' size 300, label says download now, size 300. Use qrcodepro function.
2. Retrieve product details for product ID 11121 using get_product with is_id 11121.
We need to make two function calls.
Let's call qrcodepro with size = '300, labeltext='Download Now', setlabel='Download Now', font='FFFFFF'. Probably also maybe another label type standard default? Default font is 'FFFFFF' and labeltext default 'Upgrade'. We'll just set labeltext='Download Now'. , size maybe set to 300.
Call get_products_in_category with is_id=11121.
Proceed.</think><start_function_call>call:qrcodepro{labeltext:<escape>Download Now<escape>,size:<escape>300<escape>,setlabel:<escape>Download Now<escape>,text:<escape>Download Now<escape>}<end_function_call><start_function_response>
[20]
('functiongemma/tokenizer_config.json',
, 'functiongemma/special_tokens_map.json',
, 'functiongemma/chat_template.jinja',
, 'functiongemma/tokenizer.model',
, 'functiongemma/added_tokens.json',
, 'functiongemma/tokenizer.json') Now if you want to load the LoRA adapters we just saved for inference, set False to True:
[21]
Merge LoRA adapters to base model and save to disk
We also support saving to float16 directly. Select merged_16bit for float16 or merged_4bit for int4. We also allow lora adapters as a fallback.
Use push_to_hub_merged to upload to your Hugging Face account! You can go to https://huggingface.co/settings/tokens for your personal tokens. See our docs for more deployment options.
Using these methods will require an extra GGUF conversion step to use the models in LM Studio. You may do one of the following:
- Use our natively supported GGUF/llama.cpp conversion - scroll down to the next section!
- Download the merged FP16 model and do the GGUF conversion locally with the
convert-hf-to-gguf.pyscript from llama.cpp.
Once you've saved the GGUF file locally, add it to your LM Studio model directory to chat with it directly in LM Studio. To import your file into LM Studio's directory, follow the LM Studio directory structure.
[22]
Use native GGUF conversion and save to disk
To convert your model to GGUF / llama.cpp, we support it natively now for all models! For now, you can convert easily to Q8_0, F16 or BF16 precision. Q4_K_M for 4bit will come later!
[23]
Unsloth: ##### The current model auto adds a BOS token. Unsloth: ##### Your chat template has a BOS token. We shall remove it temporarily.
Unsloth: Merging model weights to 16-bit format... Found HuggingFace hub cache directory: /root/.cache/huggingface/hub Checking cache directory for required files...
Unsloth: Copying 1 files from cache to `functiongemma-finetune`: 100%|██████████| 1/1 [00:09<00:00, 9.54s/it]
Successfully copied all 1 files from cache to `functiongemma-finetune` Checking cache directory for required files...
Unsloth: Copying 1 files from cache to `functiongemma-finetune`: 100%|██████████| 1/1 [00:00<00:00, 198.58it/s]
Successfully copied all 1 files from cache to `functiongemma-finetune`
Unsloth: Preparing safetensor model files: 100%|██████████| 1/1 [00:00<00:00, 9198.04it/s] Unsloth: Merging weights into 16bit: 100%|██████████| 1/1 [00:05<00:00, 5.17s/it]
Unsloth: Merge process complete. Saved to `/content/functiongemma-finetune` Unsloth: Converting to GGUF format... ==((====))== Unsloth: Conversion from HF to GGUF information \\ /| [0] Installing llama.cpp might take 3 minutes. O^O/ \_/ \ [1] Converting HF to GGUF f16 might take 3 minutes. \ / [2] Converting GGUF f16 to ['q8_0'] might take 10 minutes each. "-____-" In total, you will have to wait at least 16 minutes. Unsloth: Installing llama.cpp. This might take 3 minutes... Unsloth: Updating system package directories Unsloth: All required system packages already installed! Unsloth: Install llama.cpp and building - please wait 1 to 3 minutes Unsloth: Cloning llama.cpp repository Unsloth: Install GGUF and other packages Unsloth: Successfully installed llama.cpp! Unsloth: Preparing converter script... Unsloth: [1] Converting model into f16 GGUF format. This might take 3 minutes... Unsloth: Initial conversion completed! Files: ['functiongemma-270m-it.F16.gguf'] Unsloth: [2] Converting GGUF f16 into q8_0. This might take 10 minutes...
Unsloth: ##### The current model auto adds a BOS token. Unsloth: ##### We removed it in GGUF's chat template for you.
Unsloth: Model files cleanup... Unsloth: All GGUF conversions completed successfully! Generated files: ['functiongemma-270m-it.Q8_0.gguf'] Unsloth: No Ollama template mapping found for model 'unsloth/functiongemma-270m-it'. Skipping Ollama Modelfile Unsloth: example usage for text only LLMs: llama-cli --model functiongemma-270m-it.Q8_0.gguf -p "why is the sky blue?"
Likewise, if you want to instead push to GGUF to your Hugging Face account, set if False to if True and add your Hugging Face token and upload location!
Once pushed to your Hugging Face account, make the model public to find it through LM Studio's Model Search. In LM Studio's Model Search, you can find the model by pasting the Hugging Face URL into the search bar.
[24]
Now, use the functiongemma-finetune.gguf file or functiongemma-finetune-Q4_K_M.gguf file in LM Studio.
For example before finetuning, we prompt "Create a Game", and we get: 
After fine-tuning and using the same prompt we get:
And we're done! If you have any questions on Unsloth, we have a Discord channel! If you find any bugs or want to keep updated with the latest LLM stuff, or need help, join projects etc, feel free to join our Discord!
Some other resources:
- Train your own reasoning model - Llama GRPO notebook Free Colab
- Llama 3.2 Vision finetuning - Radiography use case. Free Colab
- See notebooks for DPO, ORPO, Continued pretraining, conversational finetuning and more on our documentation!
This notebook and all Unsloth notebooks are licensed LGPL-3.0.