Charactercodex Llamafile
🧩 Quizzes and Adventures 🏰 with Character Codex and llamafile
Let's build something fun with Character Codex, a newly released dataset featuring popular characters from a wide array of media types and genres...
We'll be using Haystack for orchestration and llamafile to run our models locally.
We will first build a simple quiz game, in which the user is asked to guess the character based on some clues. Then we will try to get two characters to interact in a chat and maybe even have an adventure together!
Preparation
Install dependencies
Load and look at the Character Codex dataset
/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_token.py:89: UserWarning: The secret `HF_TOKEN` does not exist in your Colab secrets. To authenticate with the Hugging Face Hub, create a token in your settings tab (https://huggingface.co/settings/tokens), set it as secret in your Google Colab and restart your session. You will be able to reuse this secret in all of your notebooks. Please note that authentication is recommended but still optional to access public models or datasets. warnings.warn(
Downloading readme: 0%| | 0.00/4.35k [00:00<?, ?B/s]
Downloading data: 0%| | 0.00/11.2M [00:00<?, ?B/s]
Generating train split: 0 examples [00:00, ? examples/s]
15939
{'media_type': 'Webcomics',
, 'genre': 'Fantasy Webcomics',
, 'character_name': 'Alana',
, 'media_source': 'Saga',
, 'description': 'Alana is one of the main characters from the webcomic "Saga." She is a strong-willed and fiercely protective mother who is on the run with her family in a war-torn galaxy. The story blends elements of fantasy and science fiction, creating a rich and complex narrative.',
, 'scenario': "You are a fellow traveler in the galaxy needing help, and Alana offers her assistance while sharing stories of her family's struggles and triumphs."} Ok, each row of this dataset contains some information about a character.
It also includes a creative scenario, which we will not use.
llamafile: download and run the model
For our experiments, we will be using the Llama-3-8B-Instruct model: a small but good language model.
llamafile is a project by Mozilla that simplifies access to LLMs. It wraps both the model and the inference engine in a single executable file.
We will use it to run our model.
llamafile is meant to run on standard computers. We will do some tricks to make it work on Colab. For instructions on how to run it on your PC, check out the docs and Haystack-llamafile integration page.
--2024-06-20 09:53:30-- https://huggingface.co/Mozilla/Meta-Llama-3-8B-Instruct-llamafile/resolve/main/Meta-Llama-3-8B-Instruct.Q5_K_M.llamafile Resolving huggingface.co (huggingface.co)... 18.239.50.103, 18.239.50.80, 18.239.50.49, ... Connecting to huggingface.co (huggingface.co)|18.239.50.103|:443... connected. HTTP request sent, awaiting response... 302 Found Location: https://cdn-lfs-us-1.huggingface.co/repos/e3/ee/e3eefe425bce2ecb595973e24457616c48776aa0665d9bab33a29b582f3dfdf0/23365cb45398a3c568dda780a404b5f9a847b865d8341ec500ca3063a1f99eed?response-content-disposition=inline%3B+filename*%3DUTF-8%27%27Meta-Llama-3-8B-Instruct.Q5_K_M.llamafile%3B+filename%3D%22Meta-Llama-3-8B-Instruct.Q5_K_M.llamafile%22%3B&Expires=1719136410&Policy=eyJTdGF0ZW1lbnQiOlt7IkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MTcxOTEzNjQxMH19LCJSZXNvdXJjZSI6Imh0dHBzOi8vY2RuLWxmcy11cy0xLmh1Z2dpbmdmYWNlLmNvL3JlcG9zL2UzL2VlL2UzZWVmZTQyNWJjZTJlY2I1OTU5NzNlMjQ0NTc2MTZjNDg3NzZhYTA2NjVkOWJhYjMzYTI5YjU4MmYzZGZkZjAvMjMzNjVjYjQ1Mzk4YTNjNTY4ZGRhNzgwYTQwNGI1ZjlhODQ3Yjg2NWQ4MzQxZWM1MDBjYTMwNjNhMWY5OWVlZD9yZXNwb25zZS1jb250ZW50LWRpc3Bvc2l0aW9uPSoifV19&Signature=swp5azcPl0FOe5CuStFZn1hmF0SHimPUOLwOqHd2ZAnoFJuYKDjhK7ESplRDWJdma9QWYOQyaCG23wkX18urieav%7E8OdxzwLaKLhL2YFx3L6RMwGEKWjrG-ql-LDfd2I1U4AcSXJZR5zHSBDmYql9M9hXKsvXHVkraIMS-cDx0ihj3s7yu4gbjUfE3SPg49aStq00ORcQnDV90mXxeheM6UjRymLRBdlxI3PCpAjzvyExcmZSgBU5vCnKtAEy5b65%7EzQoX5TVQTzQXjE9x8Qr2%7EAONSc7wy671HWYPRKNgZDrH3NJy90uFp38GKiQtab7hAy6fUlL358OQYhHzu4-Q__&Key-Pair-Id=K2FPYV99P2N66Q [following] --2024-06-20 09:53:30-- https://cdn-lfs-us-1.huggingface.co/repos/e3/ee/e3eefe425bce2ecb595973e24457616c48776aa0665d9bab33a29b582f3dfdf0/23365cb45398a3c568dda780a404b5f9a847b865d8341ec500ca3063a1f99eed?response-content-disposition=inline%3B+filename*%3DUTF-8%27%27Meta-Llama-3-8B-Instruct.Q5_K_M.llamafile%3B+filename%3D%22Meta-Llama-3-8B-Instruct.Q5_K_M.llamafile%22%3B&Expires=1719136410&Policy=eyJTdGF0ZW1lbnQiOlt7IkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MTcxOTEzNjQxMH19LCJSZXNvdXJjZSI6Imh0dHBzOi8vY2RuLWxmcy11cy0xLmh1Z2dpbmdmYWNlLmNvL3JlcG9zL2UzL2VlL2UzZWVmZTQyNWJjZTJlY2I1OTU5NzNlMjQ0NTc2MTZjNDg3NzZhYTA2NjVkOWJhYjMzYTI5YjU4MmYzZGZkZjAvMjMzNjVjYjQ1Mzk4YTNjNTY4ZGRhNzgwYTQwNGI1ZjlhODQ3Yjg2NWQ4MzQxZWM1MDBjYTMwNjNhMWY5OWVlZD9yZXNwb25zZS1jb250ZW50LWRpc3Bvc2l0aW9uPSoifV19&Signature=swp5azcPl0FOe5CuStFZn1hmF0SHimPUOLwOqHd2ZAnoFJuYKDjhK7ESplRDWJdma9QWYOQyaCG23wkX18urieav%7E8OdxzwLaKLhL2YFx3L6RMwGEKWjrG-ql-LDfd2I1U4AcSXJZR5zHSBDmYql9M9hXKsvXHVkraIMS-cDx0ihj3s7yu4gbjUfE3SPg49aStq00ORcQnDV90mXxeheM6UjRymLRBdlxI3PCpAjzvyExcmZSgBU5vCnKtAEy5b65%7EzQoX5TVQTzQXjE9x8Qr2%7EAONSc7wy671HWYPRKNgZDrH3NJy90uFp38GKiQtab7hAy6fUlL358OQYhHzu4-Q__&Key-Pair-Id=K2FPYV99P2N66Q Resolving cdn-lfs-us-1.huggingface.co (cdn-lfs-us-1.huggingface.co)... 18.239.94.84, 18.239.94.6, 18.239.94.3, ... Connecting to cdn-lfs-us-1.huggingface.co (cdn-lfs-us-1.huggingface.co)|18.239.94.84|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 5768624435 (5.4G) [binary/octet-stream] Saving to: ‘Meta-Llama-3-8B-Instruct.Q5_K_M.llamafile’ Meta-Llama-3-8B-Ins 100%[===================>] 5.37G 57.2MB/s in 1m 40s 2024-06-20 09:55:11 (54.8 MB/s) - ‘Meta-Llama-3-8B-Instruct.Q5_K_M.llamafile’ saved [5768624435/5768624435]
Running the model - relevant parameters:
--server: start an OpenAI-compatible server--nobrowser: do not open the interactive interface in the browser--port: port of the OpenAI-compatible server (in Colab, 8080 is already taken)--n-gpu-layers: offload some layers to GPU for increased performance--ctx-size: size of the prompt context
nohup: redirecting stderr to stdout
Let's try to interact with the model.
Since the server is OpenAI-compatible, we can use an OpenAIChatGenerator.
{'replies': [ChatMessage(content="I'm just a language model, I don't have emotions or feelings like humans do. However, I'm functioning properly and ready to assist you with any questions or tasks you may have. How can I help you today?<|eot_id|>", role=<ChatRole.ASSISTANT: 'assistant'>, name=None, meta={'model': 'LLaMA_CPP', 'index': 0, 'finish_reason': 'stop', 'usage': {'completion_tokens': 46, 'prompt_tokens': 14, 'total_tokens': 60}})]} 🕵️ Mystery Character Quiz
Now that everything is in place, we can build a simple game in which a random character is selected from the dataset and the LLM is used to create hints for the player.
Hint generation pipeline
This simple pipeline includes a ChatPromptBuilder and a OpenAIChatGenerator.
Thanks to the template messages, we can include the character information in the prompt and also previous hints to avoid duplicate hints.
<haystack.core.pipeline.pipeline.Pipeline object at 0x7c0f4a07f580> ,🚅 Components , - chat_prompt_builder: ChatPromptBuilder , - generator: OpenAIChatGenerator ,🛤️ Connections , - chat_prompt_builder.prompt -> generator.messages (List[ChatMessage])
The game
🕵️ Guess the character based on the hints! ✨ Hint 1: Here's a brief hint: This actor has won an Academy Award for his role in a biographical sports drama film.<|eot_id|> Your guess: Press Q to quit Tom Cruise? Guess: Tom Cruise? ❌ Wrong guess. Try again. ✨ Hint 2: Here's a new hint: This actor is known for his intense physical transformations to portray his characters, including a significant weight gain and loss for one of his most iconic roles.<|eot_id|> Your guess: Press Q to quit Brendan Fraser Guess: Brendan Fraser ❌ Wrong guess. Try again. ✨ Hint 3: Here's a new hint: This actor has played a character who is a comic book superhero.<|eot_id|> Your guess: Press Q to quit Christian Bale Guess: Christian Bale 🎉 Congratulations! You guessed it right!
💬 🤠 Chat Adventures
Let's try something different now!
Character Codex is a large collection of characters, each with a specific description. Llama 3 8B Instruct is a good model, with some world knowledge.
We can try to combine them to simulate a dialogue and perhaps an adventure involving two different characters (fictional or real).
Character pipeline
Let's create a character pipeline: ChatPromptBuilder +OpenAIChatGenerator.
This represents the core of our conversational system and will be invoked multiple times with different messages to simulate conversation.
<haystack.core.pipeline.pipeline.Pipeline object at 0x78dd00ce69e0> ,🚅 Components , - chat_prompt_builder: ChatPromptBuilder , - generator: OpenAIChatGenerator ,🛤️ Connections , - chat_prompt_builder.prompt -> generator.messages (List[ChatMessage])
Messages
We define the most relevant messages to steer our LLM engine.
-
System message (template): this instructs the Language Model to chat and act as a specific character.
-
Start message: we need to choose an initial message (and a first speaking character) to spin up the conversation.
We also define the invert_roles utility function: for example, we want the first character to see the assistant messages from the second character as user messages, etc.
The game
It's time to choose two characters and play.
We choose the popular dancer Fred Astaire and Corporal Dwayne Hicks from the Alien saga.
✨ Looks like a nice result.
Of course, you can select other characters (even randomly) and change the initial message.
The implementation is pretty basic and could be improved in many ways.
📚 Resources
- Character Codex dataset
- llamafile
- llamafile-Haystack integration page: contains examples on how to run Generative and Embedding models and build indexing and RAG pipelines.
- Haystack components used in this notebook:
(Notebook by Stefano Fiorucci)