Voyageai Embeddings Eval
agentsartificial-intelligencellmsmongodb-genai-showcaseevalsnotebooksgenerative-airag
Export
How to choose the right embedding model for your RAG application
This notebook evaluates the voyage-3-large model.
Step 1: Install required libraries
- datasets: Python library to get access to datasets available on Hugging Face Hub
- voyageai: Python library to interact with Voyage AI's APIs
- numpy: Python library that provides tools to perform mathematical operations on arrays
- pandas: Python library for data analysis, exploration and manipulation
- tdqm: Python module to show a progress bar for loops
[27]
Step 2: Setup pre-requisites
Set the Voyage API key as an environment variable, and initialize the Voyage AI client.
Steps to obtain a Voyage AI API Key can be found here.
[28]
[29]
Voyage API Key: ········
Step 3: Download the evaluation dataset
We will use MongoDB's cosmopedia-wikihow-chunked dataset, which has chunked versions of WikiHow articles from the Cosmopedia dataset released by Hugging Face. The dataset is pretty large, so we will only grab the first 2k records for testing.
[30]
Step 4: Data analysis
Make sure the length of the dataset is what we expect (2k), preview the data, drop Nones etc.
[31]
2000
[32]
[33]
[34]
352
Step 5: Creating embeddings
Define the embedding function, and run a quick test.
[35]
[36]
[37]
[38]
1024
Step 6: Evaluation
Measuring embedding latency
Create a local vector store (list) of embeddings for the entire dataset.
[39]
[40]
[41]
0%| | 0/2000 [00:00<?, ?it/s]
Measuring retrieval quality
- Create embedding for the user query
- Get the top 5 most similar documents from the local vector store using cosine similarity as the similarity metric
[18]
[19]
[20]
[21]
Query: Give me some tips to improve my mental health. Score: 0.7262 Step 2: Reach Out to Someone Trustworthy Connect with someone who cares about you—a friend, family member, mental health professional, or support group. Talking openly about your struggles can lighten your emotional burden, making it easier to manage. Sharing your feelings may also lead to helpful suggestions and advice from others. Guideline: Make sure to choose someone trustworthy who has shown empathy towards you before. If possible, reach out to more than one person to build a strong support network around you. Step 3: Engage in Physical Activity Exercise releases endorphins, which improve mood and reduce stress levels. Go for a walk, jog, bike ride, swim, or engage in any physical activity that suits your abilities. Exercising regularly can significantly impact overall wellbeing by reducing symptoms of depression and anxiety. Key Tip: Start small - aim for just five minutes of exercise if needed, then gradually increase duration over time. The most important thing is to get started! -------- Score: 0.7074 Key Tips & Guidelines: * Confide in people who genuinely care about your wellbeing * Attend meetings regularly to reinforce positive behaviors Step 8: Practice Mindfulness Incorporate mindfulness techniques into your daily routine through meditation, deep breathing exercises, or progressive muscle relaxation. These practices enhance self-awareness, emotional regulation, and stress management skills, promoting healthier coping mechanisms. Key Tips & Guidelines: * Dedicate at least 10 minutes per day to mindfulness activities * Explore different methods to find what works best for you Step 9: Prioritize Self-Care Attend to physical, emotional, and psychological needs by maintaining a nutritious diet, getting sufficient sleep, and exercising regularly. Adequate self-care strengthens resilience against relapse and improves overall quality of life. Key Tips & Guidelines: * Develop a consistent self-care plan tailored to individual requirements * Consult healthcare professionals for advice if necessary -------- Score: 0.7043 ### Step 5: Stay Active **Explanation:** Regular exercise is essential for maintaining both physical and mental health. Exercise releases endorphins that boost energy levels, reduce anxiety, and promote better sleep. * **Key Tips:** + Choose exercises that suit your fitness level and preferences, such as yoga, Pilates, dance classes, or bodyweight training routines. + Allocate time for regular walks outside if possible, taking advantage of parks or scenic routes nearby. + Utilize free resources available online, such as workout videos or apps, to guide your routine. ### Step 6: Maintain Social Connections **Explanation:** Strong social relationships contribute significantly to emotional well-being. Nurturing connections with friends and family members prevents feelings of isolation and loneliness. --------
[22]
Query: Give me some tips for writing good code. Score: 0.6609 Step 6: Improve Code Quality Strive for clean, readable, maintainable code. Adopt consistent naming conventions, indentation styles, and formatting rules. Utilize version control systems like Git to track changes and collaborate effectively. Leverage linters and static analyzers to enforce style guides automatically. Document your work using comments and dedicated documentation tools. High-quality code facilitates collaboration, promotes longevity, and simplifies troubleshooting. Step 7: Embrace Best Practices Follow established best practices relevant to your chosen language and domain. Examples include Object-Oriented Design Principles, SOLID principles, Test-Driven Development (TDD), Dependency Injection, Asynchronous Programming, etc. While seemingly overwhelming initially, integrating them gradually enhances design patterns, scalability, and extensibility. Consult authoritative blogs, books, and articles to stay updated on current trends and recommendations. -------- Score: 0.5749 Step 4: Work On Real Projects Theory alone won't make you a proficient programmer; hands-on experience does. Apply what you've learned by working on real projects. Begin with small tasks like creating calculators, text-based games, or automating repetitive tasks. Gradually move towards larger projects related to your field of interest. Participate in open-source initiatives, contribute to existing repositories, or develop your application ideas. These experiences enhance problem-solving skills and prepare you for professional challenges. Step 5: Master Debugging Techniques Debugging is inevitable when writing code. Identifying and resolving errors early improves productivity and reduces frustration. Use debuggers provided within IDEs, print statements, or log messages to trace execution flow and pinpoint issues. Familiarize yourself with common error types and causes, especially those specific to your chosen language. Over time, proactive debugging becomes second nature. -------- Score: 0.5745 Conclusion: Becoming a good programmer requires dedication, persistence, and patience. By methodically progressing through these steps, mastering core concepts, practicing diligently, engaging with peers, and committing to continuous improvement, you'll be well on your way to achieving your goal. Remember, every expert was once a beginner - keep pushing forward! --------
[23]
Query: How to create a basic webpage? Score: 0.5843 Title: How to Get a Free Website with Way2Webscape.com Introduction: Way2Webscape.com offers free websites for users who want to create their own online presence without any cost. This tutorial will guide you through the process of getting a free website using this service. By following these steps, you'll be able to create an attractive and functional website that meets your needs. Step 1: Visit Way2Webscape.com Open your web browser and navigate to <https://www.way2webscape.com/>. This will take you to the homepage of Way2Webscape.com. Key tip: Make sure you have a stable internet connection while completing the following steps. Step 2: Click on "Create Your Free Website" Button On the top right corner of the homepage, click on the green button labeled "Create Your Free Website." This action will redirect you to the registration page. -------- Score: 0.5713 Step 6: Choose a Template After logging in, select a template that suits your preferences by browsing through various categories such as business, personal, blog, etc., located on the left sidebar under "Template Categories". Once you find a suitable design, hover over it and click on the green "Use this template" button below the preview image. Step 7: Customize Your Site You can customize different aspects of your site like its layout, color scheme, background image, font styles, and more via the editor dashboard on the left panel. Remember to save changes made before navigating away from any editing screen. Step 8: Add Pages To add pages to your website, go to the "Pages" tab on the editor dashboard. Here, choose between predefined page types (e.g., Home, About Us, Services) or create custom ones according to your requirements. Don't forget to assign appropriate titles and URL slugs to these pages. -------- Score: 0.5694 Step 9: Edit Page Content For each added page, utilize the content editor to input text, images, videos, links, tables, dividers, or other elements needed. Format texts using headings, bullet points, numbered lists, indentation, alignment, and colors as necessary. Step 10: Configure Settings Navigate to the "Settings" tab on the editor dashboard. Fill in essential details about your site including name, description, keywords, contact info, social media profiles, analytics tracking codes, and SEO settings. Save changes once completed. Step 11: Preview Your Site Before publishing your site, preview it first by clicking on the eye icon near the top right corner of the editor interface. Review your entire site carefully, checking for errors or inconsistencies. If satisfied, proceed to the next step; otherwise, make adjustments accordingly. --------
[24]
Query: What are some environment-friendly practices I can incorporate in everyday life? Score: 0.6947 a) Bring reusable bags, containers, utensils, and water bottles when shopping or dining out. b) Purchase items with minimal packaging, prioritize products made from recycled materials, and avoid single-use plastics. c) Participate in composting programs or start composting at home to turn organic waste into nutrient-rich soil amendments. d) Donate unwanted items rather than discarding them, and repurpose household goods when possible. e) Learn about local recycling rules and sort waste accordingly. Key Tip: Familiarize yourself with your city's waste management system to optimize recycling efforts and divert waste from landfills. Step 4: Protect Biodiversity Biodiversity underpins the health and resilience of ecosystems, providing vital services such as pollination, decomposition, pest control, and nutrient cycling. Safeguard biodiversity through the following actions: -------- Score: 0.6556 a) Use public transportation, carpool, bike, walk, or telecommute whenever possible to minimize fuel consumption. b) If purchasing a vehicle, consider electric or hybrid options that emit fewer greenhouse gases than traditional gasoline-powered cars. c) Improve home energy efficiency through insulation, LED lighting, Energy Star appliances, and renewable energy sources like solar panels. d) Limit air travel and opt for video conferencing when feasible. e) Be mindful of your dietary choices – consume less red meat, choose locally sourced foods, and reduce food waste. f) Plant trees and support reforestation projects as they absorb CO2 from the atmosphere. g) Advocate for policies that promote clean energy and reduced emissions at local, national, and international levels. Key Tip: Calculate your carbon footprint using online tools (such as the EPA's Household Carbon Footprint Calculator) to identify areas where you can make improvements. -------- Score: 0.6265 a) Host workshops, film screenings, or community events focused on sustainability topics. b) Engage with elected officials and voice concerns regarding environmental issues. c) Collaborate with schools, libraries, and nonprofit organizations to develop educational programming related to conservation and sustainability. d) Leverage social media platforms to disseminate information and inspire action among friends, family, and followers. Key Tip: Encouraging just one person to adopt more eco-friendly habits can create a ripple effect leading to broader positive change. Conclusion: Saving our environment requires collective effort and commitment from individuals, businesses, and governments worldwide. While addressing global challenges like climate change may seem daunting, taking concrete steps toward sustainability at the personal level contributes meaningfully to larger-scale progress. Together, we can foster a greener, healthier, and more equitable world for all living beings. --------