Modaic Weaviate
vector-searchmodaicvector-databaseretrieval-augmented-generationllm-frameworksfunction-callingweaviate-recipesintegrationsPythongenerative-aillm-agent-frameworks
Export
Modaic and Weaviate
Modaic is a hub to share and manage DSPy programs!
You can learn more about the modaic sdk here and check out programs on the hub here.
This notebook will illustrate how to load the CrossEncoderRanker program from the Modaic Hub, as well as PromptToSignature.
Further, check out ce_ranker.py and pyproject.toml to see how to publish your programs on the Modaic Hub!
CrossEncoderRanker
[ ]
SSSS connor/CrossEncoderRanker
Rank 1: Figure 1: An illustration of the HyDE model. Document snippets are shown. HyDE serves all types of queries without changing the underlying InstructGPT and Contriever/mContriever models.
GPT-3 (Brown et al., 2020) models can be aligned to human intents to follow instructions faithfully.
With these
Rank 2: | | Scifact | FiQA | DBPedia |
|-----------|---------|-------|---------|
| Contriever | 64.9 | 24.5 | 29.2 |
| HyDE w/ InstructGPT | **69.1** | 27.3 | 36.8 |
| w/ GPT-3 | 65.9 | **27.9** | **40.5** |
Table 5: nDCG@10 comparing InstructGPT vs. 3-shot GPT-3 on BEIR. Bes
Rank 3: 6 Conclusion
In this paper, we introduce HyDE, a new approach for building effective dense retrievers in a completely unsupervised manner, without the need for any relevance labels. We demonstrate that some aspects of relevance modeling can be delegated to a more powerful, flexible, and general-pur
Rank 4: estimate Equation 5 by sampling N documents from g, [d̂1, d̂2, ..., d̂N]:
v̂qij = 1/N ∑_(d̂k ∼ g(qij, INSTi)) f(d̂k) (6)
= 1/N ∑_(k=1)^N f(d̂k) (7)
We also consider the query as a possible hypothesis:
v̂qij = 1/(N+1) [(∑_(k=1)^N f(d̂k)) + f(qij)] (8)
Inner p
Rank 5: | | sw | wo | ko | ja | bn |
|-----------|-----|-----|------|------|------|
| **Unsupervised** | | | | | |
| BM25 | 38.9| 28.5| 21.2 | 41.8 | |
| mContriever| 38.3| 22.3| 19.5 | 35.3 | |
| HyDE | 41.7| 30.6| 30.7 | 41.3 | |
|
PromptToSignature
[ ]
SSSS fadeleke/prompt-to-signature
Signature generation took 0.00 seconds in inference.
Prediction(
signature_name='DocumentSummaryAndEntityExtraction',
task_description='Extract a concise summary of a document and identify key entities (people, organizations, locations, dates, etc.) mentioned within it.',
signature_fields=[GeneratedField(name='document', type=<FieldType.STRING: 'str'>, role=<FieldRole.INPUT: 'input'>, description='The document text to summarize and extract entities from', pydantic_model_schema=None, literal_values=None, default_value=None), GeneratedField(name='summary', type=<FieldType.STRING: 'str'>, role=<FieldRole.OUTPUT: 'output'>, description="A concise summary of the document's main points and content", pydantic_model_schema=None, literal_values=None, default_value=None), GeneratedField(name='key_entities', type=<FieldType.PYDANTIC_MODEL: 'pydantic'>, role=<FieldRole.OUTPUT: 'output'>, description='Structured extraction of key entities found in the document', pydantic_model_schema=PydanticModelSchema(model_name='KeyEntities', description='Container for extracted entities from the document', fields=[PydanticFieldDef(name='people', type=<FieldType.LIST_STRING: 'list[str]'>, description='Names of people mentioned in the document', required=False, literal_values=None, nested_model=None), PydanticFieldDef(name='organizations', type=<FieldType.LIST_STRING: 'list[str]'>, description='Names of organizations, companies, or institutions mentioned', required=False, literal_values=None, nested_model=None), PydanticFieldDef(name='locations', type=<FieldType.LIST_STRING: 'list[str]'>, description='Geographic locations, cities, countries, or regions mentioned', required=False, literal_values=None, nested_model=None), PydanticFieldDef(name='dates', type=<FieldType.LIST_STRING: 'list[str]'>, description='Important dates or time references mentioned in the document', required=False, literal_values=None, nested_model=None), PydanticFieldDef(name='topics', type=<FieldType.LIST_STRING: 'list[str]'>, description='Main topics or themes covered in the document', required=False, literal_values=None, nested_model=None)]), literal_values=None, default_value=None)],
reasoning=None
)