Utilizing Phi3 Models
openai-chatgptlangchain-pythonchatgptgenaielasticsearchelasticopenaiAIchatlogutilizing-phi3-modelsvectordatabasePythonsearchgenaistacksupporting-blog-contentvectorelasticsearch-labslangchainapplications
Export
Elastic's Recipe: Smarter Orders with Phi-3 small models
In this notebook we will learn how to deploy phi-3 models on Azure AI Studio and using them with Elastic Open Inference Service to create a RAG application. This notebook illustrates the article Elastic's Recipe: Smarter Orders with Phi-3 small models.
Install packages and import necessary modules
[ ]
Collecting elasticsearch==8.14
Downloading elasticsearch-8.14.0-py3-none-any.whl (480 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 480.2/480.2 kB 7.6 MB/s eta 0:00:00
Collecting elastic-transport<9,>=8.13 (from elasticsearch==8.14)
Downloading elastic_transport-8.13.1-py3-none-any.whl (64 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 64.5/64.5 kB 7.9 MB/s eta 0:00:00
Requirement already satisfied: urllib3<3,>=1.26.2 in /usr/local/lib/python3.10/dist-packages (from elastic-transport<9,>=8.13->elasticsearch==8.14) (2.0.7)
Requirement already satisfied: certifi in /usr/local/lib/python3.10/dist-packages (from elastic-transport<9,>=8.13->elasticsearch==8.14) (2024.7.4)
Installing collected packages: elastic-transport, elasticsearch
Successfully installed elastic-transport-8.13.1 elasticsearch-8.14.0
Declaring variables
This code will create inputs where you can enter your credentials.
Here you can learn how to retrieve your Elasticsearch credentials: Finding Your Cloud ID.
[ ]
Elastic Cloud ID: ·········· Elastic Api Key: ·········· Azure API Key: ·········· Azure target URL: ··········
Instance Elasticsearch client
[ ]
Creating embeddings endpoint
[ ]
Embedding endpoint created successfully.
Creating completion endpoint
[ ]
--------------------------------------------------------------------------- NameError Traceback (most recent call last) <ipython-input-92-363fb39672e8> in <cell line: 1>() 8 "service": "azureaistudio", 9 "service_settings": { ---> 10 "api_key": AZURE_API_KEY, 11 "target": AZURE_TARGET_URL, 12 "provider": "microsoft_phi", NameError: name 'AZURE_API_KEY' is not defined
Creating index
[ ]
Indexing data
[ ]
[ ]
Data indexed successfully.
Retrieving relevant dishes
We use a semantic query to retrieve the most relevant dishes based on the customer request.
[ ]
Response: [
{
"code": "carbonara",
"price": 14.99,
"description": {
"text": "Pasta Carbonara \n Perfectly al dente spaghetti enrobed in a velvety sauce of farm-fresh eggs, aged Pecorino Romano, and smoky guanciale. Finished with a kiss of cracked black pepper for a classic Roman indulgence.",
"inference": {
"inference_id": "elser-embeddings",
"model_settings": {
"task_type": "sparse_embedding"
},
"chunks": [
{
"text": "Pasta Carbonara \n Perfectly al dente spaghetti enrobed in a velvety sauce of farm-fresh eggs, aged Pecorino Romano, and smoky guanciale. Finished with a kiss of cracked black pepper for a classic Roman indulgence.",
"embeddings": {
"carbon": 2.0847998,
"pasta": 2.0838325,
"spaghetti": 1.9527067,
"##ara": 1.7632319,
"romano": 1.6877614,
"al": 1.6518246,
"dent": 1.5832088,
"roman": 1.5785156,
"sauce": 1.50039,
"eggs": 1.4797868,
"velvet": 1.4146874,
"##cial": 1.3529263,
"smoky": 1.3520736,
"##ro": 1.3192937,
"egg": 1.3167425,
"kiss": 1.3027223,
"cracked": 1.278348,
"pepper": 1.2271707,
"farm": 1.2113343,
"alfredo": 1.1915079,
"gu": 1.190911,
"noodles": 1.1808913,
"italian": 1.1601502,
"fresh": 1.1369088,
"##cor": 1.0747697,
"pe": 1.0703862,
"classic": 1.0423423,
"rome": 1.0119383,
"flavor": 0.9328391,
"peppers": 0.9042534,
"##aro": 0.8550794,
"en": 0.8329167,
"##an": 0.8324837,
"black": 0.8178875,
"smoke": 0.79119015,
"italy": 0.74483544,
"##ino": 0.72506446,
"tomato": 0.71198094,
"perfectly": 0.68112665,
"ingredients": 0.6761448,
"ravi": 0.6460934,
"finished": 0.6169553,
"recipe": 0.58733445,
"perfect": 0.57436633,
"aged": 0.5583758,
"taste": 0.52448153,
"shrimp": 0.5087635,
"cave": 0.50867414,
"blend": 0.50848633,
"wine": 0.50275457,
"romans": 0.49936667,
"spice": 0.49755874,
"squash": 0.4949674,
"dinner": 0.4923304,
"cheese": 0.4829378,
"flavour": 0.457689,
"##gence": 0.45691547,
"flat": 0.4553697,
"crack": 0.45347178,
"mario": 0.4526796,
"rice": 0.45201895,
"au": 0.4349133,
"traditional": 0.43312475,
"made": 0.4282039,
"bone": 0.42157665,
"cake": 0.4134516,
"chile": 0.3934753,
"pearl": 0.38652605,
"soup": 0.3570259,
"salmon": 0.3496939,
"ar": 0.34654406,
"varieties": 0.3456518,
"crushed": 0.33395255,
"spanish": 0.31545156,
"salad": 0.31005082,
"dessert": 0.30972683,
"style": 0.30488467,
"quality": 0.30363235,
"all": 0.30355245,
"dish": 0.29586068,
"tomatoes": 0.28619272,
"food": 0.28274676,
"type": 0.2670385,
"finish": 0.26572567,
"fiery": 0.26413622,
"marble": 0.26305264,
"stuffing": 0.26177517,
"meat": 0.25015828,
"foods": 0.24370497,
"cas": 0.24352334,
"sour": 0.23882519,
"delicious": 0.23850876,
"latin": 0.23436877,
"mafia": 0.22955851,
"##che": 0.22312142,
"tennessee": 0.22106534,
"dried": 0.21666816,
"co": 0.21655259,
"aluminum": 0.2090411,
"doubt": 0.19801831,
"pancakes": 0.19658157,
"thick": 0.19645111,
"favorite": 0.18269683,
"restaurant": 0.18210879,
"pure": 0.17537574,
"polish": 0.1739754,
"cooking": 0.17368133,
"fish": 0.1716123,
"pizza": 0.17020135,
"pit": 0.15634274,
"mix": 0.15507321,
"chef": 0.14697866,
"spicy": 0.14347146,
"shape": 0.1367214,
"old": 0.13251224,
"drink": 0.13050945,
"anti": 0.13011442,
"ripe": 0.12922636,
"garlic": 0.12443257,
"rich": 0.11953975,
"ara": 0.10267873,
"with": 0.10157385,
"##o": 0.09824008,
"lump": 0.09293696,
"sicilian": 0.08502127,
"##e": 0.083423816,
"substitute": 0.080164224,
"texture": 0.07978093,
"##chi": 0.07787835,
"rico": 0.077181816,
"castle": 0.060185388,
"c": 0.05781913,
"tara": 0.054664496,
"milo": 0.046940666,
"chicken": 0.037327815,
"sa": 0.03659888,
"##ante": 0.027901433,
"mass": 0.026733113,
"##ato": 0.026702356,
"milan": 0.024935864,
"greek": 0.02269474,
"combination": 0.020557316,
"elegant": 0.014001829,
"bread": 0.012648127,
"argentina": 0.011897084,
"flavors": 0.010257504,
"tuna": 0.009524028,
"roma": 0.0055943117,
"__": 0.002049847,
"a": 0.00012992969
}
}
]
}
},
"title": "Pasta Carbonara",
"customizations": {
"vegetarian": [
true,
false
],
"extras": [
"cheese",
"garlic",
"ham"
],
"cream": [
true,
false
]
}
},
{
"code": "alfredo",
"price": 18.99,
"description": {
"text": "Chicken Alfredo \n Recipe includes golden pan-fried seasoned chicken breasts and tender fettuccine, coated in the most dreamy cream sauce ever, coated with a velvety garlic and Parmesan cream sauce.",
"inference": {
"inference_id": "elser-embeddings",
"model_settings": {
"task_type": "sparse_embedding"
},
"chunks": [
{
"text": "Chicken Alfredo \n Recipe includes golden pan-fried seasoned chicken breasts and tender fettuccine, coated in the most dreamy cream sauce ever, coated with a velvety garlic and Parmesan cream sauce.",
"embeddings": {
"alfredo": 2.564587,
"chicken": 1.8346423,
"pasta": 1.8238221,
"##ucci": 1.6461563,
"sauce": 1.6025357,
"recipe": 1.4776391,
"fried": 1.4650584,
"breasts": 1.4406543,
"cream": 1.4403517,
"golden": 1.3249015,
"##tt": 1.3037602,
"pan": 1.298015,
"garlic": 1.2871181,
"seasoned": 1.196314,
"italian": 1.1623968,
"velvet": 1.1578108,
"dream": 1.1535012,
"breast": 1.1374456,
"chickens": 1.1048578,
"flavor": 1.0701971,
"fry": 0.9859959,
"tender": 0.9708918,
"##mes": 0.9217419,
"recipes": 0.8933892,
"most": 0.7819251,
"##an": 0.7673652,
"ingredients": 0.74734217,
"fe": 0.7179425,
"stuffing": 0.7053486,
"coating": 0.66489476,
"ever": 0.6408478,
"soup": 0.6342612,
"restaurant": 0.59911627,
"shrimp": 0.57749015,
"cooked": 0.56963134,
"onion": 0.5450308,
"spice": 0.533965,
"gold": 0.5305381,
"##ula": 0.52431905,
"coated": 0.52246696,
"romano": 0.51953495,
"dinner": 0.5152075,
"cooking": 0.50969106,
"flavour": 0.50816065,
"par": 0.4996068,
"noodles": 0.49789834,
"kitchen": 0.4947274,
"best": 0.4874154,
"##scan": 0.478875,
"wings": 0.4663816,
"rice": 0.46338895,
"coat": 0.46131432,
"dish": 0.4605129,
"cake": 0.45159337,
"wing": 0.4267126,
"campbell": 0.42100388,
"##nta": 0.41588247,
"type": 0.41289526,
"combination": 0.41191906,
"mixing": 0.40398732,
"food": 0.4006494,
"julia": 0.3836181,
"delicate": 0.38125896,
"italy": 0.38078514,
"taste": 0.38036105,
"pisa": 0.3761185,
"don": 0.37580723,
"pancakes": 0.36829042,
"foods": 0.36580428,
"##y": 0.34617537,
"chef": 0.34488603,
"fra": 0.34359822,
"stuffed": 0.34257597,
"spanish": 0.3421706,
"crisp": 0.34053022,
"blend": 0.3373403,
"cook": 0.31444147,
"substitute": 0.31359094,
"technique": 0.3106372,
"surrounded": 0.2964942,
"pepper": 0.2922128,
"thick": 0.27548715,
"served": 0.25957543,
"formula": 0.25705764,
"texture": 0.2544215,
"fries": 0.24249437,
"include": 0.23216493,
"style": 0.22883052,
"dessert": 0.22751497,
"thighs": 0.22610186,
"wine": 0.22608131,
"sweet": 0.22575645,
"cheese": 0.2197249,
"delicious": 0.21336219,
"soy": 0.21156286,
"##uo": 0.21134749,
"chili": 0.20474188,
"pope": 0.20450474,
"rich": 0.20416267,
"made": 0.20068921,
"with": 0.19718379,
"ling": 0.19635552,
"spices": 0.18753126,
"stir": 0.18500854,
"minute": 0.17656763,
"anna": 0.1724526,
"milk": 0.16676955,
"creamy": 0.1629867,
"battered": 0.15504432,
"fish": 0.1532438,
"favorite": 0.15209308,
"jose": 0.14888714,
"meat": 0.14689654,
"mix": 0.14603789,
"spicy": 0.13331048,
"skill": 0.13055632,
"french": 0.12478566,
"wonderful": 0.11361911,
"ross": 0.1104974,
"elise": 0.11042087,
"basil": 0.104210526,
"mexican": 0.09911397,
"pizza": 0.09332448,
"fill": 0.09031643,
"##mbo": 0.088501684,
"package": 0.08747048,
"includes": 0.08606179,
"##ko": 0.08545729,
"f": 0.08135256,
"fond": 0.08125717,
"features": 0.06959614,
"##o": 0.06385035,
"shell": 0.062879145,
"dreams": 0.060527902,
"bones": 0.058741644,
"##ull": 0.057891022,
"ari": 0.054450814,
"cod": 0.052321326,
"contain": 0.051052306,
"egg": 0.049737327,
"##ne": 0.044573493,
"##fu": 0.043771666,
"che": 0.0429009,
"prepared": 0.040078502,
"mayo": 0.037097875,
"is": 0.035554122,
"perfect": 0.034818944,
"sherry": 0.03144467,
"serve": 0.02636094,
"meal": 0.0249497,
"gu": 0.014452303,
"tu": 0.013452006,
"and": 0.013005081,
"ribs": 0.012063167,
"##cci": 0.011064485,
"add": 0.010891278,
"pat": 0.0093868105,
"ingredient": 0.007957779,
"crunch": 0.0046977154,
"butter": 0.004236889
}
}
]
}
},
"title": "Chicken Alfredo",
"customizations": {
"vegetarian": [
true,
false
],
"extras": [
"cheese",
"onions",
"olives"
],
"cream": [
true,
false
]
}
},
{
"code": "gnocchi",
"price": 15.99,
"description": {
"text": "Four Cheese Gnocchi \n soft pillowy potato gnocchi coated in a silken cheesy sauce made of four different cheeses: Gouda, Parmigiano, Brie, and the star, Gorgonzola. The combination of four different types of cheese will make your tastebuds dance for joy.",
"inference": {
"inference_id": "elser-embeddings",
"model_settings": {
"task_type": "sparse_embedding"
},
"chunks": [
{
"text": "Four Cheese Gnocchi \n soft pillowy potato gnocchi coated in a silken cheesy sauce made of four different cheeses: Gouda, Parmigiano, Brie, and the star, Gorgonzola. The combination of four different types of cheese will make your tastebuds dance for joy.",
"embeddings": {
"cheese": 1.8433701,
"##cchi": 1.8249218,
"pillow": 1.7976478,
"##uda": 1.7632233,
"##no": 1.7160593,
"sauce": 1.6652024,
"potato": 1.6410445,
"##rgo": 1.5110843,
"four": 1.4945054,
"star": 1.4881107,
"pillows": 1.4633442,
"g": 1.4311051,
"potatoes": 1.4266229,
"silk": 1.3149087,
"che": 1.2663606,
"##gia": 1.2491798,
"##nzo": 1.2481608,
"joy": 1.2380968,
"##ie": 1.2244678,
"soft": 1.2166287,
"br": 1.1827679,
"taste": 1.1643262,
"##mi": 1.1231838,
"dance": 1.0761534,
"##la": 1.0627452,
"4": 1.0579926,
"italian": 1.0441946,
"go": 1.0375262,
"stars": 1.0235771,
"##es": 1.0135604,
"combination": 0.97541225,
"made": 0.9574184,
"pasta": 0.92463857,
"coat": 0.8985466,
"coated": 0.8865534,
"types": 0.8669388,
"varieties": 0.82150537,
"flavor": 0.79355574,
"stuffed": 0.78604937,
"par": 0.7816326,
"coating": 0.76706445,
"squash": 0.7433965,
"romano": 0.7269263,
"cake": 0.6953739,
"vegetables": 0.69372135,
"italy": 0.65606326,
"type": 0.6555585,
"ingredients": 0.6509747,
"mario": 0.61885077,
"stuffing": 0.5789762,
"texture": 0.5746486,
"##tti": 0.55706394,
"vegetable": 0.552092,
"##na": 0.53830713,
"russ": 0.53325486,
"tuck": 0.52827513,
"dancing": 0.5244148,
"shape": 0.5233079,
"##cco": 0.50105345,
"foods": 0.4941753,
"brussels": 0.488823,
"flavour": 0.48540276,
"##gg": 0.48222354,
"food": 0.44196266,
"rice": 0.4353299,
"basket": 0.4289376,
"bacon": 0.42317447,
"recipe": 0.42105818,
"bell": 0.4208563,
"fuzzy": 0.41236413,
"fourth": 0.41061163,
"##ute": 0.40760192,
"surrounded": 0.39993945,
"brands": 0.39929348,
"soup": 0.393992,
"##ce": 0.3913286,
"different": 0.38472915,
"knife": 0.3737238,
"surprise": 0.35915875,
"##i": 0.35583097,
"pisa": 0.34751344,
"lump": 0.3125717,
"alfredo": 0.31175232,
"noodles": 0.30807647,
"no": 0.2884717,
"##so": 0.2869589,
"milk": 0.285988,
"genus": 0.28054005,
"foam": 0.27264646,
"brand": 0.27150664,
"blend": 0.27041394,
"velvet": 0.26908088,
"gu": 0.26885387,
"dish": 0.2673401,
"fish": 0.26622063,
"went": 0.2660476,
"colors": 0.2594121,
"carrot": 0.24424857,
"mascot": 0.24353577,
"##ura": 0.23890406,
"mushroom": 0.2349561,
"del": 0.22825418,
"covered": 0.22789176,
"plum": 0.21863972,
"make": 0.21811977,
"layer": 0.19855213,
"##s": 0.19764976,
"direction": 0.19220816,
"variants": 0.1894919,
"product": 0.18020976,
"happy": 0.1766042,
"slice": 0.17476365,
"##gur": 0.17255041,
"dessert": 0.17195415,
"dinner": 0.16261624,
"composed": 0.16121463,
"these": 0.156481,
"__": 0.1550779,
"##bu": 0.15405245,
"combine": 0.14674234,
"flavors": 0.14650504,
"sweet": 0.14420816,
"mixing": 0.14227036,
"waltz": 0.1328341,
"layered": 0.13259411,
"##ata": 0.13255036,
"pastry": 0.13126214,
"##ese": 0.12905255,
"##oli": 0.12838961,
"bread": 0.12678272,
"shapes": 0.12635648,
"bed": 0.124999665,
"combinations": 0.12327213,
"##zone": 0.121784054,
"mixture": 0.12173065,
"cushion": 0.120557345,
"parma": 0.11741559,
"dream": 0.11702808,
"easter": 0.116614416,
"layers": 0.113192976,
"milan": 0.11088159,
"couch": 0.10817721,
"stir": 0.10738166,
"##hi": 0.10468957,
"came": 0.099799715,
"##y": 0.09693489,
"hill": 0.09482902,
"bubble": 0.094252035,
"herb": 0.093428716,
"bag": 0.090363264,
"cover": 0.0852456,
"dip": 0.08419295,
"fruit": 0.07581491,
"hawk": 0.074430756,
"with": 0.07412828,
"size": 0.0740002,
"famous": 0.07191443,
"thick": 0.071299866,
"nap": 0.070573956,
"riga": 0.06961849,
"fluffy": 0.06866412,
"pile": 0.06759319,
"slip": 0.055408016,
"onion": 0.051355843,
"rocco": 0.05095171,
"##zo": 0.050170857,
"della": 0.049794152,
"##zza": 0.046903927,
"graz": 0.046479788,
"crisp": 0.042849854,
"gr": 0.04122299,
"fried": 0.04042843,
"rome": 0.03976339,
"##rella": 0.039447036,
"garlic": 0.03556574,
"softness": 0.033182874,
"come": 0.032993965,
"##uti": 0.026017783,
"pancakes": 0.025654988,
"cot": 0.022374453,
"##ach": 0.022212872,
"dressing": 0.020076636,
"shrimp": 0.017028434,
"illusion": 0.016683934,
"rattle": 0.01558036,
"##en": 0.015386687,
"monster": 0.014637346,
"firm": 0.013829836,
"##rino": 0.01207577,
"toro": 0.011723079,
"bud": 0.011340208,
"cute": 0.0075798524,
"stuff": 0.003842591
}
}
]
}
},
"title": "Four Cheese Gnocchi",
"customizations": {
"vegetarian": [
true,
false
],
"extras": [
"cheese",
"bacon",
"mushrooms"
],
"cream": [
true,
false
]
}
}
]
Putting everything together
With this script we can ask the user to order, and keep the status of the order updated.
[ ]
What would you like to order? pasta with cheese
Result:
{
"order": [
{
"code": "carbonara",
"qty": 1,
"customizations": [
{
"extras": [
"cheese"
]
}
]
}
]
}
The current order status is:
{
"order": [
{
"code": "carbonara",
"qty": 1,
"customizations": [
{
"extras": [
"cheese"
]
}
]
}
]
}
What would you like to order? alfredo
Result:
{
"order": [
{
"code": "carbonara",
"qty": 1,
"customizations": [
{
"extras": [
"cheese"
]
}
]
},
{
"code": "alfredo",
"qty": 1,
"customizations": []
}
]
}
The current order status is:
{
"order": [
{
"code": "carbonara",
"qty": 1,
"customizations": [
{
"extras": [
"cheese"
]
}
]
},
{
"code": "alfredo",
"qty": 1,
"customizations": []
}
]
}
What would you like to order? another carbonara with extra cheese
Result:
{
"order": [
{
"code": "carbonara",
"qty": 1,
"customizations": [
{
"extras": [
"cheese"
]
}
]
},
{
"code": "alfredo",
"qty": 1,
"customizations": []
},
{
"code": "carbonara",
"qty": 1,
"customizations": [
{
"extras": [
"cheese"
]
}
]
}
]
}
The current order status is:
{
"order": [
{
"code": "carbonara",
"qty": 1,
"customizations": [
{
"extras": [
"cheese"
]
}
]
},
{
"code": "alfredo",
"qty": 1,
"customizations": []
},
{
"code": "carbonara",
"qty": 1,
"customizations": [
{
"extras": [
"cheese"
]
}
]
}
]
}
--------------------------------------------------------------------------- KeyboardInterrupt Traceback (most recent call last) <ipython-input-12-5bfa92de7a4e> in <cell line: 3>() 2 3 while True: ----> 4 query = input("What would you like to order? ") 5 6 try: /usr/local/lib/python3.10/dist-packages/ipykernel/kernelbase.py in raw_input(self, prompt) 849 "raw_input was called, but this frontend does not support input requests." 850 ) --> 851 return self._input_request(str(prompt), 852 self._parent_ident, 853 self._parent_header, /usr/local/lib/python3.10/dist-packages/ipykernel/kernelbase.py in _input_request(self, prompt, ident, parent, password) 893 except KeyboardInterrupt: 894 # re-raise KeyboardInterrupt, to truncate traceback --> 895 raise KeyboardInterrupt("Interrupted by user") from None 896 except Exception as e: 897 self.log.warning("Invalid Message:", exc_info=True) KeyboardInterrupt: Interrupted by user
Cleanup
Finally, we can delete the resources used to prevent them from consuming resources.
[ ]
<ipython-input-101-ed0931a48b5f>:2: DeprecationWarning: Passing transport options in the API method is deprecated. Use 'Elasticsearch.options()' instead. es_client.indices.delete(index="lasticco-menu", ignore=[400, 404]) <ipython-input-101-ed0931a48b5f>:5: DeprecationWarning: Passing transport options in the API method is deprecated. Use 'Elasticsearch.options()' instead. es_client.inference.delete_model(inference_id="phi3-completion", ignore=[400, 404]) <ipython-input-101-ed0931a48b5f>:8: DeprecationWarning: Passing transport options in the API method is deprecated. Use 'Elasticsearch.options()' instead. es_client.inference.delete_model(inference_id="elser-embeddings", ignore=[400, 404])
ObjectApiResponse({'acknowledged': True, 'pipelines': [], 'indexes': []})