Langfuse Sdk Performance Test
title: Langfuse SDK Performance Test category: SDKs
Langfuse SDK Performance Test
Langfuse shall have a minimal impact on latency. This is achieved by running almost entirely in the background and by batching all requests to the Langfuse API.
Coverage of this performance test:
- Langfuse SDK: trace(), generation(), span()
- Langchain Integration
- OpenAI Integration
- LlamaIndex Integration
Limitation: We test integrations using OpenAI's hosted models, making the experiment less controlled but actual latency of the integrations impact more realistic.
Setup
Note: This guide uses our Python SDK v2. We have a new, improved SDK available based on OpenTelemetry. Please check out the SDK v3 for a more powerful and simpler to use SDK.
Python SDK
trace()
count 100.000000 ,mean (sec) 0.000266 ,std (sec) 0.000381 ,min (sec) 0.000154 ,25% (sec) 0.000191 ,50% (sec) 0.000197 ,75% (sec) 0.000211 ,max (sec) 0.003784 ,dtype: float64
span()
count 100.000000 ,mean (sec) 0.000162 ,std (sec) 0.000199 ,min (sec) 0.000096 ,25% (sec) 0.000099 ,50% (sec) 0.000106 ,75% (sec) 0.000130 ,max (sec) 0.001635 ,dtype: float64
generation()
count 100.000000 ,mean (sec) 0.000196 ,std (sec) 0.000165 ,min (sec) 0.000132 ,25% (sec) 0.000137 ,50% (sec) 0.000148 ,75% (sec) 0.000173 ,max (sec) 0.001238 ,dtype: float64
event()
count 100.000000 ,mean (sec) 0.000236 ,std (sec) 0.000300 ,min (sec) 0.000152 ,25% (sec) 0.000177 ,50% (sec) 0.000189 ,75% (sec) 0.000219 ,max (sec) 0.003144 ,dtype: float64
Langchain Integration
Docs: https://langfuse.com/integrations/frameworks/langchain
Bechmark without Langfuse
count 100.000000 ,mean (sec) 0.529463 ,std (sec) 0.685193 ,min (sec) 0.306092 ,25% (sec) 0.373373 ,50% (sec) 0.407278 ,75% (sec) 0.530427 ,max (sec) 7.107237 ,dtype: float64
With Langfuse Tracing
count 100.000000 ,mean (sec) 0.618286 ,std (sec) 0.165149 ,min (sec) 0.464992 ,25% (sec) 0.518323 ,50% (sec) 0.598474 ,75% (sec) 0.675420 ,max (sec) 1.838614 ,dtype: float64
OpenAI Integration
Docs: https://langfuse.com/integrations/model-providers/openai-py
Benchmark without Langfuse
count 100.000000 ,mean (sec) 0.524097 ,std (sec) 0.220446 ,min (sec) 0.288002 ,25% (sec) 0.395479 ,50% (sec) 0.507395 ,75% (sec) 0.571789 ,max (sec) 1.789671 ,dtype: float64
With Langfuse Tracing
count 100.000000 ,mean (sec) 0.515243 ,std (sec) 0.286902 ,min (sec) 0.283431 ,25% (sec) 0.378736 ,50% (sec) 0.435775 ,75% (sec) 0.558746 ,max (sec) 2.613779 ,dtype: float64
LlamaIndex Integration
Docs: https://langfuse.com/integrations/frameworks/llamaindex
Sample documents
Bechmark without Langfuse
Index
count 100.000000 ,mean (sec) 0.171673 ,std (sec) 0.058332 ,min (sec) 0.112696 ,25% (sec) 0.136361 ,50% (sec) 0.157330 ,75% (sec) 0.178455 ,max (sec) 0.459417 ,dtype: float64
Query
count 100.000000 ,mean (sec) 0.795817 ,std (sec) 0.338263 ,min (sec) 0.445060 ,25% (sec) 0.614282 ,50% (sec) 0.756573 ,75% (sec) 0.908411 ,max (sec) 3.495263 ,dtype: float64
With Langfuse Tracing
Index
count 100.000000 ,mean (sec) 0.178796 ,std (sec) 0.101976 ,min (sec) 0.112530 ,25% (sec) 0.138217 ,50% (sec) 0.163698 ,75% (sec) 0.179563 ,max (sec) 0.992403 ,dtype: float64
Query
count 100.000000 ,mean (sec) 0.802315 ,std (sec) 0.230386 ,min (sec) 0.423413 ,25% (sec) 0.639373 ,50% (sec) 0.784945 ,75% (sec) 0.945300 ,max (sec) 2.164593 ,dtype: float64