★ FEATURED ARTICLE 人工智能LLMOps模型评测可观测性AI AgentAI 应用后端前端【免费下载链接】comet-llmDebug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.项目地址https://gitcode.com/GitHub_Trending/co/comet-llm点击查看免费下载本指南以 Opik 官方文档 Log traces 为核心系统讲解如何在 Opik 中为 LLM 应用、RAG 系统与 Agent 工作流接入全链路追踪。读完本文你将掌握 SDK 安装与配置、主流框架集成、函数装饰器与底层 SDK 两种日志方式、上下文管理器、项目与环境路由以及 trace/span 的冲刷flush与全局开关控制等实战能力。LLM 应用远比一次 API 调用复杂检索、预处理、后处理、多轮工具调用交织在一起。Tracing追踪的价值在于还原每一次请求的完整执行流让你能在海量调用中快速定位是哪个环节出了问题。Opik 的追踪能力不仅覆盖所有 LLM 调用也覆盖应用中涉及的其余每一步骤——从检索上下文、构造 prompt 到最终生成响应。Opik 通过 TypeScript SDK、Python SDK、OpenTelemetry 一等支持 以及 REST API 提供 Agent 可观测性。一、启用 Agent 可观测性1. 安装并配置 SDK接入追踪的第一步是安装并配置 Opik SDK。以日志首条 trace 为目标根据技术栈选择对应 SDKTypeScript SDKnpm install opik在.env文件中配置环境变量# Set OPIK_API_KEY and OPIK_WORKSPACE in your .env file OPIK_API_KEYyour_api_key_here OPIK_WORKSPACEyour_workspace_name # Optional if you are using Opik Cloud: OPIK_URL_OVERRIDEhttps://www.comet.com/opik/apiPython SDKpip install opik安装后可执行opik configureCLI 命令完成交互式配置会提示输入 API Key 或自托管地址也可以在 Jupyter Notebook 中调用opik.configure。OpenTelemetry如果你已使用 OpenTelemetry 采集可通过 OTLP 协议直接对接 Opikexport OTEL_EXPORTER_OTLP_ENDPOINThttps://www.comet.com/opik/api/v1/private/otel export OTEL_EXPORTER_OTLP_HEADERSAuthorizationyour-api-key,Comet-Workspacedefault # If you are using self-hosted instance: # export OTEL_EXPORTER_OTLP_ENDPOINThttp://localhost:5173/api/v1/private/otel从源码可以看到SDK 的配置项解析采用“低优先级 → 高优先级”的多级覆盖默认值Cloud 端点https://www.comet.com/opik/api/、Default Project、workspacedefault→ 配置文件 → 环境变量OPIK_前缀相关定义集中在 sdks/python/src/opik/config.py。自托管时默认 API 地址为http://localhost:5173/api/Cloud 与本地环境由此区分。2. 通过集成接入主流框架安装并配置好 SDK 后可通过官方集成零侵入地追踪 Agent 调用。Opik 提供 30 种与主流框架和模型提供商的集成完整列表见 integrations overview。以下为最常用的几种接入方式。OpenAIPythonfrom opik.integrations.openai import track_openai from openai import OpenAI # Wrap your OpenAI client client OpenAI() client track_openai(client) # Use the client as normal completion client.chat.completions.create( modelgpt-4o, messages[ {role: user, content: Hello, how are you?}, ], ) print(completion.choices[0].message.content)包装后所有 OpenAI 调用都会自动记录到 Opik还可以与track装饰器组合为 Agent 的每一步单独记录 trace。OpenAITypeScriptnpm install opik-openaiexport OPIK_API_KEYyour-api-key # Only required if you are using the Opik Cloud version export OPIK_URL_OVERRIDEhttps://www.comet.com/opik/api # Cloud version # export OPIK_URL_OVERRIDEhttp://localhost:5173/api # Self-hostingimport OpenAI from openai; import { trackOpenAI } from opik-openai; // Initialize the original OpenAI client const openai new OpenAI({ apiKey: process.env.OPENAI_API_KEY, }); // Wrap the client with Opik tracking const trackedOpenAI trackOpenAI(openai); // Use the tracked client just like the original const completion await trackedOpenAI.chat.completions.create({ model: gpt-4, messages: [{ role: user, content: Hello, how can you help me today? }], }); console.log(completion.choices[0].message.content); // Ensure all traces are sent before your app terminates await trackedOpenAI.flush();AI Vercel SDK安装opik-vercel后通过OpikExporter将 AI SDK 的遥测导出到 Opikimport { openai } from ai-sdk/openai; import { generateText } from ai; import { NodeSDK } from opentelemetry/sdk-node; import { getNodeAutoInstrumentations } from opentelemetry/auto-instrumentations-node; import { OpikExporter } from opik-vercel; const sdk new NodeSDK({ traceExporter: new OpikExporter(), instrumentations: [getNodeAutoInstrumentations()], }); sdk.start(); const result await generateText({ model: openai(gpt-4o), prompt: What is love?, experimental_telemetry: { isEnabled: true }, }); console.log(result.text);ADKPython安装opik后用OpikTracer包装 ADK Agentfrom opik.integrations.adk import OpikTracer, track_adk_agent_recursive opik_tracer OpikTracer() # Define your ADK agent # Wrap your ADK agent with the OpikTracer track_adk_agent_recursive(agent, opik_tracer)LangGraphPython用OpikTracer作为回调注入图执行from opik.integrations.langchain import OpikTracer # Create your LangGraph graph graph ... app graph.compile(...) # Wrap your LangGraph graph with the OpikTracer opik_tracer OpikTracer(graphapp.get_graph(xrayTrue)) result app.invoke({messages: [HumanMessage(content How to use LangGraph ?)]}, config{callbacks: [opik_tracer]})如果你使用的框架不在集成列表中仍可借助函数装饰器或 Opik 客户端手动记录 trace见下文相关指引同样收录于 Log traces 页面。3. 在 Opik 中分析你的 Agent接入可观测性后即可在 Opik UI 中逐条审查 Agent 调用查看 Agent 图谱、回顾 Agent 的全部工具调用快速定位失败节点。二、高级用法使用函数装饰器track装饰器是给现有应用接入 Opik 日志最快捷的方式为函数添加装饰器后Opik 会为本次调用创建 span记录输入参数与函数输出若检测到被装饰函数嵌套在另一个被装饰函数内则会自动创建嵌套 span。Python 与 TypeScript 均支持装饰器TypeScript 需 5.0且 Opik 官方将其标注为实验性功能import { track } from opik; class TranslationService { track({ type: llm }) async generateText() { // Your LLM call here return Generated text; } track({ name: translate }) async translate(text: string) { // Your translation logic here return Translated: ${text}; } track({ name: process, projectName: translation-service }) async process() { const text await this.generateText(); return this.translate(text); } }Python 中可以为应用内任意函数添加track不仅追踪 LLM 调用也追踪检索、prompt 构造等其他步骤import opik import openai client openai.OpenAI() opik.track def retrieve_context(input_text): context [ What specific information are you looking for?, How can I assist you with your interests today?, Are there any topics youd like to explore?, ] return context opik.track def generate_response(input_text, context): full_prompt ( f If the user asks a non-specific question, use the context to provide a relevant response.\n fContext: {, .join(context)}\n fUser: {input_text}\n fAI: ) response client.chat.completions.create( modelgpt-3.5-turbo, messages[{role: user, content: full_prompt}] ) return response.choices[0].message.content opik.track(namemy_llm_application) def llm_chain(input_text): context retrieve_context(input_text) response generate_response(input_text, context) return response # Use the LLM chain result llm_chain(Hello, how are you?) print(result)track支持通过opik_args参数或opik_context模块定制 trace 与 span 关联的数据例如指定会话 thread id、标签与元数据# opik_context module import opik opik.track def llm_chain(text: str) - str: opik_context.update_current_trace( tags[llm_chatbot], metadata{version: 1.0, method: simple}, thread_idconversation-123, feedback_scores[ {name: user_feedback, value: 1} ], ) opik_context.update_current_span( metadata{model: gpt-4o}, ) return fProcessed: {text}# opik_args parameter import opik opik.track def llm_chain(text: str) - str: # LLM chain code return fProcessed: {text} result llm_chain( hello world, opik_args{ span: { tags: [llm, agent], metadata: {version: 1.0, method: simple} }, trace: { thread_id: conversation-123, tags: [user-session], metadata: {user_id: user-456} } } )两个细节值得注意opik_args可以在函数调用中把配置向下传播到嵌套函数而通过opik_context.update_current_span(output...)/update_current_trace(output...)显式设置的输出优先于函数返回值——函数结束时track只会补充返回值中尚未手动设置的键。若只想记录显式设置的输出请使用opik.track(capture_outputFalse)。从 sdks/python/src/opik/decorator/base_track_decorator.py 可以看到capture_output、flush、project_name、ignore_arguments等全部由TrackOptions承载正是装饰器行为的底层来源。使用底层 SDK需要完全掌控日志过程时可直接使用底层 Opik 客户端创建 trace 与 span。TypeScriptimport { Opik } from opik; const client new Opik({ apiUrl: https://www.comet.com/opik/api, apiKey: your-api-key, // Only required if you are using Opik Cloud projectName: your-project-name, workspaceName: your-workspace-name, // Optional }); // Log a trace with an LLM span const trace client.trace({ name: Trace, input: { prompt: Hello! }, output: { response: Hello, world! }, }); const span trace.span({ name: Span, type: llm, input: { prompt: Hello, world! }, output: { response: Hello, world! }, }); // Flush the client to send all traces and spans await client.flush();客户端还会读取.env中的环境变量完成配置更完整的配置说明见 SDK 配置。Pythonfrom opik import Opik client Opik(project_nameOpik client demo) # Create a trace trace client.trace( namemy_trace, input{user_question: Hello, how are you?}, output{response: Comment ça va?} ) # Add a span trace.span( nameAdd prompt template, input{text: Hello, how are you?, prompt_template: Translate the following text to French: {text}}, output{text: Translate the following text to French: hello, how are you?} ) # Add an LLM call trace.span( namellm_call, typellm, input{prompt: Translate the following text to French: hello, how are you?}, output{response: Comment ça va?} ) # End the trace trace.end()建议在 trace 与 span 结束时显式调用trace.end()/span.end()确保结束时间被正确记录。Opik 的日志功能为生产环境而设计所有日志操作都在后台线程中执行需要确保程序退出前全部数据已上报时调用client.flush()from opik import Opik client Opik() # Log some traces client.flush()使用上下文管理器记录 trace/spanPython SDK 提供两个核心上下文管理器以 Pythonic 的方式管理 trace 与 span 的生命周期自动处理清理与错误。opik.start_as_current_trace()—— 创建并管理一个 trace代表应用的整体执行流import opik with opik.start_as_current_trace(my-trace, project_namemy-project) as trace: # Your application logic here trace.input {user_query: What is the weather?} trace.output {response: Its sunny today!} trace.tags [weather, api-call] trace.metadata {model: gpt-4, temperature: 0.7}opik.start_as_current_span()—— 在 trace 内创建 span代表单个操作或函数调用import opik with opik.start_as_current_span(llm-call, typellm, project_namemy-project) as span: span.input {prompt: Explain quantum computing} span.output {response: Quantum computing is...} span.model gpt-4 span.provider openai span.usage { prompt_tokens: 10, completion_tokens: 50, total_tokens: 60 }两个上下文管理器的核心参数对照参数start_as_current_tracestart_as_current_spanname(str)trace 名称span 名称type(SpanType)—span 类型general、tool、llm、guardrail等input/output(Dict, 可选)输入/输出数据输入/输出数据tags(List[str], 可选)分类标签分类标签metadata(Dict, 可选)附加元数据附加元数据project_name(str, 可选)项目名依次回退到活动项目上下文、客户端配置项目名thread_id(str, 可选)多线程应用线程标识—model/provider(str, 可选)—LLM span 的模型/提供方flush(bool, 可选)是否立即冲刷默认 False是否立即冲刷嵌套结构span 可以嵌套在 trace 内构建层级关系import opik with opik.start_as_current_trace(chatbot-conversation, project_namechatbot) as trace: trace.input {user_message: Help me with Python} with opik.start_as_current_span(process-input, typegeneral) as span: span.input {raw_input: Help me with Python} span.output {processed_input: Python programming help request} with opik.start_as_current_span(generate-response, typellm) as span: span.input {prompt: Python programming help request} span.output {response: Id be happy to help with Python!} span.model gpt-4 span.provider openai trace.output {final_response: Id be happy to help with Python!}错误处理上下文管理器会在异常时自动关闭并记录 traceimport opik try: with opik.start_as_current_trace(risky-operation, project_namemy-project) as trace: trace.input {data: important data} result 1 / 0 # This will raise an exception trace.output {result: result} except ZeroDivisionError: print(Error occurred, but trace was logged)动态参数更新context 内外均可修改 trace/span 参数最终以更新后的值为准见上文dynamic-trace示例在 context 内覆写input、tags、metadata。Flush 控制flushTrue在退出 context 时同步立即发送数据flushFalse默认由后台异步稍后发送import opik # Immediate flush with opik.start_as_current_trace(immediate-trace, flushTrue) as trace: trace.input {data: important} # Deferred flush (default) with opik.start_as_current_trace(deferred-trace, flushFalse) as trace: trace.input {data: less urgent}最佳实践要点为 trace/span 使用清晰描述性名称为 span 设置正确的类型llm、retrieval、general等便于过滤分析携带模型名、参数、自定义指标等对调试有用的元数据让上下文管理器负责清理、应用逻辑负责错误处理按项目组织 trace 保持仪表盘整洁仅在确实需要数据立即可见时使用flushTrue因为它会触发同步、即时的数据上传可能拖慢应用。三、记录到指定项目Project默认情况下 trace 会记录到Default Project。可通过环境变量或客户端参数改变目标项目。TypeScript使用OPIK_PROJECT_NAME环境变量或向Opik客户端传入projectNameimport { Opik } from opik; const client new Opik({ projectName: my_project, // apiKey: my_api_key, // apiUrl: https://www.comet.com/opik/api, // workspaceName: my_workspace, });Python装饰器场景在track参数中指定底层 SDK 场景在Opik客户端构造函数中指定track(project_namemy_project) def my_function(): passfrom opik import Opik client Opik(project_namemy_project)项目名解析规则Python SDK项目名的最终归属取决于是否存在活动项目上下文active project context规则如下无项目上下文时适用于顶层track函数调用、Opik()客户端或未处于任何被追踪上下文中的原生集成如track_openai、OpikTracer按顺序解析 ——显式project_name参数track(project_name...)、Opik(project_name...)、OpikTracer(project_name...)、client.trace(project_name...)客户端配置OPIK_PROJECT_NAME环境变量或~/.opik.config文件兜底Default Project此时会记录一次警告提醒配置项目名。有项目上下文时一旦由顶层track(project_name...)或opik.project_context(...)建立了活动项目上下文所有嵌套操作都使用该上下文项目名——嵌套track函数即使传入不同project_name也以外层为准会记录警告在活动上下文中初始化的原生集成同理Opik()客户端方法若显式传了project_name则以显式参数为准未传则使用上下文项目。track的项目上下文会沿整个调用树传播from opik import track track(project_namemy-agent) def agent(query): context retrieve(query) return generate(context) track def retrieve(query): # Inherits my-agent from the parent context ... track def generate(context): # Also inherits my-agent from the parent context ...若嵌套函数指定了不同项目名会被忽略并保持外层项目track(project_namemy-agent) def agent(query): helper(query) # Still logs to my-agent, NOT other-project track(project_nameother-project) def helper(query): # Warning is logged: outer project my-agent will be used ...opik.project_context()上下文管理器则为块内所有 Opik 操作设置项目名包括track函数、原生集成以及未显式传project_name的Opik()客户端调用嵌套规则相同——第一个运行的project_context或track(project_name...)拥有上下文内部不同项目名会被忽略并记录警告import opik with opik.project_context(customer-support): # track-decorated functions and native integrations # all use customer-support as the project name my_agent(query)⚠️ 一致性警告当脚本同时使用track追踪和其他 Opik API 调用如evaluate()、get_or_create_dataset()、Prompt()时如果项目名设置不一致trace 与 API 对象可能落入不同项目。务必让opik.configure(project_name...)决定tracktrace 去向与每个 API 调用显式传入的project_name保持一致import opik opik.configure(project_namemy-project) dataset client.get_or_create_dataset(namemy-dataset, project_namemy-project) evaluation evaluate( datasetdataset, taskevaluation_task, project_namemy-project, # must match opik.configure value above ... )四、记录到指定环境EnvironmentEnvironment 允许为 trace 打上生命周期阶段标签如development、staging、production便于在 Opik UI 中分段与过滤可观测数据。环境解析顺序TypeScript 与 Python 一致显式参数client.trace(environment: ...)/track(environment...)优先其次OPIK_ENVIRONMENT环境变量。TypeScript底层 SDKimport { Opik } from opik; const client new Opik({ projectName: my-project }); const trace client.trace({ name: my_trace, input: { question: Hello }, environment: production, }); trace.end(); await client.flush();Python装饰器import opik opik.track(environmentproduction) def my_pipeline(input_text: str) - str: return input_text my_pipeline(Hello, world!)Python底层 SDKfrom opik import Opik client Opik(project_namemy_project) trace client.trace( namemy_trace, input{question: Hello}, environmentproduction, ) trace.end()管理环境可以通过 SDK 编程式地管理工作区内的命名环境集合from opik import Opik client Opik() # Create a new environment env client.create_environment( nameproduction, descriptionLive production traffic, color#FF0000, ) # List all environments envs client.get_environments() # Update an environment client.update_environment(production, descriptionUpdated description) # Delete an environment client.delete_environment(production)TypeScript 侧对应client.createEnvironment(...)、client.getEnvironments()、client.updateEnvironment(...)、client.deleteEnvironment(...)。环境管理的后端接口位于 EnvironmentsResource.javacreateEnvironment等 REST 操作。按环境过滤打上环境标签后可在filter_string中使用environment字段过滤 trace、span 与会话线程支持、!、in、not_infrom opik import Opik client Opik() # Only production traces traces client.search_traces( project_namemy_project, filter_stringenvironment production ) # Multiple environments traces client.search_traces( project_namemy_project, filter_stringenvironment in (production, staging) ) # Same filtering applies to spans spans client.search_spans( project_namemy_project, filter_stringenvironment production ) # And to conversation threads threads client.search_threads( project_namemy_project, filter_stringenvironment production ) # Combine with other thread filters active_prod_threads client.search_threads( project_namemy_project, filter_stringenvironment production AND status active )五、冲刷Flush与关闭追踪冲刷 trace 与 spanPython 与 TypeScript SDK 均为生产环境设计默认在后台批量发送 trace/span。以下情况需要显式冲刷运行短生命周期脚本或排查为何 trace/span 未出现在 Opik。Pythonfrom opik import Opik client Opik() client.flush()也可为track装饰器设置flushTrue确保程序退出前数据已上报from opik import track track(flushTrue) def llm_chain(input_text): # LLM chain code return fProcessed: {input_text}TypeScriptimport { Opik } from opik; const client new Opik(); client.flush();禁用日志过程全局禁用通过OPIK_TRACK_DISABLE环境变量即可全局关闭日志TypeScript 还支持配置文件中的track_disable或向Opik客户端构造函数传入trackDisable: true。从 Config.ts 可以看到该开关的完整解析链构造函数默认trackDisable: false随后从OPIK_TRACK_DISABLE环境变量与配置文件逐级覆盖。禁用后track装饰器、各集成以及手动的client.trace()调用都会停止向 Opik 发送数据。动态开关TypeScriptimport { isTracingActive, setTracingActive, resetTracingToConfigDefault, } from opik; // Check the current state of the tracing flag console.log(isTracingActive()); // Disable the logging process setTracingActive(false); // Re-enable the logging process setTracingActive(true); // Reset to the value resolved from configuration (OPIK_TRACK_DISABLE / trackDisable) resetTracingToConfigDefault();isTracingActive/setTracingActive的实现位于 TracingRuntimeConfig.ts运行时开关会覆盖配置默认值而resetTracingToConfigDefault()会清除运行时覆盖、恢复配置默认行为。动态开关Pythonimport opik # Check the current state of the tracing flag print(opik.is_tracing_active()) # Disable the logging process opik.set_tracing_active(False) # Re-enable the logging process print(opik.set_tracing_active(True))下一步完成 Agent 可观测性接入后可以继续深入记录聊天会话记录用户反馈 / 标注 trace配置在线评估指标赞分享人工智能LLMOps模型评测可观测性AI AgentAI 应用后端前端【免费下载链接】comet-llmDebug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.项目地址https://gitcode.com/GitHub_Trending/co/comet-llm点击查看免费下载相关推荐nanobot 接入 Langfuse 可观测性实战环境变量驱动的全链路 LLM 调用追踪nanobot 接入 Langfuse 可观测性实战环境变量驱动的全链路 LLM 调用追踪 nanobot 可以通过 Langfuse 的 OpenAI SD人工智能AI AgentAgent 框架多智能体工具调用MCP Clients交互助手后端任务调度btop 快速上手5 分钟看懂 CPU、内存与 GPU定位系统瓶颈btop 快速上手5 分钟看懂 CPU、内存与 GPU定位系统瓶颈 btop 是一个跑在终端里的 C 系统资源监控工具把 CPU、内存、磁盘、网络和CLI指标监控运维AutoGen 可观测性实战用 OpenTelemetry 为 Agent 应用接入端到端链路追踪AutoGen 可观测性实战用 OpenTelemetry 为 Agent 应用接入端到端链路追踪 AutoGen本仓库 python 侧的 agentic人工智能AI AgentAgent 框架多智能体大模型工具调用创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考
阅读完成 · 觉得有帮助？
