Database

Sheds light on the database schema

Overview

This database schema defines a set of models designed to capture various aspects of an application related to projects, user interactions, traces, system information, errors, and different types of calls (LLM, Tool, and Agent). These models have specific relationships and foreign key constraints to ensure data integrity and consistency.

Database Models

  1. UserInteractionModel

  2. TraceModel

  3. ProjectInfoModel

  4. SystemInfoModel

  5. ErrorModel

  6. LLMCallModel

  7. ToolCallModel

  8. AgentCallModel

  9. MetricModel

The relationships between these models are established using SQLAlchemy's relationship function.

Model Descriptions

UserInteractionModel

This model represents a user's interaction with the application, such as an input or output. Each interaction is associated with a project, a trace, and agent. The model stores the following information:

  • id: Unique identifier for the user interaction.

  • project_id: Foreign key referencing the ProjectInfoModel.

  • trace_id: Foreign key referencing the TraceModel.

  • agent_id: Foreign key referencing the AgentCallModel.

  • interaction_type: The type of interaction, either 'input' or 'output'.

  • content: The content of the interaction.

  • timestamp: The timestamp of the interaction.

Relationships:

  • trace: Relationship with TraceModel.

  • project: Relationship with ProjectInfoModel.

  • agent: Relationship with AgentCallModel.

Example usage:

# Create a new user interaction
user_interaction = UserInteractionModel(
   project_id=project.id,
   trace_id=trace.id,
   interaction_type='input',
   content='Hello, how are you?',
   timestamp=datetime.now()
)

session.add(user_interaction)

session.commit()

TraceModel

This model represents a trace of the application's execution. Each trace is associated with a project and can have various related entities, such as system information, errors, LLM calls, tool calls, agent calls, and user interactions. The model stores the following information:

  • id: Unique identifier for the trace.

  • project_id: Foreign key referencing the ProjectInfoModel.

  • start_time: The start time of the trace.

  • end_time: The end time of the trace.

  • duration: The duration of the trace.

Relationships:

project: Relationship with ProjectInfoModel.

system_info: One-to-one relationship with SystemInfoModel.

errors: One-to-many relationship with ErrorModel.

llm_calls: One-to-many relationship with LLMCallModel.

tool_calls: One-to-many relationship with ToolCallModel.

agent_calls: One-to-many relationship with AgentCallModel.

user_interactions: One-to-many relationship with UserInteractionModel.

metrics: One-to-many relationship with MetricModel.

Example usage:

# Create a new trace for the project
trace = TraceModel(
    project_id=project.id,
    start_time=datetime.now(),
    end_time=datetime.now() + timedelta(minutes=30),
    duration=30.0
)

session.add(trace)

session.commit()

ProjectInfoModel

This model represents information about a project, such as its name, start and end times, duration, total cost, and total tokens. The model has relationships with various other entities, such as traces, user interactions, LLM calls, tool calls, agent calls, and system information. The model stores the following information:

  • id: Unique identifier for the project.

  • project_name: The name of the project.

  • start_time: The start time of the project.

  • end_time: The end time of the project.

  • duration: The duration of the project.

  • total_cost: The total cost of the project.

  • total_tokens: The total number of tokens used in the project.

Relationships:

traces: One-to-many relationship with TraceModel.

llm_calls: One-to-many relationship with LLMCallModel.

tool_calls: One-to-many relationship with ToolCallModel.

agent_calls: One-to-many relationship with AgentCallModel.

system_info: One-to-one relationship with SystemInfoModel.

errors: One-to-many relationship with ErrorModel.

user_interactions: One-to-many relationship with UserInteractionModel.

Example usage:

# Create a new project
project = ProjectInfoModel(
    project_name='My Project',
    start_time=datetime.now(),
    total_cost=1000.0,
    total_tokens=10000
)

session.add(project)

session.commit()

SystemInfoModel

This model represents information about the system on which the application was running, such as the operating system, Python version, CPU and GPU information, disk information, and installed packages. The model is associated with a project and a trace. The model stores the following information:

  • id: Unique identifier for the system information.

  • project_id: Foreign key referencing the ProjectInfoModel.

  • os_name: The name of the operating system.

  • os_version: The version of the operating system.

  • python_version: The version of Python.

  • cpu_info: Information about the CPU.

  • gpu_info: Information about the GPU (if available).

  • disk_info: Information about the disk.

  • memory_total: The total memory of the system.

  • installed_packages: A list of installed packages.

  • trace_id: Foreign key referencing the TraceModel.

  • Relationships:

trace: Relationship with TraceModel.

project: Relationship with ProjectInfoModel.

Example usage:

# Create a new system information entry
system_info = SystemInfoModel(
    project_id=project.id,
    os_name='Ubuntu',
    os_version='20.04',
    python_version='3.9.7',
    cpu_info='Intel Core i7-10700K',
    memory_total=32.0,
    installed_packages='numpy==1.21.5, pandas==1.3.4',
    trace_id=trace.id
)

session.add(system_info)

session.commit()

ErrorModel

This model represents errors that occurred during the application's execution. Each error is associated with a project, a trace, and possibly an agent, tool call, or LLM call. The model stores the following information:

  • id: Unique identifier for the error.

  • project_id: Foreign key referencing the ProjectInfoModel.

  • trace_id: Foreign key referencing the TraceModel.

  • agent_id: Foreign key referencing the AgentCallModel.

  • tool_call_id: Foreign key referencing the ToolCallModel.

  • llm_call_id: Foreign key referencing the LLMCallModel.

  • error_type: The type of error, either 'LLM', 'Tool', or 'Agent'.

  • error_message: The error message.

  • timestamp: The timestamp of the error.

Example usage:

# Create a new error
error = ErrorModel(
    project_id=project.id,
    trace_id=trace.id,
    agent_id=agent.id,
    error_type='LLM',
    error_message='Invalid input format',
    timestamp=datetime.now()
)

session.add(error)

session.commit()

LLMCallModel

This model represents a call to a Large Language Model (LLM). Each LLM call is associated with a project, an agent (if applicable), and a trace. The model stores the following information:

  • id: Unique identifier for the LLM call.

  • project_id: Foreign key referencing the ProjectInfoModel.

  • agent_id: Foreign key referencing the AgentCallModel.

  • name: The name of the LLM.

  • model: The model used for the LLM call.

  • input_prompt: The input prompt for the LLM call.

  • output: The output of the LLM call.

  • tool_call: The tool call associated with the LLM call (if applicable).

  • start_time: The start time of the LLM call.

  • end_time: The end time of the LLM call.

  • duration: The duration of the LLM call.

  • token_usage: The token usage for the LLM call.

  • cost: The cost of the LLM call.

  • memory_used: The memory used by the LLM call.

  • trace_id: Foreign key referencing the TraceModel.

Example usage:

# Create a new LLM call
llm_call = LLMCallModel(
    project_id=project.id,
    agent_id=agent.id,
    name='GPT-4o-mini',
    model='gpt-4o-mini',
    input_prompt='What is the capital of France?',
    output='The capital of France is Paris.',
    start_time=datetime.now(),
    end_time=datetime.now() + timedelta(seconds=5),
    duration=5.0,
    token_usage={'total': 20, 'prompt': 10, 'completion': 10},
    cost={'total': 0.02, 'prompt': 0.01, 'completion': 0.01},
    memory_used=512,
    trace_id=trace.id
)

session.add(llm_call)

session.commit()

ToolCallModel

This model represents a call to a tool. Each tool call is associated with a project, an agent (if applicable), and a trace. The model stores the following information:

  • id: Unique identifier for the tool call.

  • project_id: Foreign key referencing the ProjectInfoModel.

  • agent_id: Foreign key referencing the AgentCallModel.

  • name: The name of the tool.

  • input_parameters: The input parameters for the tool call.

  • output: The output of the tool call.

  • start_time: The start time of the tool call.

  • end_time: The end time of the tool call.

  • duration: The duration of the tool call.

  • memory_used: The memory used by the tool call.

  • trace_id: Foreign key referencing the TraceModel.

  • network_calls: The network calls made by the tool (if applicable).

Example usage:

# Create a new tool call
tool_call = ToolCallModel(
    project_id=project.id,
    agent_id=agent.id,
    name='Image Processor',
    input_parameters='{"image_path": "test.jpg", "mode": "grayscale"}',
    output='{"processed_image": "processed.jpg"}',
    start_time=datetime.now(),
    end_time=datetime.now() + timedelta(seconds=10),
    duration=10.0,
    memory_used=256,
    trace_id=trace.id
)

session.add(tool_call)

session.commit()

AgentCallModel

This model represents a call to an agent. Each agent call is associated with a project and a trace, and can have related LLM calls, tool calls, and user interactions. The model stores the following information:

  • id: Unique identifier for the agent call.

  • project_id: Foreign key referencing the ProjectInfoModel.

  • name: The name of the agent.

  • start_time: The start time of the agent call.

  • end_time: The end time of the agent call.

  • trace_id: Foreign key referencing the TraceModel.

  • llm_call_ids: The IDs of the related LLM calls.

  • tool_call_ids: The IDs of the related tool calls.

  • user_interaction_ids: The IDs of the related user interactions.

Example usage:

# Create a new agent call
agent = AgentCallModel(
    project_id=project.id,
    name='Helpful Agent',
    start_time=datetime.now(),
    end_time=datetime.now() + timedelta(minutes=15),
    trace_id=trace.id,
    llm_call_ids=[llm_call.id],
    tool_call_ids=[tool_call.id],
    user_interaction_ids=[user_interaction.id]
)

session.add(agent)

session.commit()

MetricModel

This model represents a metric calculated for a trace. Each metric is associated with a trace and stores the following information:

  • id: Unique identifier for the metric.

  • trace_id: Foreign key referencing the TraceModel.

  • metric_name: The name of the metric.

  • score: The score of the metric.

  • reason: The reason for the metric score.

  • result_detail: Additional details about the metric result.

  • config: The configuration used to calculate the metric.

  • start_time: The start time of the metric calculation.

  • end_time: The end time of the metric calculation.

  • duration: The duration of the metric calculation.

  • timestamp: The timestamp of the metric calculation.

Example usage:

# Create a new metric
metric = MetricModel(
    trace_id=trace.id,
    metric_name='Response Quality',
    score=4.5,
    reason='Provided a clear and informative response',
    result_detail={'sentiment': 'positive', 'coherence': 0.8},
    config={'model': 'text-davinci-002', 'temperature': 0.7},
    start_time=datetime.now(),
    end_time=datetime.now() + timedelta(seconds=2),
    duration=2.0,
    timestamp=datetime.now()
)

session.add(metric)

session.commit()

Relationships Summary

  • ProjectInfoModelTraceModel: One-to-Many

  • ProjectInfoModelUserInteractionModel, LLMCallModel, ToolCallModel, AgentCallModel: One-to-Many

  • ProjectInfoModelSystemInfoModel: One-to-One

  • TraceModelUserInteractionModel, ErrorModel, LLMCallModel, ToolCallModel, AgentCallModel: One-to-Many

  • AgentCallModelLLMCallModel, ToolCallModel, UserInteractionModel: One-to-Many

Last updated