AimableDocs
DocsAPI Reference

Observability

Observability API

The Observability domain in the Aimable Platform API enables developers and administrators to monitor, analyze, and manage AI interactions through integration with Langfuse, a powerful observability platform for LLM applications. This API allows you to configure tracing, retrieve metrics, inspect individual traces, and provide feedback on trace quality or compliance.

Observability is essential for understanding model behavior, detecting issues like PII leakage, auditing usage, and improving prompt engineering over time.

Key Concepts

  • Langfuse Config: Credentials and project settings used to connect a space or tenant to Langfuse for trace ingestion.
  • Trace: A record of an AI interaction, including inputs, outputs, model used, timestamps, and metadata.
  • Feedback: Human or automated evaluation of a trace (e.g., flagging PII or policy violations).
  • Metrics: Aggregated data such as number of traces, latency, token usage, etc., over time.
  • Space: A logical unit in Aimable (e.g., a workspace or app) that can have its own observability settings.

Authentication

All endpoints require authentication via the X-API-Key header:

X-API-Key: <your-api-key>

Unauthorized access returns 401, and insufficient permissions return 403.

Common Workflows

1. Configure Langfuse for a Space

To start collecting traces, first set up Langfuse credentials for a space.

Set Langfuse Config

curl -X PUT 'https://api-proxy/v1/spaces/123e4567-e89b-12d3-a456-426614174000/observability/config' \
  -H 'X-API-Key: akp_1234567890abcdef' \
  -H 'Content-Type: application/json' \
  -d '{
    "langfuse_public_key": "pk-lf-987654321",
    "langfuse_secret_key": "sk-lf-123456789",
    "langfuse_project_id": "proj-012345"
  }'

Response (200 OK): Configuration saved successfully.

You can retrieve it later:

curl -X GET 'https://api-proxy/v1/spaces/123e4567-e89b-12d3-a456-426614174000/observability/config' \
  -H 'X-API-Key: akp_1234567890abcdef'

Returns current config (secrets partially masked).

To disable tracing:

curl -X DELETE 'https://api-proxy/v1/spaces/123e4567-e89b-12d3-a456-426614174000/observability/config' \
  -H 'X-API-Key: akp_1234567890abcdef'

2. Monitor Usage with Metrics

Retrieve aggregated metrics for a space:

curl -X GET 'https://api-proxy/v1/spaces/123e4567-e89b-12d3-a456-426614174000/observability/metrics?from_timestamp=2024-04-01T00:00:00Z&to_timestamp=2024-04-02T00:00:00Z' \
  -H 'X-API-Key: akp_1234567890abcdef'

Response (200 OK):

{
  "total_traces": 42,
  "total_tokens": 12400,
  "avg_latency_ms": 1250,
  "traces_per_hour": [
    { "timestamp": "2024-04-01T10:00:00Z", "count": 5 },
    { "timestamp": "2024-04-01T11:00:00Z", "count": 8 }
  ]
}

Tenant-level metrics are also available:

curl -X GET '/api-proxy/v1/admin/observability/metrics' \
  -H 'X-API-Key: akp_1234567890abcdef'

3. Inspect and Audit Traces

List recent traces in a space:

curl -X GET 'https://api-proxy/v1/spaces/123e4567-e89b-12d3-a456-426614174000/observability/traces?limit=5&status=success&pii_detected=true' \
  -H 'X-API-Key: akp_1234567890abcdef'

Returns array of trace summaries.

Get a specific trace:

curl -X GET 'https://api-proxy/v1/spaces/123e4567-e89b-12d3-a456-426614174000/observability/traces/tr-98765' \
  -H 'X-API-Key: akp_1234567890abcdef'

Returns full trace details including prompts, generations, and metadata.

4. Provide Feedback on Traces

Flag a trace for PII or policy issues:

curl -X POST 'https://api-proxy/v1/spaces/123e4567-e89b-12d3-a456-426614174000/observability/traces/tr-98765/feedback' \
  -H 'X-API-Key: akp_1234567890abcdef' \
  -H 'Content-Type: application/json' \
  -d '{
    "reason_code": "pii_confirmed",
    "notes": {
      "comment": "Email address detected in output",
      "reviewer": "admin@company.com"
    }
  }'

Response (201 Created): Feedback recorded.

List all feedback for a trace:

curl -X GET 'https://api-proxy/v1/spaces/123e4567-e89b-12d3-a456-426614174000/observability/traces/tr-98765/feedback' \
  -H 'X-API-Key: akp_1234567890abcdef'

Tenant-Level Configuration

Administrators can manage tenant-wide Langfuse settings:

# Set tenant config
curl -X PUT '/api-proxy/v1/admin/observability/config' \
  -H 'X-API-Key: akp_admin_123' \
  -d '{
    "langfuse_public_key": "pk-lf-global",
    "langfuse_secret_key": "sk-lf-global"
  }'

Useful for enforcing observability policies across all spaces.

Error Handling

Invalid requests return 422 Unprocessable Entity with details:

{
  "detail": [
    {
      "loc": ["body", "langfuse_public_key"],
      "msg": "Field required",
      "type": "missing"
    }
  ]
}

Ensure correct data types and required fields are provided.

By integrating these observability tools, teams gain full visibility into AI operations, enabling better governance, debugging, and continuous improvement.

Related Endpoints