Monitoring Endpoints

Queue, worker, and system monitoring.

---

Overview

Queues

Method	Endpoint	Description
GET	/monitoring/queues	Queue depths and status

Workers

Method	Endpoint	Description
GET	/monitoring/workers	Worker pool status
GET	/monitoring/workers/health	Worker health checks

Tasks

Method	Endpoint	Description
GET	/monitoring/tasks	Recent task activity
GET	/monitoring/tasks/{task_id}	Task details
GET	/monitoring/tasks/{task_id}/media	Task media assets

Workflows

Method	Endpoint	Description
GET	/monitoring/workflows	Active workflows
GET	/monitoring/workflows/stats	Workflow statistics
GET	/monitoring/workflows/{workflow_id}	Workflow details

System

Method	Endpoint	Description
GET	/monitoring/stats	System-wide statistics

---

API Reference

Custom API - Monitoring 1.0.0

This is a very custom OpenAPI schema

---

Monitoring

---

GET /api/v1/monitoring/tasks

List Monitored Tasks

Description

List tasks with filtering, pagination, and sorting.

Returns a paginated list of tasks with computed duration and error status. All filters are optional and can be combined.

Input parameters

Parameter	In	Type	Default	Nullable	Description
HTTPBearer	header	string	N/A	No	JWT Bearer token
created_after	query			No	Filter tasks created after this datetime
created_before	query			No	Filter tasks created before this datetime
node_type	query			No	Filter by node type (e.g. 'preview_edit' isolates the look-preview flow, which otherwise blends into EDITING). 'none' selects legacy tasks with no node.
origin_instance	query			No	Filter by origin instance
origin_tenant_id	query			No	Filter by origin tenant ID
origin_zone	query			No	Filter by origin zone
page	query	integer	1	No	Page number (1-indexed)
page_size	query	integer	50	No	Items per page
provider_type	query			No	Filter by provider type
search	query			No	Search by task ID prefix or origin_reference substring (e.g. 'look_preview' isolates the look-preview flow; a backend entity id pasted here finds its task).
sort_by	query	string	created_at	No	Sort field
sort_order	query	string	desc	No	Sort order
status	query			No	Filter by status
task_type	query			No	Filter by task type

Responses

200 OK422 Unprocessable Entity

application/json

{
    "items": [
        {
            "id": "da89e544-0b6a-4ff2-b954-88c6a561b3aa",
            "task_type": "GENERATION",
            "node_type": null,
            "provider_type": null,
            "status": "PENDING",
            "progress": 10.12,
            "origin_instance": "string",
            "origin_zone": null,
            "origin_tenant_id": null,
            "created_at": "2022-04-13T15:42:05.901Z",
            "started_at": null,
            "completed_at": null,
            "duration_ms": null,
            "has_error": true
        }
    ],
    "total": 0,
    "page": 0,
    "page_size": 0,
    "pages": 0
}

⚠️ This example has been generated automatically from the schema and it is not accurate. Refer to the schema for more information.

Schema of the response body

{
    "properties": {
        "items": {
            "items": {
                "$ref": "#/components/schemas/TaskListItem"
            },
            "type": "array",
            "title": "Items"
        },
        "total": {
            "type": "integer",
            "title": "Total",
            "description": "Total number of tasks matching filters"
        },
        "page": {
            "type": "integer",
            "title": "Page",
            "description": "Current page number (1-indexed)"
        },
        "page_size": {
            "type": "integer",
            "title": "Page Size",
            "description": "Number of items per page"
        },
        "pages": {
            "type": "integer",
            "title": "Pages",
            "description": "Total number of pages"
        }
    },
    "type": "object",
    "required": [
        "items",
        "total",
        "page",
        "page_size",
        "pages"
    ],
    "title": "TaskListResponse",
    "description": "Paginated task list response."
}

application/json

{
    "detail": [
        {
            "loc": [
                null
            ],
            "msg": "string",
            "type": "string"
        }
    ]
}

⚠️ This example has been generated automatically from the schema and it is not accurate. Refer to the schema for more information.

Schema of the response body

{
    "properties": {
        "detail": {
            "items": {
                "$ref": "#/components/schemas/ValidationError"
            },
            "type": "array",
            "title": "Detail"
        }
    },
    "type": "object",
    "title": "HTTPValidationError"
}

---

GET /api/v1/monitoring/tasks/{task_id}

Get Monitored Task

Description

Get detailed task information including configuration and task-type specific fields.

Input parameters

Parameter	In	Type	Default	Nullable	Description
HTTPBearer	header	string	N/A	No	JWT Bearer token
task_id	path	string		No

Responses

200 OK422 Unprocessable Entity

application/json

{
    "id": "431509bd-cb9d-44f3-9f4d-8bff44e3f76a",
    "task_type": "GENERATION",
    "node_type": null,
    "node_schema_version": null,
    "provider_type": null,
    "status": "PENDING",
    "progress": 10.12,
    "origin_instance": "string",
    "origin_zone": null,
    "origin_tenant_id": null,
    "origin_reference": null,
    "created_at": "2022-04-13T15:42:05.901Z",
    "updated_at": "2022-04-13T15:42:05.901Z",
    "started_at": null,
    "completed_at": null,
    "duration_ms": null,
    "queue_name": "string",
    "priority": 0,
    "celery_task_id": null,
    "service_task_id": null,
    "executor_type": null,
    "executor_instance": null,
    "config": null,
    "webhook_url": null,
    "errors": null,
    "retry_count": 0,
    "is_resumed": true,
    "resume_attempt_count": 0,
    "last_resume_attempt": null,
    "service_requests": null,
    "task_data": {}
}

⚠️ This example has been generated automatically from the schema and it is not accurate. Refer to the schema for more information.

Schema of the response body

{
    "properties": {
        "id": {
            "type": "string",
            "format": "uuid",
            "title": "Id"
        },
        "task_type": {
            "$ref": "#/components/schemas/TaskType"
        },
        "node_type": {
            "anyOf": [
                {
                    "type": "string"
                },
                {
                    "type": "null"
                }
            ],
            "title": "Node Type"
        },
        "node_schema_version": {
            "anyOf": [
                {
                    "type": "integer"
                },
                {
                    "type": "null"
                }
            ],
            "title": "Node Schema Version"
        },
        "provider_type": {
            "anyOf": [
                {
                    "$ref": "#/components/schemas/ProviderType"
                },
                {
                    "type": "null"
                }
            ]
        },
        "status": {
            "$ref": "#/components/schemas/TaskStatus"
        },
        "progress": {
            "type": "number",
            "title": "Progress",
            "default": 0.0
        },
        "origin_instance": {
            "type": "string",
            "title": "Origin Instance"
        },
        "origin_zone": {
            "anyOf": [
                {
                    "type": "string"
                },
                {
                    "type": "null"
                }
            ],
            "title": "Origin Zone"
        },
        "origin_tenant_id": {
            "anyOf": [
                {
                    "type": "string"
                },
                {
                    "type": "null"
                }
            ],
            "title": "Origin Tenant Id"
        },
        "origin_reference": {
            "anyOf": [
                {
                    "type": "string"
                },
                {
                    "type": "null"
                }
            ],
            "title": "Origin Reference"
        },
        "created_at": {
            "type": "string",
            "format": "date-time",
            "title": "Created At"
        },
        "updated_at": {
            "type": "string",
            "format": "date-time",
            "title": "Updated At"
        },
        "started_at": {
            "anyOf": [
                {
                    "type": "string",
                    "format": "date-time"
                },
                {
                    "type": "null"
                }
            ],
            "title": "Started At"
        },
        "completed_at": {
            "anyOf": [
                {
                    "type": "string",
                    "format": "date-time"
                },
                {
                    "type": "null"
                }
            ],
            "title": "Completed At"
        },
        "duration_ms": {
            "anyOf": [
                {
                    "type": "number"
                },
                {
                    "type": "null"
                }
            ],
            "title": "Duration Ms",
            "description": "Duration in milliseconds"
        },
        "queue_name": {
            "type": "string",
            "title": "Queue Name",
            "description": "Celery queue name",
            "default": "inference"
        },
        "priority": {
            "type": "integer",
            "title": "Priority",
            "description": "Task priority (higher = more priority)",
            "default": 0
        },
        "celery_task_id": {
            "anyOf": [
                {
                    "type": "string"
                },
                {
                    "type": "null"
                }
            ],
            "title": "Celery Task Id"
        },
        "service_task_id": {
            "anyOf": [
                {
                    "type": "string"
                },
                {
                    "type": "null"
                }
            ],
            "title": "Service Task Id"
        },
        "executor_type": {
            "anyOf": [
                {
                    "type": "string"
                },
                {
                    "type": "null"
                }
            ],
            "title": "Executor Type"
        },
        "executor_instance": {
            "anyOf": [
                {
                    "type": "string"
                },
                {
                    "type": "null"
                }
            ],
            "title": "Executor Instance"
        },
        "config": {
            "anyOf": [
                {
                    "additionalProperties": true,
                    "type": "object"
                },
                {
                    "type": "null"
                }
            ],
            "title": "Config"
        },
        "webhook_url": {
            "anyOf": [
                {
                    "type": "string"
                },
                {
                    "type": "null"
                }
            ],
            "title": "Webhook Url",
            "description": "Webhook URL for notifications"
        },
        "errors": {
            "anyOf": [
                {
                    "items": {
                        "type": "string"
                    },
                    "type": "array"
                },
                {
                    "type": "null"
                }
            ],
            "title": "Errors"
        },
        "retry_count": {
            "type": "integer",
            "title": "Retry Count",
            "default": 0
        },
        "is_resumed": {
            "type": "boolean",
            "title": "Is Resumed",
            "description": "Whether task was resumed",
            "default": false
        },
        "resume_attempt_count": {
            "type": "integer",
            "title": "Resume Attempt Count",
            "description": "Number of resume attempts",
            "default": 0
        },
        "last_resume_attempt": {
            "anyOf": [
                {
                    "type": "string",
                    "format": "date-time"
                },
                {
                    "type": "null"
                }
            ],
            "title": "Last Resume Attempt",
            "description": "Timestamp of last resume attempt"
        },
        "service_requests": {
            "anyOf": [
                {
                    "items": {
                        "additionalProperties": true,
                        "type": "object"
                    },
                    "type": "array"
                },
                {
                    "type": "null"
                }
            ],
            "title": "Service Requests",
            "description": "Log of external service requests made during task processing"
        },
        "task_data": {
            "additionalProperties": true,
            "type": "object",
            "title": "Task Data",
            "description": "Task-type specific fields (prompt, width, height, result_image_path, etc.)"
        }
    },
    "type": "object",
    "required": [
        "id",
        "task_type",
        "status",
        "origin_instance",
        "created_at",
        "updated_at"
    ],
    "title": "TaskDetailResponse",
    "description": "Detailed task view with full configuration."
}

application/json

{
    "detail": [
        {
            "loc": [
                null
            ],
            "msg": "string",
            "type": "string"
        }
    ]
}

⚠️ This example has been generated automatically from the schema and it is not accurate. Refer to the schema for more information.

Schema of the response body

{
    "properties": {
        "detail": {
            "items": {
                "$ref": "#/components/schemas/ValidationError"
            },
            "type": "array",
            "title": "Detail"
        }
    },
    "type": "object",
    "title": "HTTPValidationError"
}

---

GET /api/v1/monitoring/tasks/{task_id}/media

Get Task Media

Description

Get input and output media URLs for a task.

Returns URLs that can be used directly in or

Input parameters

Parameter	In	Type	Default	Nullable	Description
HTTPBearer	header	string	N/A	No	JWT Bearer token
task_id	path	string		No

Responses

200 OK422 Unprocessable Entity

application/json

{
    "task_id": "595acb12-4982-447e-9e0a-f00b841bb4f3",
    "inputs": [
        {
            "name": "string",
            "path": "string",
            "url": "string",
            "media_type": "string"
        }
    ],
    "outputs": null
}

⚠️ This example has been generated automatically from the schema and it is not accurate. Refer to the schema for more information.

Schema of the response body

{
    "properties": {
        "task_id": {
            "type": "string",
            "format": "uuid",
            "title": "Task Id"
        },
        "inputs": {
            "items": {
                "$ref": "#/components/schemas/MediaItem"
            },
            "type": "array",
            "title": "Inputs",
            "description": "Input media files"
        },
        "outputs": {
            "items": {
                "$ref": "#/components/schemas/MediaItem"
            },
            "type": "array",
            "title": "Outputs",
            "description": "Output media files"
        }
    },
    "type": "object",
    "required": [
        "task_id"
    ],
    "title": "TaskMediaResponse",
    "description": "Input and output media for a task."
}

application/json

{
    "detail": [
        {
            "loc": [
                null
            ],
            "msg": "string",
            "type": "string"
        }
    ]
}

⚠️ This example has been generated automatically from the schema and it is not accurate. Refer to the schema for more information.

Schema of the response body

{
    "properties": {
        "detail": {
            "items": {
                "$ref": "#/components/schemas/ValidationError"
            },
            "type": "array",
            "title": "Detail"
        }
    },
    "type": "object",
    "title": "HTTPValidationError"
}

---

GET /api/v1/monitoring/stats

Get Monitoring Stats

Description

Get aggregate statistics for the monitoring dashboard.

Returns counts by status, provider, and task type, plus time-based metrics.

Input parameters

Parameter	In	Type	Default	Nullable	Description
HTTPBearer	header	string	N/A	No	JWT Bearer token
origin_instance	query			No	Filter by origin instance
origin_zone	query			No	Filter by origin zone

Responses

200 OK422 Unprocessable Entity

application/json

{
    "total_tasks": 0,
    "by_status": {},
    "by_provider": {},
    "by_task_type": {},
    "tasks_last_hour": 0,
    "tasks_last_24h": 0
}

⚠️ This example has been generated automatically from the schema and it is not accurate. Refer to the schema for more information.

Schema of the response body

{
    "properties": {
        "total_tasks": {
            "type": "integer",
            "title": "Total Tasks",
            "description": "Total number of tasks"
        },
        "by_status": {
            "additionalProperties": {
                "type": "integer"
            },
            "type": "object",
            "title": "By Status",
            "description": "Task count by status (PENDING, EXECUTING, COMPLETED, FAILED, CANCELLED)"
        },
        "by_provider": {
            "additionalProperties": {
                "type": "integer"
            },
            "type": "object",
            "title": "By Provider",
            "description": "Task count by provider type"
        },
        "by_task_type": {
            "additionalProperties": {
                "type": "integer"
            },
            "type": "object",
            "title": "By Task Type",
            "description": "Task count by task type"
        },
        "tasks_last_hour": {
            "type": "integer",
            "title": "Tasks Last Hour",
            "description": "Tasks created in the last hour",
            "default": 0
        },
        "tasks_last_24h": {
            "type": "integer",
            "title": "Tasks Last 24H",
            "description": "Tasks created in the last 24 hours",
            "default": 0
        }
    },
    "type": "object",
    "required": [
        "total_tasks"
    ],
    "title": "AggregateStats",
    "description": "Aggregate statistics for the monitoring dashboard."
}

application/json

{
    "detail": [
        {
            "loc": [
                null
            ],
            "msg": "string",
            "type": "string"
        }
    ]
}

⚠️ This example has been generated automatically from the schema and it is not accurate. Refer to the schema for more information.

Schema of the response body

{
    "properties": {
        "detail": {
            "items": {
                "$ref": "#/components/schemas/ValidationError"
            },
            "type": "array",
            "title": "Detail"
        }
    },
    "type": "object",
    "title": "HTTPValidationError"
}

---

GET /api/v1/monitoring/stats/overview

Get Stats Overview

Description

Windowed dashboard metrics for tasks and workflows.

For each of last_hour / last_day / total returns counts, failure rate (kept distinct between tasks and workflows), and — for the bounded windows — completion-time, wait-time and (tasks) retry percentiles alongside averages.

Filters apply to tasks; provider/task-type filters do not apply to workflows (workflows have no provider), so the workflow side honours only origin filters.

Input parameters

Parameter	In	Type	Default	Nullable	Description
HTTPBearer	header	string	N/A	No	JWT Bearer token
origin_instance	query			No	Filter by origin instance
origin_zone	query			No	Filter by origin zone
provider_type	query			No	Filter by provider type
task_type	query			No	Filter by task type

Responses

200 OK422 Unprocessable Entity

application/json

{
    "tasks": {},
    "workflows": {},
    "filters": {},
    "timestamp": "2022-04-13T15:42:05.901Z"
}

⚠️ This example has been generated automatically from the schema and it is not accurate. Refer to the schema for more information.

Schema of the response body

{
    "properties": {
        "tasks": {
            "additionalProperties": {
                "$ref": "#/components/schemas/TaskWindowStats"
            },
            "type": "object",
            "title": "Tasks",
            "description": "Task metrics keyed by window (last_hour/last_day/total)"
        },
        "workflows": {
            "additionalProperties": {
                "$ref": "#/components/schemas/WorkflowWindowStats"
            },
            "type": "object",
            "title": "Workflows",
            "description": "Workflow metrics keyed by window"
        },
        "filters": {
            "additionalProperties": {
                "type": "string"
            },
            "type": "object",
            "title": "Filters",
            "description": "Echo of the filters applied (for display)"
        },
        "timestamp": {
            "type": "string",
            "format": "date-time",
            "title": "Timestamp",
            "description": "When the stats were collected"
        }
    },
    "type": "object",
    "required": [
        "timestamp"
    ],
    "title": "StatsOverviewResponse",
    "description": "Windowed dashboard metrics for tasks and workflows."
}

application/json

{
    "detail": [
        {
            "loc": [
                null
            ],
            "msg": "string",
            "type": "string"
        }
    ]
}

⚠️ This example has been generated automatically from the schema and it is not accurate. Refer to the schema for more information.

Schema of the response body

{
    "properties": {
        "detail": {
            "items": {
                "$ref": "#/components/schemas/ValidationError"
            },
            "type": "array",
            "title": "Detail"
        }
    },
    "type": "object",
    "title": "HTTPValidationError"
}

---

GET /api/v1/monitoring/stats/timeseries

Get Stats Timeseries

Description

Bucketed task counts (created / completed / failed) for dashboard charts.

interval=hour → hourly buckets over the last 24h; interval=day → daily buckets over the last 7 days. Bounded range + indexed created_at keep the scan cheap; cached like the overview endpoint.

Input parameters

Parameter	In	Type	Default	Nullable	Description
HTTPBearer	header	string	N/A	No	JWT Bearer token
interval	query	string	hour	No	Bucket granularity
origin_instance	query			No	Filter by origin instance
origin_zone	query			No	Filter by origin zone
provider_type	query			No	Filter by provider type
task_type	query			No	Filter by task type

Responses

200 OK422 Unprocessable Entity

application/json

{
    "interval": "hour",
    "buckets": [
        {
            "bucket": "2022-04-13T15:42:05.901Z",
            "created": 0,
            "completed": 0,
            "failed": 0
        }
    ],
    "filters": {},
    "timestamp": "2022-04-13T15:42:05.901Z"
}

⚠️ This example has been generated automatically from the schema and it is not accurate. Refer to the schema for more information.

Schema of the response body

{
    "properties": {
        "interval": {
            "type": "string",
            "enum": [
                "hour",
                "day"
            ],
            "title": "Interval",
            "description": "Bucket granularity"
        },
        "buckets": {
            "items": {
                "$ref": "#/components/schemas/TimeseriesBucket"
            },
            "type": "array",
            "title": "Buckets"
        },
        "filters": {
            "additionalProperties": {
                "type": "string"
            },
            "type": "object",
            "title": "Filters"
        },
        "timestamp": {
            "type": "string",
            "format": "date-time",
            "title": "Timestamp"
        }
    },
    "type": "object",
    "required": [
        "interval",
        "timestamp"
    ],
    "title": "TimeseriesResponse",
    "description": "Bucketed task counts over a range, for dashboard charts."
}

application/json

{
    "detail": [
        {
            "loc": [
                null
            ],
            "msg": "string",
            "type": "string"
        }
    ]
}

⚠️ This example has been generated automatically from the schema and it is not accurate. Refer to the schema for more information.

Schema of the response body

{
    "properties": {
        "detail": {
            "items": {
                "$ref": "#/components/schemas/ValidationError"
            },
            "type": "array",
            "title": "Detail"
        }
    },
    "type": "object",
    "title": "HTTPValidationError"
}

---

GET /api/v1/monitoring/workflows

List Monitored Workflows

Description

List workflows with filtering, pagination, and sorting.

Returns a paginated list of workflows with computed duration and error status. All filters are optional and can be combined.

Input parameters

Parameter	In	Type	Default	Nullable	Description
HTTPBearer	header	string	N/A	No	JWT Bearer token
created_after	query			No	Filter workflows created after this datetime
created_before	query			No	Filter workflows created before this datetime
origin_instance	query			No	Filter by origin instance
origin_tenant_id	query			No	Filter by origin tenant ID
origin_zone	query			No	Filter by origin zone
page	query	integer	1	No	Page number (1-indexed)
page_size	query	integer	50	No	Items per page
search	query			No	Search by workflow name or ID prefix
sort_by	query	string	created_at	No	Sort field
sort_order	query	string	desc	No	Sort order
status	query			No	Filter by status

Responses

200 OK422 Unprocessable Entity

application/json

{
    "items": [
        {
            "id": "c1cc51b9-5b6f-41f2-aec5-e90be77069fe",
            "name": "string",
            "description": null,
            "status": "PENDING",
            "total_steps": 0,
            "completed_steps": 0,
            "progress": 10.12,
            "current_step": null,
            "final_step_id": null,
            "origin_instance": "string",
            "origin_zone": null,
            "origin_tenant_id": null,
            "created_at": "2022-04-13T15:42:05.901Z",
            "started_at": null,
            "completed_at": null,
            "duration_ms": null,
            "has_error": true
        }
    ],
    "total": 0,
    "page": 0,
    "page_size": 0,
    "pages": 0
}

⚠️ This example has been generated automatically from the schema and it is not accurate. Refer to the schema for more information.

Schema of the response body

{
    "properties": {
        "items": {
            "items": {
                "$ref": "#/components/schemas/WorkflowListItem"
            },
            "type": "array",
            "title": "Items"
        },
        "total": {
            "type": "integer",
            "title": "Total",
            "description": "Total number of workflows matching filters"
        },
        "page": {
            "type": "integer",
            "title": "Page",
            "description": "Current page number (1-indexed)"
        },
        "page_size": {
            "type": "integer",
            "title": "Page Size",
            "description": "Number of items per page"
        },
        "pages": {
            "type": "integer",
            "title": "Pages",
            "description": "Total number of pages"
        }
    },
    "type": "object",
    "required": [
        "items",
        "total",
        "page",
        "page_size",
        "pages"
    ],
    "title": "MonitoringWorkflowListResponse",
    "description": "Paginated workflow list response for monitoring."
}

application/json

{
    "detail": [
        {
            "loc": [
                null
            ],
            "msg": "string",
            "type": "string"
        }
    ]
}

⚠️ This example has been generated automatically from the schema and it is not accurate. Refer to the schema for more information.

Schema of the response body

{
    "properties": {
        "detail": {
            "items": {
                "$ref": "#/components/schemas/ValidationError"
            },
            "type": "array",
            "title": "Detail"
        }
    },
    "type": "object",
    "title": "HTTPValidationError"
}

---

GET /api/v1/monitoring/workflows/stats

Get Workflow Monitoring Stats

Description

Get aggregate statistics for workflow monitoring.

Returns counts by status and time-based metrics.

Input parameters

Parameter	In	Type	Default	Nullable	Description
HTTPBearer	header	string	N/A	No	JWT Bearer token
origin_instance	query			No	Filter by origin instance
origin_zone	query			No	Filter by origin zone

Responses

200 OK422 Unprocessable Entity

application/json

{
    "total_workflows": 0,
    "by_status": {},
    "workflows_last_hour": 0,
    "workflows_last_24h": 0,
    "avg_steps_per_workflow": 10.12
}

⚠️ This example has been generated automatically from the schema and it is not accurate. Refer to the schema for more information.

Schema of the response body

{
    "properties": {
        "total_workflows": {
            "type": "integer",
            "title": "Total Workflows",
            "description": "Total number of workflows"
        },
        "by_status": {
            "additionalProperties": {
                "type": "integer"
            },
            "type": "object",
            "title": "By Status",
            "description": "Workflow count by status (PENDING, RUNNING, PAUSED, COMPLETED, FAILED, CANCELLED)"
        },
        "workflows_last_hour": {
            "type": "integer",
            "title": "Workflows Last Hour",
            "description": "Workflows created in the last hour",
            "default": 0
        },
        "workflows_last_24h": {
            "type": "integer",
            "title": "Workflows Last 24H",
            "description": "Workflows created in the last 24 hours",
            "default": 0
        },
        "avg_steps_per_workflow": {
            "type": "number",
            "title": "Avg Steps Per Workflow",
            "description": "Average number of steps per workflow",
            "default": 0.0
        }
    },
    "type": "object",
    "required": [
        "total_workflows"
    ],
    "title": "WorkflowAggregateStats",
    "description": "Aggregate statistics for workflow monitoring."
}

application/json

{
    "detail": [
        {
            "loc": [
                null
            ],
            "msg": "string",
            "type": "string"
        }
    ]
}

⚠️ This example has been generated automatically from the schema and it is not accurate. Refer to the schema for more information.

Schema of the response body

{
    "properties": {
        "detail": {
            "items": {
                "$ref": "#/components/schemas/ValidationError"
            },
            "type": "array",
            "title": "Detail"
        }
    },
    "type": "object",
    "title": "HTTPValidationError"
}

---

GET /api/v1/monitoring/workflows/{workflow_id}

Get Monitored Workflow

Description

Get detailed workflow information including all steps.

Steps are returned in topological order (respecting dependencies).

Input parameters

Parameter	In	Type	Default	Nullable	Description
HTTPBearer	header	string	N/A	No	JWT Bearer token
workflow_id	path	string		No

Responses

200 OK422 Unprocessable Entity

application/json

{
    "id": "33fdca2c-212f-4427-b9d5-625f61de440b",
    "name": "string",
    "description": null,
    "status": "PENDING",
    "total_steps": 0,
    "completed_steps": 0,
    "progress": 10.12,
    "current_step": null,
    "final_step_id": null,
    "canvas_task_id": null,
    "origin_instance": "string",
    "origin_zone": null,
    "origin_tenant_id": null,
    "origin_reference": null,
    "config": {},
    "webhook_url": null,
    "created_at": "2022-04-13T15:42:05.901Z",
    "updated_at": "2022-04-13T15:42:05.901Z",
    "started_at": null,
    "completed_at": null,
    "duration_ms": null,
    "error_message": null,
    "errors": [
        "string"
    ],
    "steps": [
        {
            "id": "ede9a2c0-0dad-4768-b40d-b74101d57699",
            "step_id": "string",
            "task_type": "string",
            "node_type": null,
            "node_schema_version": null,
            "status": "PENDING",
            "task_id": null,
            "dependencies": [
                "string"
            ],
            "input_mappings": [
                {}
            ],
            "config": {},
            "started_at": null,
            "completed_at": null,
            "duration_ms": null,
            "error_message": null,
            "retry_count": 0,
            "max_retries": 0,
            "result": null
        }
    ]
}

⚠️ This example has been generated automatically from the schema and it is not accurate. Refer to the schema for more information.

Schema of the response body

{
    "properties": {
        "id": {
            "type": "string",
            "format": "uuid",
            "title": "Id"
        },
        "name": {
            "type": "string",
            "title": "Name"
        },
        "description": {
            "anyOf": [
                {
                    "type": "string"
                },
                {
                    "type": "null"
                }
            ],
            "title": "Description"
        },
        "status": {
            "$ref": "#/components/schemas/WorkflowStatus"
        },
        "total_steps": {
            "type": "integer",
            "title": "Total Steps",
            "default": 0
        },
        "completed_steps": {
            "type": "integer",
            "title": "Completed Steps",
            "default": 0
        },
        "progress": {
            "type": "number",
            "title": "Progress",
            "default": 0.0
        },
        "current_step": {
            "anyOf": [
                {
                    "type": "string"
                },
                {
                    "type": "null"
                }
            ],
            "title": "Current Step"
        },
        "final_step_id": {
            "anyOf": [
                {
                    "type": "string"
                },
                {
                    "type": "null"
                }
            ],
            "title": "Final Step Id"
        },
        "canvas_task_id": {
            "anyOf": [
                {
                    "type": "string"
                },
                {
                    "type": "null"
                }
            ],
            "title": "Canvas Task Id"
        },
        "origin_instance": {
            "type": "string",
            "title": "Origin Instance"
        },
        "origin_zone": {
            "anyOf": [
                {
                    "type": "string"
                },
                {
                    "type": "null"
                }
            ],
            "title": "Origin Zone"
        },
        "origin_tenant_id": {
            "anyOf": [
                {
                    "type": "string"
                },
                {
                    "type": "null"
                }
            ],
            "title": "Origin Tenant Id"
        },
        "origin_reference": {
            "anyOf": [
                {
                    "type": "string"
                },
                {
                    "type": "null"
                }
            ],
            "title": "Origin Reference"
        },
        "config": {
            "additionalProperties": true,
            "type": "object",
            "title": "Config",
            "description": "Workflow configuration"
        },
        "webhook_url": {
            "anyOf": [
                {
                    "type": "string"
                },
                {
                    "type": "null"
                }
            ],
            "title": "Webhook Url"
        },
        "created_at": {
            "type": "string",
            "format": "date-time",
            "title": "Created At"
        },
        "updated_at": {
            "type": "string",
            "format": "date-time",
            "title": "Updated At"
        },
        "started_at": {
            "anyOf": [
                {
                    "type": "string",
                    "format": "date-time"
                },
                {
                    "type": "null"
                }
            ],
            "title": "Started At"
        },
        "completed_at": {
            "anyOf": [
                {
                    "type": "string",
                    "format": "date-time"
                },
                {
                    "type": "null"
                }
            ],
            "title": "Completed At"
        },
        "duration_ms": {
            "anyOf": [
                {
                    "type": "number"
                },
                {
                    "type": "null"
                }
            ],
            "title": "Duration Ms",
            "description": "Duration in milliseconds"
        },
        "error_message": {
            "anyOf": [
                {
                    "type": "string"
                },
                {
                    "type": "null"
                }
            ],
            "title": "Error Message"
        },
        "errors": {
            "items": {
                "type": "string"
            },
            "type": "array",
            "title": "Errors",
            "description": "List of error messages"
        },
        "steps": {
            "items": {
                "$ref": "#/components/schemas/WorkflowStepMonitor"
            },
            "type": "array",
            "title": "Steps",
            "description": "Workflow steps in execution order"
        }
    },
    "type": "object",
    "required": [
        "id",
        "name",
        "status",
        "origin_instance",
        "created_at",
        "updated_at"
    ],
    "title": "WorkflowDetailResponse",
    "description": "Detailed workflow view with all steps."
}

application/json

{
    "detail": [
        {
            "loc": [
                null
            ],
            "msg": "string",
            "type": "string"
        }
    ]
}

⚠️ This example has been generated automatically from the schema and it is not accurate. Refer to the schema for more information.

Schema of the response body

{
    "properties": {
        "detail": {
            "items": {
                "$ref": "#/components/schemas/ValidationError"
            },
            "type": "array",
            "title": "Detail"
        }
    },
    "type": "object",
    "title": "HTTPValidationError"
}

---

GET /api/v1/monitoring/queues

Get Queue Stats

Description

Get queue depths, active task counts, wait times, and throughput.

Returns statistics for all known Celery queues including: - Queue depth (pending tasks) - Active workers consuming the queue - Active and reserved tasks - Average wait time (time from created to started) - Tasks completed in the last hour

Input parameters

Parameter	In	Type	Default	Nullable	Description
HTTPBearer	header	string	N/A	No	JWT Bearer token

Responses

200 OK

application/json

{
    "queues": [
        {
            "name": "string",
            "depth": 0,
            "active_workers": 0,
            "active_tasks": 0,
            "reserved_tasks": 0,
            "wait_time": {
                "avg_ms": null,
                "p50_ms": null,
                "p95_ms": null,
                "p99_ms": null,
                "sample_size": 0
            },
            "exec_time": null,
            "tasks_completed_last_hour": 0
        }
    ],
    "total_pending": 0,
    "total_active": 0,
    "timestamp": "2022-04-13T15:42:05.901Z"
}

⚠️ This example has been generated automatically from the schema and it is not accurate. Refer to the schema for more information.

Schema of the response body

{
    "properties": {
        "queues": {
            "items": {
                "$ref": "#/components/schemas/QueueStats"
            },
            "type": "array",
            "title": "Queues",
            "description": "Statistics per queue"
        },
        "total_pending": {
            "type": "integer",
            "title": "Total Pending",
            "description": "Total pending tasks across all queues",
            "default": 0
        },
        "total_active": {
            "type": "integer",
            "title": "Total Active",
            "description": "Total active tasks across all queues",
            "default": 0
        },
        "timestamp": {
            "type": "string",
            "format": "date-time",
            "title": "Timestamp",
            "description": "Timestamp when stats were collected"
        }
    },
    "type": "object",
    "required": [
        "timestamp"
    ],
    "title": "QueueStatsResponse",
    "description": "Response for queue statistics endpoint."
}

---

GET /api/v1/monitoring/workers

Get Worker Stats

Description

Get worker utilization, capacity, and throughput.

Returns statistics for all active Celery workers including: - Worker name and queues - Concurrency and active tasks - Utilization percentage - Capacity status - Total tasks completed and throughput rate

Input parameters

Parameter	In	Type	Default	Nullable	Description
HTTPBearer	header	string	N/A	No	JWT Bearer token

Responses

200 OK

application/json

{
    "workers": [
        {
            "name": "string",
            "queues": [
                "string"
            ],
            "concurrency": 0,
            "active_tasks": 0,
            "utilization_pct": 10.12,
            "has_capacity": true,
            "tasks_completed_total": 0,
            "tasks_per_minute": 10.12,
            "memory_mb": null,
            "last_active_at": null
        }
    ],
    "total_workers": 0,
    "total_capacity": 0,
    "total_active": 0,
    "total_utilization_pct": 10.12,
    "workers_at_capacity": 0,
    "timestamp": "2022-04-13T15:42:05.901Z"
}

⚠️ This example has been generated automatically from the schema and it is not accurate. Refer to the schema for more information.

Schema of the response body

{
    "properties": {
        "workers": {
            "items": {
                "$ref": "#/components/schemas/WorkerInfo"
            },
            "type": "array",
            "title": "Workers",
            "description": "Statistics per worker"
        },
        "total_workers": {
            "type": "integer",
            "title": "Total Workers",
            "description": "Total number of workers",
            "default": 0
        },
        "total_capacity": {
            "type": "integer",
            "title": "Total Capacity",
            "description": "Total capacity (sum of all worker concurrency)",
            "default": 0
        },
        "total_active": {
            "type": "integer",
            "title": "Total Active",
            "description": "Total active tasks across all workers",
            "default": 0
        },
        "total_utilization_pct": {
            "type": "number",
            "title": "Total Utilization Pct",
            "description": "Overall utilization percentage",
            "default": 0.0
        },
        "workers_at_capacity": {
            "type": "integer",
            "title": "Workers At Capacity",
            "description": "Number of workers at full capacity",
            "default": 0
        },
        "timestamp": {
            "type": "string",
            "format": "date-time",
            "title": "Timestamp",
            "description": "Timestamp when stats were collected"
        }
    },
    "type": "object",
    "required": [
        "timestamp"
    ],
    "title": "WorkerStatsResponse",
    "description": "Response for worker statistics endpoint."
}

---

GET /api/v1/monitoring/workers/scaling

Get Scaling Stats

Description

Get CREMA autoscaling state for each worker-pool queue.

Surfaces, per queue, the live/busy instance counts, in-flight and queued task counts, per-instance concurrency, and the desired-load / desired-instance values CREMA derives from Redis — so operators can see why a pool is (or isn't) scaling without inspecting Redis directly.

Input parameters

Parameter	In	Type	Default	Nullable	Description
HTTPBearer	header	string	N/A	No	JWT Bearer token

Responses

200 OK

application/json

{
    "queues": [
        {
            "queue": "string",
            "live_instances": 0,
            "busy_instances": 0,
            "idle_instances": 0,
            "in_flight_tasks": 0,
            "queued_tasks": 0,
            "concurrency": 0,
            "desired_load": 0,
            "desired_instances": 0
        }
    ],
    "available": true,
    "timestamp": "2022-04-13T15:42:05.901Z"
}

⚠️ This example has been generated automatically from the schema and it is not accurate. Refer to the schema for more information.

Schema of the response body

{
    "properties": {
        "queues": {
            "items": {
                "$ref": "#/components/schemas/QueueScalingInfo"
            },
            "type": "array",
            "title": "Queues",
            "description": "Per-queue scaling state"
        },
        "available": {
            "type": "boolean",
            "title": "Available",
            "description": "False when scaling state could not be read from Redis (stale/unavailable)",
            "default": true
        },
        "timestamp": {
            "type": "string",
            "format": "date-time",
            "title": "Timestamp",
            "description": "Timestamp when stats were collected"
        }
    },
    "type": "object",
    "required": [
        "timestamp"
    ],
    "title": "ScalingStatsResponse",
    "description": "Response for the CREMA scaling-stats endpoint."
}

---

GET /api/v1/monitoring/workers/health

Get Workers Health

Description

Quick health check for worker availability.

Returns a simple health status indicating: - Overall health (at least one worker with capacity) - Total workers and workers with available capacity - Queues with and without workers

Use this endpoint for alerting on capacity issues.

Input parameters

Parameter	In	Type	Default	Nullable	Description
HTTPBearer	header	string	N/A	No	JWT Bearer token

Responses

200 OK

application/json

{
    "healthy": true,
    "total_workers": 0,
    "workers_with_capacity": 0,
    "queues_with_workers": [
        "string"
    ],
    "queues_without_workers": [
        "string"
    ],
    "timestamp": "2022-04-13T15:42:05.901Z"
}

⚠️ This example has been generated automatically from the schema and it is not accurate. Refer to the schema for more information.

Schema of the response body

{
    "properties": {
        "healthy": {
            "type": "boolean",
            "title": "Healthy",
            "description": "Overall health status"
        },
        "total_workers": {
            "type": "integer",
            "title": "Total Workers",
            "description": "Total number of workers",
            "default": 0
        },
        "workers_with_capacity": {
            "type": "integer",
            "title": "Workers With Capacity",
            "description": "Number of workers with available capacity",
            "default": 0
        },
        "queues_with_workers": {
            "items": {
                "type": "string"
            },
            "type": "array",
            "title": "Queues With Workers",
            "description": "Queues that have at least one worker"
        },
        "queues_without_workers": {
            "items": {
                "type": "string"
            },
            "type": "array",
            "title": "Queues Without Workers",
            "description": "Queues that have no workers"
        },
        "timestamp": {
            "type": "string",
            "format": "date-time",
            "title": "Timestamp",
            "description": "Timestamp when health was checked"
        }
    },
    "type": "object",
    "required": [
        "healthy",
        "timestamp"
    ],
    "title": "WorkerHealthResponse",
    "description": "Quick health check response for worker availability."
}

---

Schemas

AggregateStats

Name	Type	Description
by_provider		Task count by provider type
by_status		Task count by status (PENDING, EXECUTING, COMPLETED, FAILED, CANCELLED)
by_task_type		Task count by task type
tasks_last_24h	integer	Tasks created in the last 24 hours
tasks_last_hour	integer	Tasks created in the last hour
total_tasks	integer	Total number of tasks

HTTPValidationError

Name	Type	Description
detail	Array<ValidationError>

LatencyStats

Name	Type	Description
avg_ms		Mean latency (ms)
p50_ms		Median (50th percentile) latency (ms)
p95_ms		95th percentile latency (ms)
p99_ms		99th percentile latency (ms)
sample_size	integer	Number of samples the stats were computed over

MediaItem

Name	Type	Description
media_type	string	Type of media: 'image' or 'video'
name	string	Descriptive name (e.g., 'base_image', 'result_image')
path	string	Relative path in storage
url	string	Full URL for preview

MonitoringWorkflowListResponse

Name	Type	Description
items	Array<WorkflowListItem>
page	integer	Current page number (1-indexed)
page_size	integer	Number of items per page
pages	integer	Total number of pages
total	integer	Total number of workflows matching filters

ProviderType

Type: string

QueueScalingInfo

Name	Type	Description
busy_instances	integer	Live instances with at least one active task
concurrency	integer	Per-instance concurrency (CREMA's listLength divisor)
desired_instances	integer	Instances CREMA targets: ceil(desired_load / concurrency)
desired_load	integer	Published CREMA load (LLEN of crema:load:); the scaler divides this by concurrency
idle_instances	integer	Live instances with no active tasks
in_flight_tasks	integer	Sum of active tasks across live instances serving the queue
live_instances	integer	Workers currently serving this queue (online heartbeats)
queue	string	Queue / worker-pool name
queued_tasks	integer	Tasks waiting in the broker (LLEN of the queue)

QueueStats

Name	Type	Description
active_tasks	integer	Number of tasks currently being executed
active_workers	integer	Number of workers consuming this queue
depth	integer	Number of pending tasks in queue
exec_time	LatencyStats	Execution duration (started→completed), last hour
name	string	Queue name
reserved_tasks	integer	Number of tasks reserved by workers (prefetched)
tasks_completed_last_hour	integer	Number of tasks completed in the last hour
wait_time	LatencyStats	Time tasks wait in queue before execution (created→started), last hour

QueueStatsResponse

Name	Type	Description
queues	Array<QueueStats>	Statistics per queue
timestamp	string(date-time)	Timestamp when stats were collected
total_active	integer	Total active tasks across all queues
total_pending	integer	Total pending tasks across all queues

RetryStats

Name	Type	Description
avg		Mean retry count
p95		95th percentile retry count
pct_with_retries		Percent of terminal tasks that needed >=1 retry (0-100)
sample_size	integer	Terminal tasks the stats were computed over

ScalingStatsResponse

Name	Type	Description
available	boolean	False when scaling state could not be read from Redis (stale/unavailable)
queues	Array<QueueScalingInfo>	Per-queue scaling state
timestamp	string(date-time)	Timestamp when stats were collected

StatsOverviewResponse

Name	Type	Description
filters		Echo of the filters applied (for display)
tasks		Task metrics keyed by window (last_hour/last_day/total)
timestamp	string(date-time)	When the stats were collected
workflows		Workflow metrics keyed by window

StepStatus

Type: string

TaskDetailResponse

Name	Type	Description
celery_task_id
completed_at
config
created_at	string(date-time)
duration_ms		Duration in milliseconds
errors
executor_instance
executor_type
id	string(uuid)
is_resumed	boolean	Whether task was resumed
last_resume_attempt		Timestamp of last resume attempt
node_schema_version
node_type
origin_instance	string
origin_reference
origin_tenant_id
origin_zone
priority	integer	Task priority (higher = more priority)
progress	number
provider_type
queue_name	string	Celery queue name
resume_attempt_count	integer	Number of resume attempts
retry_count	integer
service_requests		Log of external service requests made during task processing
service_task_id
started_at
status	TaskStatus
task_data		Task-type specific fields (prompt, width, height, result_image_path, etc.)
task_type	TaskType
updated_at	string(date-time)
webhook_url		Webhook URL for notifications

TaskListItem

Name	Type	Description
completed_at
created_at	string(date-time)
duration_ms		Duration in milliseconds (computed from started_at to completed_at)
has_error	boolean	Whether task has any errors
id	string(uuid)
node_type
origin_instance	string
origin_tenant_id
origin_zone
progress	number
provider_type
started_at
status	TaskStatus
task_type	TaskType

TaskListResponse

Name	Type	Description
items	Array<TaskListItem>
page	integer	Current page number (1-indexed)
page_size	integer	Number of items per page
pages	integer	Total number of pages
total	integer	Total number of tasks matching filters

TaskMediaResponse

Name	Type	Description
inputs	Array<MediaItem>	Input media files
outputs	Array<MediaItem>	Output media files
task_id	string(uuid)

TaskStatus

Type: string

TaskType

Type: string

TaskWindowStats

Name	Type	Description
cancelled	integer
completed	integer
completion_time	LatencyStats	Execution duration (started→completed)
executing	integer
failed	integer
failure_rate		failed / (completed + failed), 0-1; None if no terminal tasks
pending	integer
retries	RetryStats
total	integer	Tasks created in this window
wait_time	LatencyStats	Queue wait (created→started)
window	string

TimeseriesBucket

Name	Type	Description
bucket	string(date-time)	Start of the bucket (UTC)
completed	integer
created	integer
failed	integer

TimeseriesResponse

Name	Type	Description
buckets	Array<TimeseriesBucket>
filters
interval	string	Bucket granularity
timestamp	string(date-time)

ValidationError

Name	Type	Description
loc	Array<>
msg	string
type	string

WorkerHealthResponse

Name	Type	Description
healthy	boolean	Overall health status
queues_with_workers	Array<string>	Queues that have at least one worker
queues_without_workers	Array<string>	Queues that have no workers
timestamp	string(date-time)	Timestamp when health was checked
total_workers	integer	Total number of workers
workers_with_capacity	integer	Number of workers with available capacity

WorkerInfo

Name	Type	Description
active_tasks	integer	Number of tasks currently executing
concurrency	integer	Maximum concurrent tasks
has_capacity	boolean	Whether worker has capacity for more tasks
last_active_at		When this worker last completed (or failed) a task; used as the 'last active' signal for idle workers. None if unknown.
memory_mb		Total memory usage in MB (main process + pool children)
name	string	Worker name (e.g., worker-fal@host1)
queues	Array<string>	Queues this worker consumes
tasks_completed_total	integer	Total tasks completed by this worker
tasks_per_minute	number	Average tasks completed per minute
utilization_pct	number	Utilization percentage (active_tasks / concurrency * 100)

WorkerStatsResponse

Name	Type	Description
timestamp	string(date-time)	Timestamp when stats were collected
total_active	integer	Total active tasks across all workers
total_capacity	integer	Total capacity (sum of all worker concurrency)
total_utilization_pct	number	Overall utilization percentage
total_workers	integer	Total number of workers
workers	Array<WorkerInfo>	Statistics per worker
workers_at_capacity	integer	Number of workers at full capacity

WorkflowAggregateStats

Name	Type	Description
avg_steps_per_workflow	number	Average number of steps per workflow
by_status		Workflow count by status (PENDING, RUNNING, PAUSED, COMPLETED, FAILED, CANCELLED)
total_workflows	integer	Total number of workflows
workflows_last_24h	integer	Workflows created in the last 24 hours
workflows_last_hour	integer	Workflows created in the last hour

WorkflowDetailResponse

Name	Type	Description
canvas_task_id
completed_at
completed_steps	integer
config		Workflow configuration
created_at	string(date-time)
current_step
description
duration_ms		Duration in milliseconds
error_message
errors	Array<string>	List of error messages
final_step_id
id	string(uuid)
name	string
origin_instance	string
origin_reference
origin_tenant_id
origin_zone
progress	number
started_at
status	WorkflowStatus
steps	Array<WorkflowStepMonitor>	Workflow steps in execution order
total_steps	integer
updated_at	string(date-time)
webhook_url

WorkflowListItem

Name	Type	Description
completed_at
completed_steps	integer
created_at	string(date-time)
current_step
description
duration_ms		Duration in milliseconds (computed from started_at to completed_at)
final_step_id
has_error	boolean	Whether workflow has any errors
id	string(uuid)
name	string
origin_instance	string
origin_tenant_id
origin_zone
progress	number
started_at
status	WorkflowStatus
total_steps	integer

WorkflowStatus

Type: string

WorkflowStepMonitor

Name	Type	Description
completed_at
config		Step configuration
dependencies	Array<string>	List of step IDs this step depends on
duration_ms		Duration in milliseconds
error_message
id	string(uuid)
input_mappings	Array<>	Input mappings from previous steps
max_retries	integer
node_schema_version
node_type
result		Output data from this step
retry_count	integer
started_at
status	StepStatus
step_id	string	Logical step identifier (e.g., 'generate', 'refine')
task_id		ID of the task created for this step
task_type	string	Task type for this step

WorkflowWindowStats

Name	Type	Description
completed	integer
duration	LatencyStats	Workflow duration (started→completed)
failed	integer
failure_rate		failed / (completed + failed), 0-1; None if no terminal workflows
running	integer
total	integer	Workflows created in this window
window	string

Security schemes

Name	Type	Scheme	Description
HTTPBearer	http	bearer