Skip to content

Evaluation: Add dataset filter option #1137

Description

@AkhileshNegi

Is your feature request related to a problem?
The GET /api/v1/evaluations endpoint currently returns all evaluation runs in a project without the ability to filter by a specific dataset. This requires clients to fetch and filter results client-side, leading to unreliable data viewing due to potential issues with pagination.

Describe the solution you'd like

  • Add an optional dataset_id query parameter to GET /api/v1/evaluations.
  • Filter runs in SQL through list_evaluation_runs in app/crud/evaluations/core.py before applying ordering/limit/offset.
  • Maintain compatibility with the existing type filter and pagination.
  • If the parameter is omitted, retain the current behavior of showing runs across all datasets in the project.
  • Ensure dataset_id from another project returns no runs.
  • Update app/api/docs/evaluation/list_evaluations.md to include the new parameter documentation.
Original issue

Is your feature request related to a problem?

GET /api/v1/evaluations lists every evaluation run in a project with no way to scope the result to a single evaluation dataset. A client that only cares about one dataset's runs has to fetch runs and filter them client-side, which breaks against limit/offset — the runs it wants may not be on the page it fetched. Any per-dataset view (e.g. "runs for this dataset") is therefore unreliable today.

Describe the solution you'd like

  • Add an optional dataset_id query param to GET /api/v1/evaluations.
  • Thread dataset_id through list_evaluation_runs in app/crud/evaluations/core.py so filtering happens in SQL, applied before ordering/limit/offset.
  • Compose with the existing type filter and pagination rather than replacing them.
  • Omitting the param keeps current behavior: runs across all datasets in the project.
  • Keep results org/project-scoped — a dataset_id belonging to another project returns no runs.
  • Update app/api/docs/evaluation/list_evaluations.md to document the param.

Comments: Implemented in #1136

Original issue

Backend: GET /api/v1/evaluations (list evaluation runs) has no way to scope results to a single evaluation dataset. A client that wants only one dataset's runs must page through every run in the project and filter client-side, which breaks against limit/offset — the runs they want may not be on the page fetched. Ask: add an optional dataset_id query param to the list endpoint (and the underlying crud list_evaluation_runs), composing with the existing type filter and pagination. Repo: ProjectTech4DevAI/kaapi-backend. Related PR: #1136

Metadata

Metadata

Assignees

Type

No type

Projects

Status
Closed

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions