Discord: Automate Stat Messages - #1012
Conversation
|
Important Review skippedAuto reviews are limited based on label configuration. 🏷️ Required labels (at least one) (1)
Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds daily statistics aggregation, Discord formatting and delivery, webhook configuration, a monitored protected cron endpoint, periodic invoker wiring, and tests. ChangesDaily statistics pipeline
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant CronInvoker
participant daily_stats_cron_job
participant get_daily_stats
participant format_sections
participant post_to_discord
participant Discord
CronInvoker->>daily_stats_cron_job: GET /cron/daily-stats
daily_stats_cron_job->>get_daily_stats: retrieve statistics
get_daily_stats-->>daily_stats_cron_job: categorized statistics
daily_stats_cron_job->>format_sections: format sections
format_sections-->>daily_stats_cron_job: Markdown sections
daily_stats_cron_job->>post_to_discord: post sections
post_to_discord->>Discord: POST webhook chunks
Possibly related PRs
Suggested labels: Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
github-actions
Bot
changed the title
OpenAPI changes ⚪ No API surface changesNote This PR does not modify the API contract.
|
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
Prajna1999
changed the title
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 5
🧹 Nitpick comments (3)
backend/app/tests/services/test_stats.py (1)
134-134: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueSilence Ruff unused-argument warnings in mock helpers.
Ruff flags unused
url,timeout(andjsoninflaky_post) in the mock callables. Use*_args, **_kwargsto match therequests.postsignature without naming unused parameters.♻️ Proposed fix
- def fake_post(url, json, timeout): + def fake_post(*_args, **kwargs): - posted.append(json["content"]) + posted.append(kwargs["json"]["content"])- def flaky_post(url, json, timeout): + def flaky_post(*_args, **_kwargs): calls["n"] += 1 raise requests.ConnectionError("nope")Also applies to: 161-161
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@backend/app/tests/services/test_stats.py` at line 134, Update the mock helpers fake_post and flaky_post in the stats tests to accept unused positional and keyword arguments via *_args and **_kwargs instead of naming unused request parameters, while preserving each helper’s existing behavior.Source: Linters/SAST tools
backend/app/crud/stats.py (1)
76-88: 🔒 Security & Privacy | 🔵 Trivial | ⚡ Quick winSQL identifier interpolation via f-string is a fragile pattern.
_org_count_sqlinterpolatestabledirectly into the SQL string. All current callers pass hardcoded literals, so there is no immediate injection risk, but this pattern will silently allow SQL injection if a future caller passes dynamic input. Consider adding a whitelist assertion or a comment documenting the constraint.🛡️ Proposed guard
def _org_count_sql(table: str) -> TextClause: + # table must be a hardcoded literal — never user input + assert table.isidentifier(), f"Invalid table name: {table}" return text( f"""🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@backend/app/crud/stats.py` around lines 76 - 88, Protect the table identifier interpolated by `_org_count_sql` from future dynamic input. Add an explicit whitelist validation for the supported table names before constructing the SQL, or document and enforce that callers may only provide trusted constants; raise an appropriate error for unsupported values.backend/app/api/routes/cron.py (1)
151-163: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueUse consistent logging format.
Line 151 and 162 use f-strings while line 156 uses %-style interpolation. Prefer %-style for logging to enable lazy evaluation, or at minimum be consistent within the same function.
♻️ Suggested consistency fix (f-string → %-style)
def daily_stats_cron_job(session: SessionDep, hours: int | None = None) -> dict[str, Any]: - logger.info(f"[daily_stats_cron_job] Cron job invoked | hours={hours}") + logger.info("[daily_stats_cron_job] Cron job invoked | hours=%s", hours) try: result = collect_daily_stats(session=session, window_hours=hours) post_daily_stats_to_discord(format_daily_stats_message(result)) logger.info( "[daily_stats_cron_job] Completed | window: %s", result["window"], ) return result except Exception as e: logger.error( - f"[daily_stats_cron_job] Error executing cron job: {e}", + "[daily_stats_cron_job] Error executing cron job: %s", + e, exc_info=True, )🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@backend/app/api/routes/cron.py` around lines 151 - 163, Use consistent lazy %-style logging in the daily_stats_cron_job function: replace the f-string messages in the invocation and exception logger calls with format strings and separate arguments, matching the existing completion log.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@backend/app/api/routes/cron.py`:
- Line 150: Update the return annotation of daily_stats_cron_job from dict to
dict[str, Any], matching collect_daily_stats and the project’s requirement for
parameterized, specific return types; ensure Any is imported if needed.
In `@backend/app/crud/stats.py`:
- Line 76: Replace the Any annotations in _org_count_sql and _rows with
sqlalchemy.TextClause, importing TextClause as needed; keep the existing text()
SQL construction unchanged.
In `@backend/app/services/stats.py`:
- Around line 22-25: In the window calculation within the stats function,
replace the truthiness check on window_hours with an explicit None check so a
value of 0 produces a zero-duration window while only omitted values use
DAILY_WINDOW.
- Around line 120-131: Update _chunk_message to split any individual section
that exceeds _DISCORD_CHUNK_LIMIT into smaller newline-based chunks before or
during normal paragraph chunking, ensuring every returned chunk stays within the
limit. Preserve section content and ordering so post_daily_stats_to_discord can
send the complete digest without oversized Discord messages.
In `@scripts/python/invoke-cron.py`:
- Line 21: The ENDPOINTS list in invoke-cron.py incorrectly includes
/api/v1/cron/daily-stats in the shared five-minute loop, causing repeated daily
digests. Remove it from the shared list and invoke it through a separate daily
scheduler or guard it with a daily-only condition consistent with the configured
0 0 * * * schedule.
---
Nitpick comments:
In `@backend/app/api/routes/cron.py`:
- Around line 151-163: Use consistent lazy %-style logging in the
daily_stats_cron_job function: replace the f-string messages in the invocation
and exception logger calls with format strings and separate arguments, matching
the existing completion log.
In `@backend/app/crud/stats.py`:
- Around line 76-88: Protect the table identifier interpolated by
`_org_count_sql` from future dynamic input. Add an explicit whitelist validation
for the supported table names before constructing the SQL, or document and
enforce that callers may only provide trusted constants; raise an appropriate
error for unsupported values.
In `@backend/app/tests/services/test_stats.py`:
- Line 134: Update the mock helpers fake_post and flaky_post in the stats tests
to accept unused positional and keyword arguments via *_args and **_kwargs
instead of naming unused request parameters, while preserving each helper’s
existing behavior.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 1695d50d-6e21-4a4d-848b-95ea5e43a0ae
📒 Files selected for processing (6)
backend/app/api/routes/cron.pybackend/app/core/config.pybackend/app/crud/stats.pybackend/app/services/stats.pybackend/app/tests/services/test_stats.pyscripts/python/invoke-cron.py
| ) | ||
|
|
||
|
|
||
| def _org_count_sql(table: str) -> Any: |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Narrow Any type hints to TextClause.
_org_count_sql returns -> Any and _rows accepts stmt: Any, but both can be narrowed to sqlalchemy.TextClause since text() always returns that type. As per coding guidelines, -> Any is not acceptable unless the type cannot be narrowed.
♻️ Proposed fix
-from sqlalchemy import text
+from sqlalchemy import TextClause, text
-def _org_count_sql(table: str) -> Any:
+def _org_count_sql(table: str) -> TextClause:
-def _rows(session: Session, stmt: Any, params: dict[str, Any]) -> list[dict[str, Any]]:
+def _rows(session: Session, stmt: TextClause, params: dict[str, Any]) -> list[dict[str, Any]]:Also applies to: 97-97
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@backend/app/crud/stats.py` at line 76, Replace the Any annotations in
_org_count_sql and _rows with sqlalchemy.TextClause, importing TextClause as
needed; keep the existing text() SQL construction unchanged.
Source: Coding guidelines
| ) | ||
|
|
||
|
|
||
| def _org_count_sql(table: str) -> Any: |
There was a problem hiding this comment.
I think we can avoid the any type here. This is good suggestion given by coderabbit https://github.com/ProjectTech4DevAI/kaapi-backend/pull/1012/changes#r3558059640. Please handle it.
| def section_counts(result: dict[str, Any]) -> dict[str, int]: | ||
| return { | ||
| section: len(rows) | ||
| for section, rows in result["stats"].items() | ||
| if isinstance(rows, list) | ||
| } |
There was a problem hiding this comment.
this function is not used anywhere, if this not needed please remove this dead code.
|
|
||
| logger = logging.getLogger(__name__) | ||
|
|
||
| DAILY_WINDOW = timedelta(hours=168) |
There was a problem hiding this comment.
this timedelta(hours=168) is 7 days, not daily. as I can see this cron(here) run daily but posts a rolling 7-day window every run → ~85% of each day's report overlaps yesterday's. Intended? If yes, rename to DEFAULT_STATS_WINDOW / WEEKLY_WINDOW so the daily name doesn't mislead. If the intent was true daily deltas, then I think window should be 24h.
Ayush8923
added
reviewed
and removed
enhancement
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@backend/app/api/routes/cron.py`:
- Around line 148-151: Restore the optional hours override across the
daily-stats flow: in backend/app/api/routes/cron.py lines 148-151, validate the
hours query parameter and pass it through daily_stats_cron_job to
get_daily_stats and the CRUD boundary; in backend/app/crud/stats.py lines
10-132, use named window parameters while preserving the default 168-hour
behavior and omit *_7d fields for other windows; in
backend/app/services/stats.py lines 67-74, format labels using the selected
window, defaulting to 7d.
In `@backend/app/services/stats.py`:
- Around line 77-81: Update _post to call raise_for_status() on the
requests.post response, catch failures, and log with the [_post] prefix while
recording only the exception type rather than interpolating the exception value.
Update the successful fake response mock to implement raise_for_status(), and
add coverage confirming non-success responses are handled as failures.
In `@backend/app/tests/services/test_stats.py`:
- Around line 9-67: Update _sample_stats to return a concrete mapping type
matching the fixture structure, and add explicit parameter and return
annotations to each test function in this diff. Annotate fake_post’s parameters
and return value as well, using leading-underscore names for arguments whose
values are intentionally ignored.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: 8d9913fd-78fe-4981-b9fe-7468ece8d36c
📒 Files selected for processing (5)
backend/app/api/routes/cron.pybackend/app/core/config.pybackend/app/crud/stats.pybackend/app/services/stats.pybackend/app/tests/services/test_stats.py
🚧 Files skipped from review as they are similar to previous changes (1)
- backend/app/core/config.py
| def _sample_stats() -> dict: | ||
| return { | ||
| "LLM Calls": [ | ||
| { | ||
| "organization": "Acme", | ||
| "project": "Alpha", | ||
| "calls_24h": 3, | ||
| "calls_7d": 15, | ||
| }, | ||
| ], | ||
| "STT Results": [], | ||
| } | ||
|
|
||
|
|
||
| def test_format_sections_renders_bold_title_and_aligned_table(): | ||
| sections = format_sections(_sample_stats()) | ||
| llm_section = next(s for s in sections if s.startswith("**LLM Calls**")) | ||
| assert "organization project calls_24h calls_7d" in llm_section | ||
| assert "Acme Alpha 3 15" in llm_section | ||
| assert llm_section.count("```") == 2 # wrapped in one code block | ||
|
|
||
|
|
||
| def test_format_sections_marks_empty_sections(): | ||
| sections = format_sections(_sample_stats()) | ||
| stt_section = next(s for s in sections if s.startswith("**STT Results**")) | ||
| assert stt_section == "**STT Results**\n_no data_" | ||
|
|
||
|
|
||
| def test_post_to_discord_noop_when_webhook_unset(): | ||
| with patch.object(stats_mod.settings, "DISCORD_STATS_WEBHOOK_URL", None), patch( | ||
| "app.services.stats.requests.post" | ||
| ) as mock_post: | ||
| post_to_discord(["anything"]) | ||
| mock_post.assert_not_called() | ||
|
|
||
|
|
||
| def test_post_to_discord_packs_sections_under_size_limit(): | ||
| posted: list[str] = [] | ||
|
|
||
| def fake_post(url, json, timeout): | ||
| posted.append(json["content"]) | ||
|
|
||
| big_sections = ["x" * 1000 for _ in range(4)] | ||
| with patch.object( | ||
| stats_mod.settings, "DISCORD_STATS_WEBHOOK_URL", "https://x/hook" | ||
| ), patch("app.services.stats.requests.post", side_effect=fake_post): | ||
| post_to_discord(big_sections) | ||
| assert len(posted) >= 2 # split into multiple messages | ||
| assert all(len(content) <= 2000 for content in posted) | ||
|
|
||
|
|
||
| def test_post_to_discord_swallows_request_exception(): | ||
| with patch.object( | ||
| stats_mod.settings, "DISCORD_STATS_WEBHOOK_URL", "https://x/hook" | ||
| ), patch( | ||
| "app.services.stats.requests.post", | ||
| side_effect=requests.ConnectionError("boom"), | ||
| ): | ||
| post_to_discord(["hello"]) # must not raise |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
rg -n '^\s*def ' backend/app/tests/services/test_stats.pyRepository: ProjectTech4DevAI/kaapi-backend
Length of output: 538
🏁 Script executed:
#!/bin/bash
set -euo pipefail
python3 - <<'PY'
from pathlib import Path
import ast
path = Path("backend/app/tests/services/test_stats.py")
src = path.read_text()
tree = ast.parse(src)
issues = []
for i, node in enumerate(tree.body, 1):
if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)):
if node.name == "fake_post":
return_annot = ast.unparse(node.returns) if node.returns else "<missing>"
args = [(arg.arg, ast.unparse(arg.annotation) if arg.annotation else "<missing>") for arg in node.args.args]
issues.append(f"fake_post: return={return_annot}, args={args}")
elif node.name == "_sample_stats":
if node.returns and isinstance(node.returns, ast.Name) and node.returns.id == "dict":
issues.append(f"{node.name}: generic return value dict instead of concrete type")
else:
issues.append(f"{node.name}: no return-annotation issue")
else:
if not node.returns:
issues.append(f"{node.name}: no return annotation")
print("\n".join(issues))
PYRepository: ProjectTech4DevAI/kaapi-backend
Length of output: 591
Add narrow annotations to the test helper and test functions.
_sample_stats uses the generic dict return type. All test functions and the nested fake_post stub lack parameter and return annotations. Use a concrete fixture mapping type and annotate ignored stub arguments with leading underscores.
🧰 Tools
🪛 ast-grep (0.45.0)
[info] 37-39: no timeout was given on call to external resource
Context: patch(
"app.services.stats.requests.post"
)
Note: [CWE-1088] Synchronous Access of Remote Resource without Timeout.
(requests-timeout)
[info] 53-53: no timeout was given on call to external resource
Context: patch("app.services.stats.requests.post", side_effect=fake_post)
Note: [CWE-1088] Synchronous Access of Remote Resource without Timeout.
(requests-timeout)
[info] 62-65: no timeout was given on call to external resource
Context: patch(
"app.services.stats.requests.post",
side_effect=requests.ConnectionError("boom"),
)
Note: [CWE-1088] Synchronous Access of Remote Resource without Timeout.
(requests-timeout)
🪛 Ruff (0.16.0)
[warning] 48-48: Unused function argument: url
(ARG001)
[warning] 48-48: Unused function argument: timeout
(ARG001)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@backend/app/tests/services/test_stats.py` around lines 9 - 67, Update
_sample_stats to return a concrete mapping type matching the fixture structure,
and add explicit parameter and return annotations to each test function in this
diff. Annotate fake_post’s parameters and return value as well, using
leading-underscore names for arguments whose values are intentionally ignored.
Sources: Coding guidelines, Linters/SAST tools
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (2)
.claude/agents/senior-engineer.md (1)
49-50: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winDo not require loop unrolling.
Loop unrolling duplicates statements and fixes behavior to a specific item count. It reduces maintainability when the item count changes.
Replace “unroll loops” with “prefer explicit multi-line loops when they improve clarity.” Also change “loops that involves” to “loops that involve.”
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.claude/agents/senior-engineer.md around lines 49 - 50, Update the Coding Style guidance in the senior-engineer instructions to remove the requirement to unroll loops and instead prefer explicit multi-line loops only when they improve clarity; also correct “loops that involves” to “loops that involve,” while preserving the existing guidance against nested or complex one-liner loops.backend/app/tests/services/test_stats.py (1)
68-74: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick winAssert that packing retains every section.
The test checks message count and size only. An implementation that drops a section can still pass. Use unique section values and assert that every input section appears in the combined posted content.
Proposed test change
-big_sections = ["x" * 1000 for _ in range(4)] +big_sections = [f"section-{section_index}: " + "x" * 990 for section_index in range(4)] ... assert len(posted) >= 2 assert all(len(content) <= 2000 for content in posted) +combined_content = "\n".join(posted) +assert all(section in combined_content for section in big_sections)🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@backend/app/tests/services/test_stats.py` around lines 68 - 74, Update the test around post_to_discord to use distinct section values and verify that the combined content of all entries in posted contains every input section, while retaining the existing message-count and 2000-character size assertions.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@backend/app/tests/api/routes/test_cron.py`:
- Around line 277-298: Add an optional, validated hours query parameter to
daily_stats_cron_job, propagate it through the daily-stats service into
get_daily_stats and the CRUD layer, and update those contracts to apply the
requested reporting window while preserving the default behavior. Extend
test_daily_stats_cron_job_success with a /cron/daily-stats?hours=48 request and
assert the value is forwarded through the call chain.
In `@backend/app/tests/crud/test_stats.py`:
- Line 17: Add the narrow return annotation -> None to the
test_get_daily_stats_runs_every_section_and_maps_rows function definition,
leaving its existing test behavior unchanged.
---
Nitpick comments:
In @.claude/agents/senior-engineer.md:
- Around line 49-50: Update the Coding Style guidance in the senior-engineer
instructions to remove the requirement to unroll loops and instead prefer
explicit multi-line loops only when they improve clarity; also correct “loops
that involves” to “loops that involve,” while preserving the existing guidance
against nested or complex one-liner loops.
In `@backend/app/tests/services/test_stats.py`:
- Around line 68-74: Update the test around post_to_discord to use distinct
section values and verify that the combined content of all entries in posted
contains every input section, while retaining the existing message-count and
2000-character size assertions.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: c73471ae-3954-4aaf-b5ad-83be1e5ff44a
📒 Files selected for processing (5)
.claude/agents/senior-engineer.mdbackend/app/services/stats.pybackend/app/tests/api/routes/test_cron.pybackend/app/tests/crud/test_stats.pybackend/app/tests/services/test_stats.py
🚧 Files skipped from review as they are similar to previous changes (1)
- backend/app/services/stats.py
|
@Prajna1999 what does the UI look like for the message being sent to Disocrd? Is this latest UI? |
There was a problem hiding this comment.
@Prajna1999 this UI here feels a bit unstructured. can we update it and structure it more like this so that everything is presented more clearly?
There was a problem hiding this comment.
structured as in not having multiple sub-headings and sort in descending order like above ^^? Or only two columns in one go? Since we have to show stats grouped by Org/Project having less columns and make it concise we will lose information.
There was a problem hiding this comment.
in this, i am thinking that right now, all the stats are being shown in a single message, which doesn’t feel like the best or most organized way to present them. Instead, i am thinking we could trigger separate messages for each stat category in different message blocks. If any stat count is 0, we can create a separate message to clearly indicate that as well.
- No activity this week
• Evaluation runs
• STT results
• TTS results
• Assessments
There was a problem hiding this comment.
One combined message gets the job done. trigerring 5 messages would be an overkill with no added benefit. Anyway, the stats messages would be rarely out of order to be significant, and most of the time background noise.
| o.name AS organization, | ||
| p.name AS project, | ||
| j.job_type AS job_type, | ||
| COUNT(*) FILTER (WHERE j.inserted_at >= now() - INTERVAL '24 hours') AS jobs_24h, |
There was a problem hiding this comment.
can we rename column to last 24hrs & last 7days in discord message
| ORDER BY o.name, p.name, j.job_type | ||
| """ | ||
|
|
||
| EVALUATION_RUNS = """ |
There was a problem hiding this comment.
most of the query is same and only thing that is different is the table name so can we instead use a function
def _count_query(table: str) -> str:
return f"""
SELECT
o.name AS organization,
p.name AS project,
COUNT(*) FILTER (WHERE t.inserted_at >= now() - INTERVAL '24 hours') AS count_24h,
COUNT(*) AS count_7d
FROM {table} t
INNER JOIN organization o ON t.organization_id = o.id
INNER JOIN project p ON t.project_id = p.id
WHERE t.inserted_at >= now() - INTERVAL '168 hours'
GROUP BY o.name, p.name
ORDER BY count_7d DESC
"""
EVALUATION_RUNS = _count_query("evaluation_run")
STT_RESULTS = _count_query("stt_result")
TTS_RESULTS = _count_query("tts_result")
ASSESSMENTS = _count_query("assessment")
There was a problem hiding this comment.
Yes the abstracted manner as above ^^ was earlier implemented. But overrode in support of straight forward queries and not having utils and make it simpler to read as it's just an SQL query.
There was a problem hiding this comment.
i understand the reasoning around keeping the SQL straightforward and avoiding unnecessary abstractions. however, in this case, the queries have the exact same structure and logic, with only the table name changing. Duplicating the same query for each table would go against the DRY principle.
There was a problem hiding this comment.
i would suggest keeping a small reusable function like the one above and passing the table name as an argument. Since the abstraction is very minimal here, I don’t think it hurts readability or maintainability; instead, it keeps a single source of truth for this query logic.
There was a problem hiding this comment.
@Prajna1999 we should use functions when having such duplication in code. This way is more cleaner since queries are exactly the same
| def get_daily_stats(*, session: Session) -> dict[str, list[dict[str, Any]]]: | ||
| stats: dict[str, list[dict[str, Any]]] = {} | ||
| stats["LLM Calls"] = _rows(session, LLM_CALLS) | ||
| stats["LLM Tokens"] = _rows(session, LLM_TOKENS) | ||
| stats["LLM Modality"] = _rows(session, LLM_MODALITY) | ||
| stats["Jobs by Type"] = _rows(session, JOBS) | ||
| stats["Evaluation Runs"] = _rows(session, EVALUATION_RUNS) | ||
| stats["STT Results"] = _rows(session, STT_RESULTS) | ||
| stats["TTS Results"] = _rows(session, TTS_RESULTS) | ||
| stats["Assessments"] = _rows(session, ASSESSMENTS) | ||
| return stats |
There was a problem hiding this comment.
for cleaner code
SECTIONS = {
"LLM Calls": LLM_CALLS,
"LLM Tokens": LLM_TOKENS,
"LLM Modality": LLM_MODALITY,
"Jobs by Type": JOBS,
"Evaluation Runs": EVALUATION_RUNS,
"STT Results": STT_RESULTS,
"TTS Results": TTS_RESULTS,
"Assessments": ASSESSMENTS,
}
def get_daily_stats(*, session: Session) -> dict[str, list[dict[str, Any]]]:
return {name: _rows(session, sql) for name, sql in SECTIONS.items()}
```
There was a problem hiding this comment.
If this comment hasn’t been addressed yet, please address this one as well.
There was a problem hiding this comment.
In this, these 4 query are duplicate just only the table name change, so I think we can optimize this more:
EVALUATION_RUNSSTT_RESULTSTTS_RESULTSASSESSMENT
like as:
_SIMPLE_COUNT = """
SELECT
o.name AS organization,
p.name AS project,
COUNT(*) FILTER (WHERE t.inserted_at >= now() - INTERVAL '24 hours') AS count_24h,
COUNT(*) AS count_7d
FROM {table} t
INNER JOIN organization o ON t.organization_id = o.id
INNER JOIN project p ON t.project_id = p.id
WHERE t.inserted_at >= now() - INTERVAL '168 hours'
GROUP BY o.name, p.name
ORDER BY count_7d DESC
"""
then create the one mapping object which is associated with their table name:
SIMPLE_COUNT_TABLES = {
"Evaluation Runs": "evaluation_run",
"STT Results": "stt_result",
"TTS Results": "tts_result",
"Assessments": "assessment",
}
then create the one private function,
def _simple_count_sql(table: str) -> str:
if not _IDENTIFIER.match(table):
raise ValueError(f"unsafe table identifier: {table!r}")
return _SIMPLE_COUNT.format(table=table)
and the aggregator, to fetch the count corresponding to the table:
def get_daily_stats(*, session: Session) -> dict[str, list[dict[str, Any]]]:
stats: dict[str, list[dict[str, Any]]] = {
"LLM Calls": _rows(session, LLM_CALLS),
"LLM Tokens": _rows(session, LLM_TOKENS),
"LLM Modality": _rows(session, LLM_MODALITY),
"Jobs by Type": _rows(session, JOBS),
}
for label, table in SIMPLE_COUNT_TABLES.items():
stats[label] = _rows(session, _simple_count_sql(table))
return stats
There was a problem hiding this comment.
This comment is addressed here #1012 (comment)
There was a problem hiding this comment.
in this, i am thinking that right now, all the stats are being shown in a single message, which doesn’t feel like the best or most organized way to present them. Instead, i am thinking we could trigger separate messages for each stat category in different message blocks. If any stat count is 0, we can create a separate message to clearly indicate that as well.
- No activity this week
• Evaluation runs
• STT results
• TTS results
• Assessments
| _post(str(url), embed) | ||
|
|
||
|
|
||
| def _new_embed(title: str, description: str) -> dict[str, Any]: |
There was a problem hiding this comment.
Type safety check, try to avoid the any.
There was a problem hiding this comment.
It's alright. It's a simple util for markdown arrangement.
| } | ||
|
|
||
|
|
||
| def _embed_size(embed: dict[str, Any]) -> int: |
| ) | ||
|
|
||
|
|
||
| def _post(url: str, embed: dict[str, Any]) -> None: |
There was a problem hiding this comment.
fix this in all places where this any type used.
Co-authored-by: Ayush <80516839+Ayush8923@users.noreply.github.com>
Co-authored-by: Ayush <80516839+Ayush8923@users.noreply.github.com>
…oup empty sections Rebuilds the stats formatter to match the requested Discord embed design (bold group headings, monospace tables with proper column spacing, full model names), replaces dict[str, Any] with typed StatRow/EmbedField/DiscordEmbed shapes, fixes stale constant references, and collapses inactive sections into a single "No activity this week" field instead of one per empty section. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Ayush8923
left a comment
Ayush8923
left a comment
There was a problem hiding this comment.
approving this PR. However, please make sure all the review comments are addressed and resolved before merging, so that we all on the same page regarding the reviewers feedback.
| monitor_slug="daily-stats-cron-job", | ||
| monitor_config=DAILY_STATS_CRON_MONITOR_CONFIG, | ||
| ) | ||
| def daily_stats_cron_job(session: SessionDep) -> dict[str, Any]: |
There was a problem hiding this comment.
avoid to use the Any type.
There was a problem hiding this comment.
and please update this all the places.
| """ | ||
|
|
||
|
|
||
| def _rows(session: Session, sql: str) -> list[dict[str, Any]]: |
There was a problem hiding this comment.
Avoid to use the Any type
3 participants
Issue
Closes #825
Summary
Automates a daily platform stats digest to a Discord channel via webhook. Adds /cron/daily-stats that runs a set of per-organization SQL rollups (LLM call counts, total tokens by model, modality mix, job types, evaluation runs, STT/TTS results, assessments), formats them as compact plain-text tables, chunks past Discord's 2000-char limit, and posts fire-and-forget so a webhook outage never breaks the cron.
Query/format/post logic lives in
app/services/stats.py;the route is a thin orchestrator. Window size is hardcoded to 7day and 24 hours. Every morning 9 AM, the cron job is triggered that executes the count queriesChecklist
Before submitting a pull request, please ensure that you mark these task.
fastapi run --reload app/main.pyordocker compose upin the repository root and test.Summary by CodeRabbit
New Features
Tests