diff --git a/TRACKER.md b/TRACKER.md index 640cb43..8c7b05a 100644 --- a/TRACKER.md +++ b/TRACKER.md @@ -4,9 +4,9 @@ ## Status -- **Current phase:** 6 — Lifecycle, scale, hardening **COMPLETE** (Gate 6 passed, M6 reached). 6F field-hardening done bar the data-blocked 6F.6 detection half. -- **Next step:** publish 0.5.0 to npm (needs the user's creds — `npm publish`), then the tester re-validates via `ui-lineage-mcp`. After that, Phase 7 (backend parsers & federation) — detail the sketch after v1 feedback. 6F.6 detection half still blocked on a real failing test file from the tester. -- **Done:** 0.1–0.4, 1.1–1.6, 2.1–2.5, 3.1–3.6, 4.1–4.6, 5.1–5.7, 6F.1–6F.5, 6F.7–6F.10 + 6F.6 (defensive half), **6.1–6.5 (Phase 6 complete)** · **0.5.0 prepared** (all packages + generator/CLI/MCP version strings bumped, README changelog; `npm pack` verified; publish pending user's npm creds) (all packages bumped, changelog + version strings, `npm pack` → ui-lineage-0.4.1.tgz verified: both bins, no @coderadar leak; publish to npm pending). 0.4.0 tagged/merged earlier. **Self-validated on Grafana frontend** (6,461 files, 15,334 nodes / 18,367 edges in 72 s): 55 RTK-query data sources, 32 routes, 1,009 coverage edges, gibberish declines — all previously 0/1 in the field run. +- **Current phase:** 7 — Backend parsers & federation (v2 horizon). Phase 6 complete (Gate 6, M6). **0.5.0 published to npm.** +- **Next step:** publish 0.6.0 to npm (needs the user's creds — `npm publish`), then the tester validates the wider data-source coverage. After that, the backend-parser steps (7.4 Python, 7.5 Go, 7.6 federation) — larger, new-package work, still sketch-level; best held until that feedback confirms priority. 6F.6 detection half still blocked on a real failing test file from the tester. +- **Done:** 0.1–0.4, 1.1–1.6, 2.1–2.5, 3.1–3.6, 4.1–4.6, 5.1–5.7, 6F.1–6F.5, 6F.7–6F.10 + 6F.6 (defensive half), **6.1–6.5 (Phase 6 complete)**, **7.1 (GraphQL adapter), 7.2 (Next.js server data), 7.3 (push channels)** · **0.5.0 released & published to npm** (tag v0.5.0); **0.6.0 prepared** (7.1–7.3; all packages + generator/CLI/MCP version strings bumped, README changelog; `npm pack` verified; publish pending user's npm creds). **Self-validated on Grafana frontend** (6,461 files, 15,334 nodes / 18,367 edges in 72 s): 55 RTK-query data sources, 32 routes, 1,009 coverage edges, gibberish declines — all previously 0/1 in the field run. - **Gates passed:** Gate 0 (CI + red-path, #5/#6) · Gate 1 (precision 1.000, recall 0.895, zero poison) · Gate 2 (C1 instance attribution 1.000 · B1 4-level handler chains · C6 store writers↔readers · A9 portals — scorecard 137/0/0, precision & recall 1.000) · Gate 3 (B3 action effects · B4 routers · B6 cyclic journeys terminate · B7/B8 form & non-JSX events · G5 flag/role conditions — precision & recall 1.000) · Gate 4 (A4 rarity · A10 fuzzy/OCR · A1 structural · A6 subtree · E3 vision annotations · E2 aliases · G4 corrections — high-conf correct 1.000, ambiguity honesty 1.000, poison rate 0.000) · Gate 5 (F1 context bundle · F2 blast radius · F3 test coverage · F4 response schema · F5 git history · MCP server over stdio — scorecard 265/0/0, all honesty metrics 1.000; **M5 reached** — ticket in → budgeted context bundle out, over MCP) ## What CodeRadar is @@ -705,11 +705,23 @@ formally stamped: 6F.6 test-coverage hardening (blocked on a real failing sample ## Phase 7 — Backend parsers & federation (v2 horizon) -Sketch level — detail before starting the phase, after v1 feedback. - -- **7.1 GraphQL adapter** (C4): operations/fragments as data sources; codegen types feed 5.5. -- **7.2 Next.js server data** (C9): RSC async components, `getServerSideProps`, server actions as first-class data sources. -- **7.3 Push channels** (C8): WebSocket/SSE subscription nodes. +Sketch level for 7.2–7.6 — detail each before starting, after v1 feedback. + +### [x] 7.1 GraphQL adapter +**Failure modes:** C4 +**Build:** recognize GraphQL operations a component runs (Apollo/urql/graphql-request `useQuery`/`useMutation`/`useSubscription`/`useLazyQuery` with a `gql`/`graphql` tagged-template argument, inline or a resolved const) and emit them as data sources: operation NAME as identity, operation TYPE as method. +**Accept:** fixture with query/mutation/subscription operations (imported const + inline tag) → one `graphql` data source each with the right method and a `fetches-from` edge; react-query `useQuery`/`useMutation` (non-gql arg) still classify as react-query. +**Done:** new `graphql.ts` — `parseGraphqlOperation` reads the operation type/name (+ root selection fields as an anonymous-op fallback) from a gql document, ignoring `#` comments and handling shorthand `{ … }`; `graphqlOperationFromArg` resolves a hook's first argument to an operation (inline tagged template, or an identifier followed via `getDefinitionNodes()` so an imported/co-located gql const resolves cross-file). `detectDataSource` gains a GraphQL branch **before** the react-query branch (Apollo/urql reuse `useQuery`/`useMutation`, so it only claims the call when the argument is an actual gql document, else falls through). Emits `sourceKind: "graphql"`, `method` = query/mutation/subscription, `endpoint` = operation name (the value federation/attribution join on) — reusing the existing data-source node + `fetches-from` wiring, no schema change (`graphql` was already a `DataSourceKind`). New fixture `c4-graphql` (imported-const `useQuery`/`useMutation` + inline `useSubscription`); 9 unit tests incl. the react-query disambiguation on the c5 fixture. eval 326/0/0/0, determinism 1.000, all metrics 1.000; typecheck + lint clean. **Codegen response types (feeding 5.5) deferred** — needs a `.graphql`/codegen fixture; the operation→data-source spine is what unblocks federation. +### [x] 7.2 Next.js server data +**Failure modes:** C9 +**Build:** attribute server-side data fetching to the page it feeds — async RSC server components (fetch in their own body) and pages-router `getServerSideProps`/`getStaticProps`/`getStaticPaths` (fetch in a separate exported function). +**Accept:** fixture with an async RSC page, a `getServerSideProps` page, and a `getStaticProps` page → each page attributes its server-side endpoint; the data functions produce no component/hook nodes. +**Done:** RSC async server components already worked — the async component is recognized and its inline `await fetch(...)` is caught by the body walk (locked with a test). The gap was pages-router data functions: `getServerSideProps`/`getStaticProps`/`getStaticPaths` are top-level exported functions, not components/hooks, so their fetches were never scanned. The main loop now collects those functions per file and, once the file's default-export page component is known, runs the data-source extraction on their bodies attributed to that page (`fetches-from`). New fixture `c9-nextjs-server-data` (async RSC + getServerSideProps + getStaticProps); 4 unit tests. eval 341/0/0/0, determinism 1.000, all metrics 1.000; parser-react 168 tests, typecheck + lint clean. **Server actions deferred** (`"use server"` mutations bound to `
` — a mutation/event surface, fuzzier than the data-in path; revisit with feedback). +### [x] 7.3 Push channels +**Failure modes:** C8 +**Build:** recognize server-push channels (`new WebSocket(url)`, `new EventSource(url)`) as data sources — long-lived connections, not request/response fetches — with a `fetches-from` edge from the component that opens them. +**Accept:** fixture with a WebSocket and an SSE channel → one `websocket` / one `sse` data source with the URL as endpoint and a `fetches-from` edge each; no stray sources from unrelated constructors. +**Done:** new `sse` `DataSourceKind` (schema regenerated, drift gate green); new `pushchannels.ts` — `detectPushChannel` matches a `new WebSocket(...)` / `new EventSource(...)` expression (by constructor name, allowing a `window.`/`self.` qualifier — a very low false-positive set) and returns a data source with the URL resolved through the existing `resolveEndpoint` (constant-folded, `:param`-normalized). `extractBodyFacts` gains a `NewExpression` pass that emits the node + `fetches-from` edge, mirroring the fetch path (skipped inside API wrappers). `websocket` for WebSocket, `sse` for EventSource, transport in `method` (`WS`/`SSE`). New fixture `c8-push-channels` (WebSocket `wss://…` + SSE `/api/…/stream`); 4 unit tests. eval 332/0/0/0, determinism 1.000, all metrics 1.000; parser-react 164 tests, typecheck + lint clean. **socket.io deferred** (`io()` needs import-aware detection to avoid false positives on the common `io` identifier). - **7.4 Python backend parser** (C10, G6): FastAPI/Django route decorators → `ServesNode{ method, pattern }`; tree-sitter based, emits the same LineageGraph JSON. - **7.5 Go backend parser**: mux/gin/echo handler registration → `ServesNode`. - **7.6 Federation** (G6): endpoint-pattern join across graphs (`fetches-from: /api/users` ↔ `serves: /api/users`); multi-graph loader in agent-sdk; bundle lineage extends to the owning service + handler file. diff --git a/eval/fixtures/c4-graphql/app/InviteForm.tsx b/eval/fixtures/c4-graphql/app/InviteForm.tsx new file mode 100644 index 0000000..3960d33 --- /dev/null +++ b/eval/fixtures/c4-graphql/app/InviteForm.tsx @@ -0,0 +1,16 @@ +import { useMutation } from "@apollo/client"; + +import { CREATE_USER } from "./queries"; + +// Apollo useMutation with an imported gql const → graphql data source +// "CreateUser" (method: mutation). Shares the useMutation name with react-query, +// but the gql argument makes it a GraphQL source, not a react-query one. +export function InviteForm() { + const [createUser] = useMutation(CREATE_USER); + return ( + createUser({ variables: { input: {} } })}> +

Invite teammate

+ +
+ ); +} diff --git a/eval/fixtures/c4-graphql/app/LiveTicker.tsx b/eval/fixtures/c4-graphql/app/LiveTicker.tsx new file mode 100644 index 0000000..29b79e5 --- /dev/null +++ b/eval/fixtures/c4-graphql/app/LiveTicker.tsx @@ -0,0 +1,21 @@ +import { gql, useSubscription } from "@apollo/client"; + +// Inline gql tagged template (not a const) with a subscription operation → +// graphql data source "OnTick" (method: subscription). +export function LiveTicker() { + const { data } = useSubscription( + gql` + subscription OnTick { + tick { + at + } + } + `, + ); + return ( +
+

Live updates

+ {data?.tick?.at ?? "waiting"} +
+ ); +} diff --git a/eval/fixtures/c4-graphql/app/UsersPanel.tsx b/eval/fixtures/c4-graphql/app/UsersPanel.tsx new file mode 100644 index 0000000..a4cdc20 --- /dev/null +++ b/eval/fixtures/c4-graphql/app/UsersPanel.tsx @@ -0,0 +1,14 @@ +import { useQuery } from "@apollo/client"; + +import { GET_USERS } from "./queries"; + +// Apollo useQuery with an imported gql const → graphql data source "GetUsers". +export function UsersPanel() { + const { data } = useQuery(GET_USERS); + return ( +
+

User directory

+ +
+ ); +} diff --git a/eval/fixtures/c4-graphql/app/queries.ts b/eval/fixtures/c4-graphql/app/queries.ts new file mode 100644 index 0000000..875b630 --- /dev/null +++ b/eval/fixtures/c4-graphql/app/queries.ts @@ -0,0 +1,21 @@ +import { gql } from "@apollo/client"; + +// GraphQL operations defined as gql consts and imported by the components that +// run them — the common Apollo/urql pattern. The operation NAME is the data +// source identity; a component that runs it gets a fetches-from edge. +export const GET_USERS = gql` + query GetUsers { + users { + id + name + } + } +`; + +export const CREATE_USER = gql` + mutation CreateUser($input: UserInput!) { + createUser(input: $input) { + id + } + } +`; diff --git a/eval/fixtures/c4-graphql/golden.json b/eval/fixtures/c4-graphql/golden.json new file mode 100644 index 0000000..9afadb4 --- /dev/null +++ b/eval/fixtures/c4-graphql/golden.json @@ -0,0 +1,21 @@ +{ + "failureMode": "C4", + "note": "GraphQL adapter (7.1): Apollo/urql operations become data sources keyed by operation NAME, with the operation TYPE (query/mutation/subscription) as the method. Covers an imported gql const via useQuery (GetUsers) and useMutation (CreateUser) — which share their hook names with react-query but are disambiguated by the gql document argument — and an inline gql tagged template via useSubscription (OnTick). Each running component gets a fetches-from edge to its operation.", + "expect": { + "components": [ + { "name": "UsersPanel", "instances": 0 }, + { "name": "InviteForm", "instances": 0 }, + { "name": "LiveTicker", "instances": 0 } + ], + "attributions": [ + { "component": "UsersPanel", "endpoints": ["GetUsers"] }, + { "component": "InviteForm", "endpoints": ["CreateUser"] }, + { "component": "LiveTicker", "endpoints": ["OnTick"] } + ], + "queries": [ + { "terms": ["User directory"], "status": "ok", "top": "UsersPanel" }, + { "terms": ["Invite teammate"], "status": "ok", "top": "InviteForm" }, + { "terms": ["Live updates"], "status": "ok", "top": "LiveTicker" } + ] + } +} diff --git a/eval/fixtures/c8-push-channels/app/ChatPanel.tsx b/eval/fixtures/c8-push-channels/app/ChatPanel.tsx new file mode 100644 index 0000000..d1adc5f --- /dev/null +++ b/eval/fixtures/c8-push-channels/app/ChatPanel.tsx @@ -0,0 +1,17 @@ +import { useEffect, useState } from "react"; + +// WebSocket push channel → websocket data source. +export function ChatPanel() { + const [messages, setMessages] = useState([]); + useEffect(() => { + const socket = new WebSocket("wss://chat.example.com/socket"); + socket.onmessage = (e) => setMessages((m) => [...m, e.data]); + return () => socket.close(); + }, []); + return ( +
+

Team chat

+ +
+ ); +} diff --git a/eval/fixtures/c8-push-channels/app/NotificationsFeed.tsx b/eval/fixtures/c8-push-channels/app/NotificationsFeed.tsx new file mode 100644 index 0000000..0a3aadd --- /dev/null +++ b/eval/fixtures/c8-push-channels/app/NotificationsFeed.tsx @@ -0,0 +1,17 @@ +import { useEffect, useState } from "react"; + +// Server-Sent Events push channel → sse data source. +export function NotificationsFeed() { + const [count, setCount] = useState(0); + useEffect(() => { + const source = new EventSource("/api/notifications/stream"); + source.onmessage = () => setCount((c) => c + 1); + return () => source.close(); + }, []); + return ( + + ); +} diff --git a/eval/fixtures/c8-push-channels/golden.json b/eval/fixtures/c8-push-channels/golden.json new file mode 100644 index 0000000..6267afb --- /dev/null +++ b/eval/fixtures/c8-push-channels/golden.json @@ -0,0 +1,18 @@ +{ + "failureMode": "C8", + "note": "Push channels (7.3): server-push data arrives over a long-lived connection, not a request/response fetch. `new WebSocket(url)` becomes a websocket data source and `new EventSource(url)` an sse data source, each with the URL as its endpoint and a fetches-from edge from the component that opens it — so a component fed by a live channel shows that channel in its lineage.", + "expect": { + "components": [ + { "name": "ChatPanel", "instances": 0 }, + { "name": "NotificationsFeed", "instances": 0 } + ], + "attributions": [ + { "component": "ChatPanel", "endpoints": ["wss://chat.example.com/socket"] }, + { "component": "NotificationsFeed", "endpoints": ["/api/notifications/stream"] } + ], + "queries": [ + { "terms": ["Team chat"], "status": "ok", "top": "ChatPanel" }, + { "terms": ["Live notifications"], "status": "ok", "top": "NotificationsFeed" } + ] + } +} diff --git a/eval/fixtures/c9-nextjs-server-data/app/BlogPage.tsx b/eval/fixtures/c9-nextjs-server-data/app/BlogPage.tsx new file mode 100644 index 0000000..d52c209 --- /dev/null +++ b/eval/fixtures/c9-nextjs-server-data/app/BlogPage.tsx @@ -0,0 +1,15 @@ +// Pages-router: getStaticProps fetches at build time; feeds the page. +export async function getStaticProps() { + const res = await fetch("/api/posts"); + const posts = await res.json(); + return { props: { posts } }; +} + +export default function BlogPage({ posts }: { posts: { id: string }[] }) { + return ( +
+

Blog index

+ {posts.length} posts +
+ ); +} diff --git a/eval/fixtures/c9-nextjs-server-data/app/DashboardPage.tsx b/eval/fixtures/c9-nextjs-server-data/app/DashboardPage.tsx new file mode 100644 index 0000000..9510ab5 --- /dev/null +++ b/eval/fixtures/c9-nextjs-server-data/app/DashboardPage.tsx @@ -0,0 +1,11 @@ +// App-router RSC: an async server component that fetches in its own body. +export default async function DashboardPage() { + const res = await fetch("/api/dashboard/summary"); + const data = await res.json(); + return ( +
+

Dashboard totals

+

{data.total}

+
+ ); +} diff --git a/eval/fixtures/c9-nextjs-server-data/app/ReportsPage.tsx b/eval/fixtures/c9-nextjs-server-data/app/ReportsPage.tsx new file mode 100644 index 0000000..d6115f6 --- /dev/null +++ b/eval/fixtures/c9-nextjs-server-data/app/ReportsPage.tsx @@ -0,0 +1,15 @@ +// Pages-router: getServerSideProps fetches server-side; the page renders props. +export async function getServerSideProps() { + const res = await fetch("/api/reports/latest"); + const report = await res.json(); + return { props: { report } }; +} + +export default function ReportsPage({ report }: { report: { title: string } }) { + return ( +
+

Report viewer

+ {report.title} +
+ ); +} diff --git a/eval/fixtures/c9-nextjs-server-data/golden.json b/eval/fixtures/c9-nextjs-server-data/golden.json new file mode 100644 index 0000000..3805740 --- /dev/null +++ b/eval/fixtures/c9-nextjs-server-data/golden.json @@ -0,0 +1,21 @@ +{ + "failureMode": "C9", + "note": "Next.js server data (7.2): server-side data fetching feeds a page even though the fetch is not in the component body. An async RSC server component fetches inline (already covered), while pages-router getServerSideProps/getStaticProps run separately from the page — their fetches are attributed to the file's default-export page component so its lineage shows the data it renders.", + "expect": { + "components": [ + { "name": "DashboardPage", "instances": 0 }, + { "name": "ReportsPage", "instances": 0 }, + { "name": "BlogPage", "instances": 0 } + ], + "attributions": [ + { "component": "DashboardPage", "endpoints": ["/api/dashboard/summary"] }, + { "component": "ReportsPage", "endpoints": ["/api/reports/latest"] }, + { "component": "BlogPage", "endpoints": ["/api/posts"] } + ], + "queries": [ + { "terms": ["Dashboard totals"], "status": "ok", "top": "DashboardPage" }, + { "terms": ["Report viewer"], "status": "ok", "top": "ReportsPage" }, + { "terms": ["Blog index"], "status": "ok", "top": "BlogPage" } + ] + } +} diff --git a/eval/package.json b/eval/package.json index 5a6c848..52285fe 100644 --- a/eval/package.json +++ b/eval/package.json @@ -1,6 +1,6 @@ { "name": "@coderadar/eval", - "version": "0.5.0", + "version": "0.6.0", "private": true, "description": "CodeRadar eval harness — scans failure-mode fixtures, diffs against golden outputs, emits the scorecard that gates every phase.", "license": "MIT", diff --git a/package.json b/package.json index 28f59eb..7b1fa36 100644 --- a/package.json +++ b/package.json @@ -1,7 +1,7 @@ { "name": "coderadar", "private": true, - "version": "0.5.0", + "version": "0.6.0", "description": "Static analysis toolkit that maps UI components to their data sources (APIs, state, events) — enabling AI agents to trace any screenshot back to the code and data behind it.", "license": "MIT", "packageManager": "pnpm@11.2.2", diff --git a/packages/agent-sdk/package.json b/packages/agent-sdk/package.json index 3afe89a..1c82956 100644 --- a/packages/agent-sdk/package.json +++ b/packages/agent-sdk/package.json @@ -1,6 +1,6 @@ { "name": "@coderadar/agent-sdk", - "version": "0.5.0", + "version": "0.6.0", "private": true, "description": "resolveContext orchestrator — classifies a ticket and runs the matching engine. Deterministic, no LLM in the node. Bundled into the published ui-lineage package.", "license": "MIT", diff --git a/packages/cli/README.md b/packages/cli/README.md index ebfae69..4e5acb4 100644 --- a/packages/cli/README.md +++ b/packages/cli/README.md @@ -111,7 +111,15 @@ Turn a screenshot into `{ terms, structure, annotations }` and match it — term ## What it understands -Endpoints (constants, templates, API wrappers, react-query/SWR), i18n text, cross-file instance trees & per-instance prop-flow, Redux/Zustand stores, portals/modals/toasts, React Router & Next.js routes, action effects (navigate/fetch/dispatch/setState), form libraries & non-JSX events (react-hook-form, `addEventListener`, hotkeys), and feature-flag/role conditions. +Endpoints (constants, templates, API wrappers, react-query/SWR/RTK Query), GraphQL operations, WebSocket/SSE push channels, Next.js server-side data (RSC async components, `getServerSideProps`/`getStaticProps`), i18n text, cross-file instance trees & per-instance prop-flow, Redux/Zustand stores, portals/modals/toasts, React Router & Next.js routes, action effects (navigate/fetch/dispatch/setState), form libraries & non-JSX events (react-hook-form, `addEventListener`, hotkeys), and feature-flag/role conditions. + +## New in 0.6.0 + +Wider data-source coverage — the graph now sees the ways modern React apps actually fetch, beyond REST calls in the component body: + +- **GraphQL** — Apollo/urql operations become data sources, keyed by operation name and typed by operation (`query` / `mutation` / `subscription`). Works with an inline `gql` tag or an imported/co-located `gql` const (resolved across files), and is disambiguated from react-query, which shares the `useQuery`/`useMutation` hook names. +- **Push channels** — `new WebSocket(url)` and `new EventSource(url)` become `websocket` / `sse` data sources, so a component fed by a live channel shows that channel in its lineage. +- **Next.js server data** — a page's server-side fetching is attributed to the page even when it isn't in the component body: pages-router `getServerSideProps` / `getStaticProps` / `getStaticPaths`, alongside app-router async RSC server components. ## New in 0.5.0 diff --git a/packages/cli/package.json b/packages/cli/package.json index 403f474..8dbb1fa 100644 --- a/packages/cli/package.json +++ b/packages/cli/package.json @@ -1,6 +1,6 @@ { "name": "ui-lineage", - "version": "0.5.0", + "version": "0.6.0", "description": "Map UI components to their data sources and user journeys — trace any screenshot back to the code, APIs, state, and navigation behind it. Deterministic static analysis for React/TSX.", "license": "MIT", "type": "module", diff --git a/packages/cli/src/index.ts b/packages/cli/src/index.ts index bb04115..d71942f 100644 --- a/packages/cli/src/index.ts +++ b/packages/cli/src/index.ts @@ -34,7 +34,7 @@ program .description( "Map UI components to their data sources and user journeys — trace any screenshot back to the code, APIs, state, and navigation behind it.", ) - .version("0.5.0"); + .version("0.6.0"); program .command("scan") diff --git a/packages/core/package.json b/packages/core/package.json index f9812c6..ee8c8f2 100644 --- a/packages/core/package.json +++ b/packages/core/package.json @@ -1,6 +1,6 @@ { "name": "@coderadar/core", - "version": "0.5.0", + "version": "0.6.0", "private": true, "description": "CodeRadar lineage graph schema — the language-agnostic contract every parser emits and every agent consumes. Bundled into the published `ui-lineage` package.", "license": "MIT", diff --git a/packages/core/src/types.ts b/packages/core/src/types.ts index 3398dba..98484ef 100644 --- a/packages/core/src/types.ts +++ b/packages/core/src/types.ts @@ -162,6 +162,7 @@ export type DataSourceKind = | "swr" | "graphql" | "websocket" + | "sse" | "unknown"; /** How much of an endpoint was statically resolvable. */ diff --git a/packages/mcp/package.json b/packages/mcp/package.json index b5a89a6..b91e130 100644 --- a/packages/mcp/package.json +++ b/packages/mcp/package.json @@ -1,6 +1,6 @@ { "name": "@coderadar/mcp", - "version": "0.5.0", + "version": "0.6.0", "private": true, "description": "Model Context Protocol server exposing CodeRadar as a context-provider node: resolve_context, find_component, trace_lineage, journeys, blast_radius over a pre-built graph.", "license": "MIT", diff --git a/packages/mcp/src/server.ts b/packages/mcp/src/server.ts index bbf9f22..cac6aef 100644 --- a/packages/mcp/src/server.ts +++ b/packages/mcp/src/server.ts @@ -33,7 +33,7 @@ function envelope(value: unknown): ToolResult { /** Build a configured MCP server bound to one pre-built graph. */ export function createServer(graph: LineageGraph): McpServer { - const server = new McpServer({ name: "coderadar", version: "0.5.0" }); + const server = new McpServer({ name: "coderadar", version: "0.6.0" }); server.registerTool( "resolve_context", diff --git a/packages/parser-react/package.json b/packages/parser-react/package.json index 9a93d60..3abe94f 100644 --- a/packages/parser-react/package.json +++ b/packages/parser-react/package.json @@ -1,6 +1,6 @@ { "name": "@coderadar/parser-react", - "version": "0.5.0", + "version": "0.6.0", "private": true, "description": "React/TSX parser for CodeRadar — extracts components, hooks, data sources, state, and events into a lineage graph. Bundled into the published `ui-lineage` package.", "license": "MIT", diff --git a/packages/parser-react/src/graphql.test.ts b/packages/parser-react/src/graphql.test.ts new file mode 100644 index 0000000..bcbc9b4 --- /dev/null +++ b/packages/parser-react/src/graphql.test.ts @@ -0,0 +1,84 @@ +import path from "node:path"; +import { fileURLToPath } from "node:url"; + +import type { DataSourceNode } from "@coderadar/core"; +import { describe, expect, it } from "vitest"; + +import { parseGraphqlOperation } from "./graphql.js"; +import { resolveHookEdges, scanReact } from "./scan.js"; + +const fixtures = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "../../../eval/fixtures"); + +describe("parseGraphqlOperation (7.1, C4)", () => { + it("parses a named query with its root field", () => { + expect(parseGraphqlOperation("query GetUsers { users { id name } }")).toStrictEqual({ + type: "query", + name: "GetUsers", + rootFields: ["users"], + }); + }); + + it("parses a named mutation with variables", () => { + expect( + parseGraphqlOperation("mutation CreateUser($input: UserInput!) { createUser(input: $input) { id } }"), + ).toStrictEqual({ type: "mutation", name: "CreateUser", rootFields: ["createUser"] }); + }); + + it("parses a subscription", () => { + const op = parseGraphqlOperation("subscription OnTick { tick { at } }"); + expect(op?.type).toBe("subscription"); + expect(op?.name).toBe("OnTick"); + }); + + it("parses an anonymous shorthand query via its root fields", () => { + expect(parseGraphqlOperation("{ me { id } settings { theme } }")).toStrictEqual({ + type: "query", + name: null, + rootFields: ["me", "settings"], + }); + }); + + it("ignores # comments and leading whitespace", () => { + const op = parseGraphqlOperation("\n # fetch the current user\n query Me { me { id } }\n"); + expect(op).toStrictEqual({ type: "query", name: "Me", rootFields: ["me"] }); + }); + + it("returns null for a fragment-only document (not an operation)", () => { + expect(parseGraphqlOperation("fragment UserFields on User { id name }")).toBeNull(); + }); +}); + +describe("GraphQL data-source extraction (7.1, C4)", () => { + const graph = resolveHookEdges(scanReact({ root: path.join(fixtures, "c4-graphql/app") })); + const source = (endpoint: string): DataSourceNode | undefined => + graph.nodes.find((n): n is DataSourceNode => n.kind === "data-source" && n.endpoint === endpoint); + + it("emits a graphql source per operation, keyed by name, typed by operation", () => { + expect(source("GetUsers")).toMatchObject({ sourceKind: "graphql", method: "query", resolved: "full" }); + expect(source("CreateUser")).toMatchObject({ sourceKind: "graphql", method: "mutation" }); + expect(source("OnTick")).toMatchObject({ sourceKind: "graphql", method: "subscription" }); + }); + + it("wires a fetches-from edge from each running component", () => { + const from = (comp: string, ds: string): boolean => + graph.edges.some( + (e) => + e.kind === "fetches-from" && + e.from === `component:${comp}.tsx#${comp}` && + e.to.endsWith(`#graphql:${ds}`), + ); + expect(from("UsersPanel", "GetUsers")).toBe(true); + expect(from("InviteForm", "CreateUser")).toBe(true); + expect(from("LiveTicker", "OnTick")).toBe(true); + }); + + it("does not misclassify react-query useQuery as graphql (disambiguation)", () => { + // The c5 fixture uses react-query useQuery/useMutation with URL queryFns, no gql. + const rq = resolveHookEdges(scanReact({ root: path.join(fixtures, "c5-queryfn-indirection/app") })); + const graphqlSources = rq.nodes.filter( + (n) => n.kind === "data-source" && n.sourceKind === "graphql", + ); + expect(graphqlSources).toStrictEqual([]); + expect(rq.nodes.some((n) => n.kind === "data-source" && n.sourceKind === "react-query")).toBe(true); + }); +}); diff --git a/packages/parser-react/src/graphql.ts b/packages/parser-react/src/graphql.ts new file mode 100644 index 0000000..cb000d3 --- /dev/null +++ b/packages/parser-react/src/graphql.ts @@ -0,0 +1,134 @@ +/** + * GraphQL operation extraction (TRACKER step 7.1, failure mode C4). + * + * Recognizes GraphQL operations a component runs — Apollo / urql / graphql-request + * hooks (`useQuery`, `useMutation`, `useSubscription`, `useLazyQuery`) whose + * argument is a `gql`/`graphql` tagged template (inline, or a const that resolves + * to one). Each operation becomes a data source: the operation NAME is its + * identity (the value federation and attribution join on), the operation TYPE + * (query/mutation/subscription) is its "method". Anonymous operations fall back + * to their first root selection field. + */ +import { Node } from "ts-morph"; + +/** The GraphQL hook names that take an operation document as their first argument. */ +export const GRAPHQL_HOOKS: ReadonlySet = new Set([ + "useQuery", + "useMutation", + "useSubscription", + "useLazyQuery", +]); + +/** Tag names that mark a template literal as a GraphQL document. */ +const GQL_TAGS: ReadonlySet = new Set(["gql", "graphql"]); + +export interface GraphqlOperation { + type: "query" | "mutation" | "subscription"; + /** Operation name, or null for an anonymous operation. */ + name: string | null; + /** Top-level selection field names (best-effort), a fallback identity. */ + rootFields: string[]; +} + +/** Strip `# ...` line comments and the surrounding backticks from a template's text. */ +function documentText(raw: string): string { + return raw.replace(/^`|`$/g, "").replace(/#[^\n]*/g, " "); +} + +/** + * Parse a GraphQL document's leading operation. Handles named and anonymous + * `query`/`mutation`/`subscription` operations and the shorthand `{ … }` query. + * Returns null for a fragment-only document (a reusable selection, not a fetch). + */ +export function parseGraphqlOperation(rawDocument: string): GraphqlOperation | null { + const text = documentText(rawDocument); + const opMatch = /\b(query|mutation|subscription)\b\s*([A-Za-z_]\w*)?/.exec(text); + let type: GraphqlOperation["type"]; + let name: string | null; + let bodyStart: number; + if (opMatch !== null) { + type = opMatch[1] as GraphqlOperation["type"]; + name = opMatch[2] ?? null; + bodyStart = opMatch.index + opMatch[0].length; + } else if (/^\s*\{/.test(text)) { + // Shorthand anonymous query: `{ field { … } }`. + type = "query"; + name = null; + bodyStart = text.indexOf("{"); + } else { + return null; + } + return { type, name, rootFields: rootSelectionFields(text, bodyStart) }; +} + +/** + * The field names selected directly inside the operation's top-level `{ … }` + * (depth 1). Skips argument lists `( … )` and nested selections, and drops the + * variable-definition list that can precede the body. + */ +function rootSelectionFields(text: string, from: number): string[] { + const open = text.indexOf("{", from); + if (open === -1) return []; + const fields: string[] = []; + let depth = 0; + let parens = 0; + let token = ""; + const flush = (): void => { + // A field is a bareword captured at brace depth 1; an alias `a: b` keeps `a`. + const name = token.split(":")[0]?.trim(); + if (name !== undefined && /^[A-Za-z_]\w*$/.test(name) && name !== "on") fields.push(name); + token = ""; + }; + for (let i = open; i < text.length; i += 1) { + const ch = text.charAt(i); + if (ch === "(") parens += 1; + else if (ch === ")") parens -= 1; + else if (parens > 0) continue; + else if (ch === "{") { + if (depth === 1) flush(); + depth += 1; + token = ""; + } else if (ch === "}") { + if (depth === 1) flush(); + depth -= 1; + if (depth === 0) break; + } else if (depth === 1) { + if (/\s/.test(ch)) flush(); + else token += ch; + } + } + // Dedupe, preserve order. + return [...new Set(fields)]; +} + +/** The GraphQL document text of a `gql`/`graphql` tagged template, or null. */ +function taggedGqlText(node: Node): string | null { + if (!Node.isTaggedTemplateExpression(node)) return null; + const tagName = node.getTag().getText().split(".").pop() ?? ""; + if (!GQL_TAGS.has(tagName)) return null; + return node.getTemplate().getText(); +} + +/** + * Resolve a hook argument to a GraphQL operation: an inline `gql`/`graphql` + * tagged template, or an identifier bound to one (in this file or an imported + * module — ts-morph follows the symbol). Returns null when the argument is not + * a GraphQL document (e.g. a react-query key/options object). + */ +export function graphqlOperationFromArg(arg: Node | undefined): GraphqlOperation | null { + if (arg === undefined) return null; + const inline = taggedGqlText(arg); + if (inline !== null) return parseGraphqlOperation(inline); + if (Node.isIdentifier(arg)) { + // getDefinitionNodes follows imports to the real declaration, so a gql const + // defined in another module (the common codegen/co-located pattern) resolves. + for (const decl of arg.getDefinitionNodes()) { + if (Node.isVariableDeclaration(decl)) { + const init = decl.getInitializer(); + const text = init !== undefined ? taggedGqlText(init) : null; + if (text !== null) return parseGraphqlOperation(text); + } + } + } + return null; +} diff --git a/packages/parser-react/src/nextjs.test.ts b/packages/parser-react/src/nextjs.test.ts new file mode 100644 index 0000000..9dc9722 --- /dev/null +++ b/packages/parser-react/src/nextjs.test.ts @@ -0,0 +1,40 @@ +import path from "node:path"; +import { fileURLToPath } from "node:url"; + +import { describe, expect, it } from "vitest"; + +import { resolveHookEdges, scanReact } from "./scan.js"; + +const fixtures = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "../../../eval/fixtures"); +const graph = resolveHookEdges(scanReact({ root: path.join(fixtures, "c9-nextjs-server-data/app") })); + +/** Endpoints a component fetches, via fetches-from edges. */ +function endpointsOf(component: string): string[] { + const id = `component:${component}.tsx#${component}`; + return graph.edges + .filter((e) => e.kind === "fetches-from" && e.from === id) + .map((e) => graph.nodes.find((n) => n.id === e.to)) + .flatMap((n) => (n?.kind === "data-source" ? [n.endpoint] : [])); +} + +describe("Next.js server data (7.2, C9)", () => { + it("attributes an async RSC server component's inline fetch to the page", () => { + expect(endpointsOf("DashboardPage")).toStrictEqual(["/api/dashboard/summary"]); + }); + + it("attributes getServerSideProps fetches to the default-export page", () => { + expect(endpointsOf("ReportsPage")).toStrictEqual(["/api/reports/latest"]); + }); + + it("attributes getStaticProps fetches to the default-export page", () => { + expect(endpointsOf("BlogPage")).toStrictEqual(["/api/posts"]); + }); + + it("does not create a component/hook node for the server-data functions", () => { + const names = graph.nodes + .filter((n) => n.kind === "component" || n.kind === "hook") + .map((n) => n.name); + expect(names).not.toContain("getServerSideProps"); + expect(names).not.toContain("getStaticProps"); + }); +}); diff --git a/packages/parser-react/src/pushchannels.test.ts b/packages/parser-react/src/pushchannels.test.ts new file mode 100644 index 0000000..7b881cc --- /dev/null +++ b/packages/parser-react/src/pushchannels.test.ts @@ -0,0 +1,50 @@ +import path from "node:path"; +import { fileURLToPath } from "node:url"; + +import type { DataSourceNode } from "@coderadar/core"; +import { describe, expect, it } from "vitest"; + +import { resolveHookEdges, scanReact } from "./scan.js"; + +const fixtures = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "../../../eval/fixtures"); +const graph = resolveHookEdges(scanReact({ root: path.join(fixtures, "c8-push-channels/app") })); + +const source = (endpoint: string): DataSourceNode | undefined => + graph.nodes.find((n): n is DataSourceNode => n.kind === "data-source" && n.endpoint === endpoint); + +describe("push-channel extraction (7.3, C8)", () => { + it("emits a websocket data source for new WebSocket(url)", () => { + expect(source("wss://chat.example.com/socket")).toMatchObject({ + sourceKind: "websocket", + method: "WS", + resolved: "full", + }); + }); + + it("emits an sse data source for new EventSource(url)", () => { + expect(source("/api/notifications/stream")).toMatchObject({ + sourceKind: "sse", + method: "SSE", + resolved: "full", + }); + }); + + it("wires a fetches-from edge from the component that opens the channel", () => { + const has = (comp: string, ds: string): boolean => + graph.edges.some( + (e) => e.kind === "fetches-from" && e.from === `component:${comp}.tsx#${comp}` && e.to === ds, + ); + expect(has("ChatPanel", "data-source:ChatPanel.tsx#websocket:wss://chat.example.com/socket")).toBe( + true, + ); + expect(has("NotificationsFeed", "data-source:NotificationsFeed.tsx#sse:/api/notifications/stream")).toBe( + true, + ); + }); + + it("does not treat an unrelated constructor as a data source", () => { + // Sanity: only WebSocket/EventSource are push channels — no stray sources. + const kinds = graph.nodes.flatMap((n) => (n.kind === "data-source" ? [n.sourceKind] : [])); + expect(new Set(kinds)).toStrictEqual(new Set(["websocket", "sse"])); + }); +}); diff --git a/packages/parser-react/src/pushchannels.ts b/packages/parser-react/src/pushchannels.ts new file mode 100644 index 0000000..20a43f7 --- /dev/null +++ b/packages/parser-react/src/pushchannels.ts @@ -0,0 +1,42 @@ +/** + * Push-channel extraction (TRACKER step 7.3, failure mode C8). + * + * Server-push data arrives over a long-lived connection, not a request/response + * fetch: the browser `WebSocket` and Server-Sent-Events `EventSource` + * constructors. Each `new WebSocket(url)` / `new EventSource(url)` becomes a data + * source — `websocket` / `sse` sourceKind, the URL as its endpoint — so a + * component fed by a live channel shows that channel in its lineage. + */ +import type { DataSourceKind } from "@coderadar/core"; +import { Node } from "ts-morph"; + +import { resolveEndpoint, type ResolvedEndpoint } from "./endpoint.js"; + +export interface PushChannel extends ResolvedEndpoint { + sourceKind: DataSourceKind; + /** Transport label: "WS" for WebSocket, "SSE" for EventSource. */ + method: string; +} + +/** The last identifier in a (possibly namespaced) callee, e.g. `window.WebSocket` → `WebSocket`. */ +function calleeName(expr: string): string { + return expr.split(".").pop() ?? expr; +} + +/** + * A push-channel data source for a `new WebSocket(url)` / `new EventSource(url)` + * expression, or null. Only the standard global constructors are matched (by + * name, including a `window.`/`self.` qualifier) — a very low false-positive set. + */ +export function detectPushChannel(node: Node, baseUrls: string[]): PushChannel | null { + if (!Node.isNewExpression(node)) return null; + const name = calleeName(node.getExpression().getText()); + const urlArg = node.getArguments()?.[0]; + if (name === "WebSocket") { + return { sourceKind: "websocket", method: "WS", ...resolveEndpoint(urlArg, baseUrls) }; + } + if (name === "EventSource") { + return { sourceKind: "sse", method: "SSE", ...resolveEndpoint(urlArg, baseUrls) }; + } + return null; +} diff --git a/packages/parser-react/src/scan.ts b/packages/parser-react/src/scan.ts index 819fa53..15c0623 100644 --- a/packages/parser-react/src/scan.ts +++ b/packages/parser-react/src/scan.ts @@ -33,6 +33,8 @@ import { import { fetchMethod, resolveEndpoint, type ResolvedEndpoint, resolveStringValue } from "./endpoint.js"; import { decodeEntities } from "./entities.js"; +import { GRAPHQL_HOOKS, graphqlOperationFromArg } from "./graphql.js"; +import { detectPushChannel } from "./pushchannels.js"; import { i18nRenderedText, type I18nOptions, loadLocaleTable, type LocaleTable } from "./i18n.js"; import { linkOpenApiResponses, loadOpenApi, responseFromCall } from "./response.js"; import { detectRoutes } from "./routes.js"; @@ -149,6 +151,18 @@ const DEFAULT_FLAG_CALLEES = [ /** Heuristic for role/permission guards classified as `role` conditions (3.5). */ const ROLE_PATTERN = /\brole\b|\bisAdmin\b|\bisSuperuser\b|hasRole|hasPermission|\bcan\(|\bpermission|useRole|usePermission/i; +/** + * Next.js pages-router server-data functions (7.2, C9). These run on the server + * and fetch the data the page renders; their fetches feed the file's default- + * export page component, so they are attributed to it (they are not components + * or hooks, so the normal body walk skips them). + */ +const NEXT_DATA_FNS: ReadonlySet = new Set([ + "getServerSideProps", + "getStaticProps", + "getStaticPaths", +]); + /** The include globs a scan discovers source files with. */ function scanInclude(options: ScanOptions): string[] { return options.include ?? ["**/*.tsx", "**/*.jsx", "**/*.ts"]; @@ -239,13 +253,22 @@ export function scanProject(project: Project, root: string, options: ScanOptions // Test files are swept separately (5.4) — they exercise components, they // don't define the app's UI, so they must not produce component/hook nodes. if (isTestFile(file)) continue; + // The file's default-export page and any Next.js server-data functions, + // resolved together so the latter's fetches attribute to the former (7.2). + let pageComponentId: string | undefined; + const nextDataFns: { name: string; fn: Node }[] = []; for (const decl of collectDeclarations(sourceFile, file)) { + if (NEXT_DATA_FNS.has(decl.name)) { + nextDataFns.push({ name: decl.name, fn: decl.fn }); + continue; + } const isComponent = COMPONENT_NAME.test(decl.name) && returnsJsx(decl.fn); const isHook = HOOK_NAME.test(decl.name); if (!isComponent && !isHook) continue; const kind = isComponent ? "component" : "hook"; const id = nodeId(kind, file, decl.name); + if (isComponent && decl.exportName === "default") pageComponentId = id; if (isComponent) { // Portal components (A9): rendered into document.body etc., far from @@ -277,6 +300,16 @@ export function scanProject(project: Project, root: string, options: ScanOptions extractBodyFacts(decl.name, decl.fn, id, file, nodes, addEdge, baseUrls, wrappers, stores, handlerExprs, flagCallees); } + // Next.js server data (7.2, C9): a page's getServerSideProps/getStaticProps + // fetches on the server; those data sources feed the default-export page, + // so attribute them to it. (RSC async server components fetch in their own + // body and are already covered by the walk above.) + if (pageComponentId !== undefined) { + for (const { name, fn } of nextDataFns) { + extractBodyFacts(name, fn, pageComponentId, file, nodes, addEdge, baseUrls, wrappers, stores, handlerExprs, flagCallees); + } + } + scanClassComponents(sourceFile, file, nodes, addEdge, baseUrls, wrappers, localeTable, pendingInstances, stores, handlerExprs, flagCallees); collectHocAliases(sourceFile, hocAliases); } @@ -330,7 +363,7 @@ export function scanProject(project: Project, root: string, options: ScanOptions version: 2, root, generatedAt: new Date().toISOString(), - generator: "ui-lineage@0.5.0", + generator: "ui-lineage@0.6.0", // Canonical output order (6.3, G8): sort nodes and edges by stable keys so // the serialized graph is byte-identical across runs and machines. Node ids // are unique; edges are keyed by every identifying field. The query side @@ -892,6 +925,31 @@ function extractBodyFacts( } } + // Push channels (7.3, C8): new WebSocket(url) / new EventSource(url) are + // long-lived server-push data sources, not request/response fetches. A + // wrapper's own body carries a placeholder URL, so skip it like fetch sources. + if (!declIsWrapper) { + for (const expr of body.getDescendantsOfKind(SyntaxKind.NewExpression)) { + const channel = detectPushChannel(expr, baseUrls); + if (channel === null) continue; + const dsId = nodeId("data-source", file, `${channel.sourceKind}:${channel.endpoint}`); + if (!nodes.has(dsId)) { + nodes.set(dsId, { + id: dsId, + kind: "data-source", + name: channel.endpoint, + loc: locOf(expr, file), + sourceKind: channel.sourceKind, + method: channel.method, + endpoint: channel.endpoint, + raw: channel.raw, + resolved: channel.resolved, + }); + } + addEdge({ from: ownerId, to: dsId, kind: "fetches-from" }); + } + } + for (const attr of body.getDescendantsOfKind(SyntaxKind.JsxAttribute)) { const attrName = attr.getNameNode().getText(); // / — a link that leaves the app (B9). @@ -1011,6 +1069,23 @@ function detectDataSource( }; } + // GraphQL (7.1, C4): Apollo/urql hooks reuse `useQuery`/`useMutation`, so this + // must run before the react-query branch — it only claims the call when the + // argument is an actual gql document (else it falls through to react-query). + if (GRAPHQL_HOOKS.has(callee)) { + const op = graphqlOperationFromArg(call.getArguments()[0]); + if (op !== null) { + const identity = op.name ?? op.rootFields[0] ?? ""; + return { + sourceKind: "graphql", + method: op.type, + endpoint: identity, + raw: (call.getArguments()[0]?.getText() ?? callee).slice(0, 120), + resolved: op.name !== null ? "full" : op.rootFields.length > 0 ? "partial" : "none", + }; + } + } + if (callee === "useQuery" || callee === "useMutation" || callee === "useInfiniteQuery") { let keyText: string | undefined; let fnExpr: Node | undefined; diff --git a/packages/vision/package.json b/packages/vision/package.json index f928082..add3703 100644 --- a/packages/vision/package.json +++ b/packages/vision/package.json @@ -1,6 +1,6 @@ { "name": "@coderadar/vision", - "version": "0.5.0", + "version": "0.6.0", "private": true, "description": "Screenshot → { terms, structure, annotations } vision adapter for ui-lineage. Images are processed in memory and never persisted (G7). Bundled into the published ui-lineage package.", "license": "MIT", diff --git a/schemas/lineage-graph.schema.json b/schemas/lineage-graph.schema.json index a9bdd51..49aba57 100644 --- a/schemas/lineage-graph.schema.json +++ b/schemas/lineage-graph.schema.json @@ -467,6 +467,7 @@ "swr", "graphql", "websocket", + "sse", "unknown" ] },