Skip to content

Commit 6b2ec10

Browse files
committed
fix(ai): deliver a stream rejection as an error part
The guard appends the fallback message as another text delta, which the client cannot tell from the answer: it concatenates deltas, so the rejection lands glued to the truncated prefix already rendered. Emit an error part instead. The UI message stream carries it on its own channel, so the client can drop what it has and show a failure. Text already sent still cannot be taken back, but the client now knows to discard it.
1 parent 93a9662 commit 6b2ec10

3 files changed

Lines changed: 43 additions & 11 deletions

File tree

src/integrations/vercel-ai/index.ts

Lines changed: 19 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -58,7 +58,10 @@ function guardedTransform<TOOLS extends ToolSet>(guard: StreamGuard, onReject: (
5858
let rejectReported = false;
5959

6060
/**
61-
* Forward the guard's verdict downstream, reporting a rejection at most once
61+
* Forward the guard's verdict downstream, reporting a rejection at most once.
62+
*
63+
* A rejection travels as an error part rather than more text, so the client
64+
* can tell it apart from the answer and drop what it has already rendered.
6265
*
6366
* @param verdict - what the guard allows to be sent
6467
* @param controller - transform stream controller
@@ -69,19 +72,28 @@ function guardedTransform<TOOLS extends ToolSet>(guard: StreamGuard, onReject: (
6972
controller: TransformStreamDefaultController<TextStreamPart<TOOLS>>,
7073
id: string | null
7174
): void => {
75+
if (verdict.rejected) {
76+
if (!rejectReported) {
77+
rejectReported = true;
78+
79+
controller.enqueue({
80+
type: 'error',
81+
error: new Error(verdict.emit),
82+
} as TextStreamPart<TOOLS>);
83+
84+
onReject();
85+
}
86+
87+
return;
88+
}
89+
7290
if (verdict.emit && id !== null) {
7391
controller.enqueue({
7492
type: 'text-delta',
7593
id,
7694
text: verdict.emit,
7795
} as TextStreamPart<TOOLS>);
7896
}
79-
80-
if (verdict.rejected && !rejectReported) {
81-
rejectReported = true;
82-
83-
onReject();
84-
}
8597
};
8698

8799
return new TransformStream<TextStreamPart<TOOLS>, TextStreamPart<TOOLS>>({

src/services/askAi/security/holdback.ts

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -54,10 +54,9 @@ export interface StreamGuard {
5454
* next one's id. Its tail is kept as scanning context instead, without which a
5555
* nonce split across two blocks would pass unseen.
5656
*
57-
* On rejection the rest of the answer is replaced by
58-
* {@link SUGGESTION_FALLBACK_MESSAGE}, emitted once. Text already sent cannot be
59-
* taken back, and the holdback cuts it at an arbitrary character, so the client
60-
* is left with a truncated prefix followed by the fallback.
57+
* On rejection nothing more is released and {@link SUGGESTION_FALLBACK_MESSAGE}
58+
* is returned once, for the transport to deliver as it sees fit. Text already
59+
* sent cannot be taken back, and the holdback cuts it at an arbitrary character.
6160
*
6261
* @param nonce - per-request nonce used in the prompt markers
6362
* @returns {StreamGuard} guard for a single stream, not reusable

test/integrations/vercel-ai.test.ts

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -239,6 +239,27 @@ describe('VercelAIApi', () => {
239239
expect(emitted).toEqual([ 'tail' ]);
240240
});
241241

242+
it('should deliver a rejection as an error part instead of more text', async () => {
243+
const guard = stubGuard([
244+
{
245+
emit: 'Could not generate an answer.',
246+
rejected: true,
247+
},
248+
{
249+
emit: '',
250+
rejected: true,
251+
},
252+
]);
253+
254+
const { parts } = await runTransformRaw(guard, [delta('first'), delta('second')]);
255+
256+
expect(parts.filter((part) => part.type === 'text-delta')).toEqual([]);
257+
expect(parts.filter((part) => part.type === 'error')).toEqual([ {
258+
type: 'error',
259+
error: new Error('Could not generate an answer.'),
260+
} ]);
261+
});
262+
242263
it('should report a rejection once even when the guard keeps reporting it', async () => {
243264
const onReject = jest.fn();
244265
const guard = stubGuard([

0 commit comments

Comments
 (0)