{"id":899,"date":"1970-01-01T00:00:00","date_gmt":"1970-01-01T00:00:00","guid":{"rendered":"https:\/\/imcodinggenius.com\/?p=899"},"modified":"1970-01-01T00:00:00","modified_gmt":"1970-01-01T00:00:00","slug":"building-a-real-time-ai-chat-feature-in-node-js-using-openais-streaming-api","status":"publish","type":"post","link":"https:\/\/imcodinggenius.com\/?p=899","title":{"rendered":"Building a Real-Time AI Chat Feature in Node.js Using OpenAI’s Streaming API"},"content":{"rendered":"

I almost shipped a bad AI integration once. The feature worked technically: the model answered questions correctly, the UI looked clean, and everyone nodded along in our internal demo. Then we put real users on it, and within 48 hours a product manager Slack’d me asking why 60% of sessions were ending with no response received.<\/p>\n

The problem wasn’t the API or the server. It was eight seconds of silence.<\/p>\n

Users sent a message, saw nothing happen, assumed the request had failed, and closed the tab. The response would arrive shortly after, to nobody. We had built a functional AI feature that felt completely broken, and the fix wasn’t an architectural overhaul or a performance optimization. It was streaming: start sending tokens the moment the model produces them, instead of batching the full response and delivering it at once.<\/p>\n

I have seen this mistake in codebases across multiple companies now. The non-streaming version is simpler to write, tests pass, and nobody notices in development because developers wait for things, while real users usually do not.<\/p>\n

This tutorial builds the complete streaming implementation: a Node.js backend using Server-Sent Events to push tokens to the browser as they arrive, with session management that gives the model memory across turns, and proper abort handling so you are not paying for tokens nobody reads.<\/p>\n

The complete code is at github.com\/ziaongit\/nodejs-openai-streaming<\/a>.<\/p>\n

No OpenAI key?<\/strong> I tested this entire article using Groq<\/a> instead. It is free, it speaks the same OpenAI SDK format, and getting a key takes about two minutes. The repo has a USE_GROQ=true flag already wired in. Copy .env.example to .env, paste your Groq key, and you are running. I used llama-3.3-70b-versatile. The output is good enough that you would not notice the difference in a dev session.<\/p>\n

Prerequisites<\/h2>\n

Node.js v18.11 or later
\nAn OpenAI API key
\nSolid familiarity with Express and async\/await<\/p>\n

The Architecture Before Any Code<\/h2>\n

Before we write any code, it helps to see what is actually happening across the three layers. Debugging streaming issues without this picture gets painful fast.<\/p>\n

\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510<\/p>\n

\u2502 Browser \u2502<\/p>\n

\u2502 \u2502<\/p>\n

\u2502 1. POST \/session \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u25ba receives sessionId \u2502<\/p>\n

\u2502 2. POST \/chat { message, sessionId } \u2502<\/p>\n

\u2502 3. response.body.getReader() \u2500\u2500\u25ba ReadableStream \u2502<\/p>\n

\u2502 4. decode chunks \u2500\u2500\u25ba parse SSE events \u2500\u2500\u25ba append tokens to UI \u2502<\/p>\n

\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518<\/p>\n

\u2502 HTTP (text\/event-stream)<\/p>\n

\u25bc<\/p>\n

\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510<\/p>\n

\u2502 Express Server \u2502<\/p>\n

\u2502 \u2502<\/p>\n

\u2502 sessions Map \u2500\u2500 { sessionId: { messages: […] } } \u2502<\/p>\n

\u2502 \u2502<\/p>\n

\u2502 POST \/chat \u2502<\/p>\n

\u2502 \u251c\u2500 load session history \u2502<\/p>\n

\u2502 \u251c\u2500 push user message \u2502<\/p>\n

\u2502 \u251c\u2500 set SSE headers + flushHeaders() \u2502<\/p>\n

\u2502 \u251c\u2500 openai.create({ stream: true, signal: controller.signal }) \u2502<\/p>\n

\u2502 \u251c\u2500 for await chunk \u2192 res.write(`data: ${token}nn`) \u2502<\/p>\n

\u2502 \u251c\u2500 accumulate fullResponse \u2192 push to session \u2502<\/p>\n

\u2502 \u2514\u2500 send { done: true } \u2192 res.end() \u2502<\/p>\n

\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518<\/p>\n

\u2502 HTTPS chunked transfer encoding<\/p>\n

\u25bc<\/p>\n

\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510<\/p>\n

\u2502 OpenAI API \u2502<\/p>\n

\u2502 gpt-4o-mini \u00b7 stream: true \u2502<\/p>\n

\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518<\/p>\n

The session store lives on the server. Every time the user sends a message, that full conversation history goes to OpenAI inside the messages array, which is how the model knows what was said earlier. The API itself remembers nothing between calls, so you carry the state.<\/p>\n

The mechanism is simpler than it sounds: open a regular HTTP response and never close it. The server keeps writing chunks into the socket as tokens arrive, and when the model finishes you call res.end() and the connection drops.<\/p>\n

Project Setup<\/h2>\n

mkdir nodejs-openai-streaming<\/p>\n

cd<\/span> nodejs-openai-streaming<\/p>\n

npm init -y<\/p>\n

npm install express openai cors dotenv<\/p>\n

mkdir public<\/p>\n

Add `«type»<\/span>: «module»<\/span>` to `package.json`:<\/p>\n

{<\/p>\n

«name»<\/span>: «nodejs-openai-streaming»<\/span>,<\/p>\n

«version»<\/span>: «1.0.0»<\/span>,<\/p>\n

«type»<\/span>: «module»<\/span>,<\/p>\n

«scripts»<\/span>: {<\/p>\n

«start»<\/span>: «node server.js»<\/span>,<\/p>\n

«dev»<\/span>: «node —watch server.js»<\/span><\/p>\n

},<\/p>\n

«dependencies»<\/span>: {<\/p>\n

«cors»<\/span>: «^2.8.6»<\/span>,<\/p>\n

«dotenv»<\/span>: «^16.6.1»<\/span>,<\/p>\n

«express»<\/span>: «^4.22.2»<\/span>,<\/p>\n

«openai»<\/span>: «^4.104.0»<\/span><\/p>\n

}<\/p>\n

}<\/p>\n

Create .env:<\/p>\n

OPENAI_API_KEY=your_api_key_here<\/p>\n

PORT=3000<\/p>\n

Project structure:<\/p>\n

nodejs-openai-streaming\/<\/p>\n

\u251c\u2500\u2500 server.js<\/p>\n

\u251c\u2500\u2500 public\/<\/p>\n

\u2502 \u2514\u2500\u2500 index.html<\/p>\n

\u251c\u2500\u2500 .env<\/p>\n

\u251c\u2500\u2500 .env.example<\/p>\n

\u251c\u2500\u2500 .gitignore<\/p>\n

\u2514\u2500\u2500 package.json<\/p>\n

Why SSE and Not WebSockets<\/h2>\n

People ask about WebSockets every time, usually because they sound more capable and therefore more correct.<\/p>\n

For AI chat, they usually are not. Responses travel one way, server to client, and SSE is built for exactly that. It runs over plain HTTP with no protocol upgrade and no special reverse proxy configuration, and it works through HTTP\/2 without touching a config file. I have shipped SSE to production multiple times and never once worried about browser support.<\/p>\n

WebSockets make sense when the client needs to push data back at the same frequency the server pushes it: gaming, shared whiteboards, live bidding. In an AI chat, the user types one message and sits there reading, so SSE fits the traffic pattern better.<\/p>\n

One genuine limitation of SSE worth knowing: the browser’s built-in EventSource API only supports GET requests. Since you need to post a message body, you use fetch with a ReadableStream reader instead. The SSE wire protocol is identical; you are just reading the stream manually rather than through the browser’s event wrapper.<\/p>\n

Session Management<\/h2>\n

The first chatbot I built on top of this API had a bug I could not figure out for two days. Users would ask a follow-up question and the model would answer like it was hearing from them for the first time, with no context or memory of the earlier turns.<\/p>\n

The API does not store anything between calls, so each request goes in cold. The messages array is how you fix that: pack the full conversation history into every request, user turns and assistant turns both, and the model responds as if it remembers the whole thread. Leave it out and you get that same broken experience I shipped by accident.<\/p>\n

const<\/span> sessions = new<\/span> Map<\/span>();<\/p>\n

const<\/span> SESSION_TTL_MS = 30<\/span> * 60<\/span> * 1000<\/span>; \/\/ 30 minutes<\/span><\/p>\n

function<\/span> getOrCreateSession<\/span>(sessionId<\/span>) <\/span>{<\/p>\n

if<\/span> (!sessions.has(sessionId)) {<\/p>\n

sessions.set(sessionId, {<\/p>\n

messages<\/span>: [],<\/p>\n

lastActive<\/span>: Date<\/span>.now(),<\/p>\n

});<\/p>\n

}<\/p>\n

const<\/span> session = sessions.get(sessionId);<\/p>\n

session.lastActive = Date<\/span>.now();<\/p>\n

return<\/span> session;<\/p>\n

}<\/p>\n

setInterval<\/span>(() => {<\/p>\n

const<\/span> now = Date<\/span>.now();<\/p>\n

for<\/span> (const<\/span> [id, session] of<\/span> sessions) {<\/p>\n

if<\/span> (now — session.lastActive > SESSION_TTL_MS) {<\/p>\n

sessions.delete(id);<\/p>\n

}<\/p>\n

}<\/p>\n

}, 10<\/span> * 60<\/span> * 1000<\/span>);<\/p>\n

The Map is fine at development scale and in single-process deployments. The lastActive timestamp gets bumped on every request. The cleanup interval runs every ten minutes and removes sessions that have gone quiet for half an hour.<\/p>\n

The hard limit of this approach is that a Node.js process restart wipes it entirely. If you run multiple instances behind a load balancer, each instance has its own Map, and a user whose requests hit different instances will lose their conversation mid-sentence. The fix is Redis: one client, same data structure, TTL handled natively. I am not going to build that here because it would double the length of this article without adding anything about streaming, but do not ship this to production without it.<\/p>\n

The Streaming Endpoint<\/h2>\n

Four things happen in sequence before tokens start flowing: input validation, SSE setup, abort handling, and the OpenAI call with its token loop. Each step has at least one non-obvious decision.<\/p>\n

SSE Setup<\/h3>\n

res.setHeader(‘Content-Type’<\/span>, ‘text\/event-stream’<\/span>);<\/p>\n

res.setHeader(‘Cache-Control’<\/span>, ‘no-cache’<\/span>);<\/p>\n

res.setHeader(‘Connection’<\/span>, ‘keep-alive’<\/span>);<\/p>\n

res.flushHeaders();<\/p>\n

flushHeaders() is the line I see missing in more streaming PRs than I can count. Skip it and Express holds onto the response headers until the first res.write() call. The browser opens the request, gets nothing back, sits there, and after a few seconds the whole thing looks like a hung request. You will spend an hour checking your OpenAI call, your async logic, and your middleware, and the fix is one line at the top. Call flushHeaders() right after setting the headers; nothing else changes.<\/p>\n

AbortController<\/h3>\n

const<\/span> controller = new<\/span> AbortController();<\/p>\n

res.on(‘close’<\/span>, () => {<\/p>\n

if<\/span> (!res.writableEnded) controller.abort();<\/p>\n

});<\/p>\n

One important detail: attach the close listener to res, not req. The req object fires close when the request body is fully consumed, which happens immediately after the POST body arrives and would abort the stream before it starts. res.on(‘close’) fires when the actual response connection drops, which is what you want. The !res.writableEnded guard prevents calling abort() after a clean finish.<\/p>\n

When the connection drops, controller.abort() propagates a cancellation signal to the OpenAI SDK and the for await loop exits on the next iteration. Without this, you keep pulling tokens from the API and writing them into a closed socket, which means you are spending money and achieving nothing.<\/p>\n

The OpenAI Streaming Call<\/h3>\n

const<\/span> stream = await<\/span> openai.chat.completions.create(<\/p>\n

{<\/p>\n

model<\/span>: ‘gpt-4o-mini’<\/span>,<\/p>\n

messages<\/span>: [<\/p>\n

{<\/p>\n

role<\/span>: ‘system’<\/span>,<\/p>\n

content<\/span>: ‘You are a helpful assistant. Be concise and clear.’<\/span>,<\/p>\n

},<\/p>\n

…session.messages,<\/p>\n

],<\/p>\n

stream<\/span>: true<\/span>,<\/p>\n

max_tokens: 800<\/span>,<\/p>\n

},<\/p>\n

{ signal<\/span>: controller.signal }<\/p>\n

);<\/p>\n

Set max_tokens. This is not optional. I left it uncapped during a testing session and watched my usage dashboard spike in real time. Left alone, the model will write a dissertation in response to a three-word question. I use 800 for chat, which works fine for most things, and you can tune it once you see what your users actually send.<\/p>\n

The signal parameter threads the AbortController through to the SDK. If the client disconnects, the SDK cancels the underlying HTTP request to OpenAI, not just the local loop.<\/p>\n

Token Loop and History<\/h3>\n

let<\/span> fullResponse = »<\/span>;<\/p>\n

for<\/span> await<\/span> (const<\/span> chunk of<\/span> stream) {<\/p>\n

if<\/span> (req.socket.destroyed) break<\/span>;<\/p>\n

const<\/span> token = chunk.choices[0<\/span>]?.delta?.content;<\/p>\n

if<\/span> (token) {<\/p>\n

fullResponse += token;<\/p>\n

res.write(`data: ${JSON<\/span>.stringify({ token })}<\/span>nn`<\/span>);<\/p>\n

}<\/p>\n

}<\/p>\n

if<\/span> (fullResponse) {<\/p>\n

session.messages.push({ role<\/span>: ‘assistant’<\/span>, content<\/span>: fullResponse });<\/p>\n

}<\/p>\n

Each chunk contains one token in delta.content. You concatenate them into fullResponse as they arrive, relay each one to the client, and save the complete string to the session once the loop ends.<\/p>\n

The double newline in res.write() is mandatory; it is how SSE signals the end of a single event. The browser buffers data until it sees nn before firing. Omit it and the client receives all tokens at once when the connection closes, which is the same broken behavior as not streaming at all.<\/p>\n

Do not save fullResponse if the stream was aborted. A truncated assistant reply in the session history will corrupt the conversation context, and the model’s next turn will be built on incomplete information.<\/p>\n

The Complete server.js<\/h2>\n

import<\/span> ‘dotenv\/config’<\/span>;<\/p>\n

import<\/span> express from<\/span> ‘express’<\/span>;<\/p>\n

import<\/span> cors from<\/span> ‘cors’<\/span>;<\/p>\n

import<\/span> OpenAI from<\/span> ‘openai’<\/span>;<\/p>\n

import<\/span> path from<\/span> ‘path’<\/span>;<\/p>\n

import<\/span> crypto from<\/span> ‘crypto’<\/span>;<\/p>\n

import<\/span> { fileURLToPath } from<\/span> ‘url’<\/span>;<\/p>\n

const<\/span> __filename = fileURLToPath(import<\/span>.meta.url);<\/p>\n

const<\/span> __dirname = path.dirname(__filename);<\/p>\n

const<\/span> app = express();<\/p>\n

const<\/span> openai = new<\/span> OpenAI({ apiKey<\/span>: process.env.OPENAI_API_KEY });<\/p>\n

app.use(cors());<\/p>\n

app.use(express.json());<\/p>\n

app.use(express.static(path.join(__dirname, ‘public’<\/span>)));<\/p>\n

const<\/span> sessions = new<\/span> Map<\/span>();<\/p>\n

const<\/span> SESSION_TTL_MS = 30<\/span> * 60<\/span> * 1000<\/span>;<\/p>\n

function<\/span> getOrCreateSession<\/span>(sessionId<\/span>) <\/span>{<\/p>\n

if<\/span> (!sessions.has(sessionId)) {<\/p>\n

sessions.set(sessionId, { messages<\/span>: [], lastActive<\/span>: Date<\/span>.now() });<\/p>\n

}<\/p>\n

const<\/span> session = sessions.get(sessionId);<\/p>\n

session.lastActive = Date<\/span>.now();<\/p>\n

return<\/span> session;<\/p>\n

}<\/p>\n

setInterval<\/span>(() =><\/span> {<\/p>\n

const<\/span> now = Date<\/span>.now();<\/p>\n

for<\/span> (const<\/span> [id, session] of<\/span> sessions) {<\/p>\n

if<\/span> (now — session.lastActive > SESSION_TTL_MS) sessions.delete(id);<\/p>\n

}<\/p>\n

}, 10<\/span> * 60<\/span> * 1000<\/span>);<\/p>\n

app.post(‘\/session’<\/span>, (req, res<\/span>) =><\/span> {<\/p>\n

const<\/span> sessionId = crypto.randomUUID();<\/p>\n

getOrCreateSession(sessionId);<\/p>\n

console<\/span>.log(`[\/session] created: ${sessionId}<\/span>`<\/span>);<\/p>\n

res.json({ sessionId });<\/p>\n

});<\/p>\n

app.delete(‘\/session\/:sessionId’<\/span>, (req, res<\/span>) =><\/span> {<\/p>\n

sessions.delete(req.params.sessionId);<\/p>\n

res.json({ ok<\/span>: true<\/span> });<\/p>\n

});<\/p>\n

app.post(‘\/chat’<\/span>, async<\/span> (req, res) => {<\/p>\n

const<\/span> { message, sessionId } = req.body;<\/p>\n

if<\/span> (!message || typeof<\/span> message !== ‘string’<\/span>) {<\/p>\n

return<\/span> res.status(400<\/span>).json({ error<\/span>: ‘message is required’<\/span> });<\/p>\n

}<\/p>\n

if<\/span> (!sessionId || typeof<\/span> sessionId !== ‘string’<\/span>) {<\/p>\n

return<\/span> res.status(400<\/span>).json({ error<\/span>: ‘sessionId is required’<\/span> });<\/p>\n

}<\/p>\n

const<\/span> session = getOrCreateSession(sessionId);<\/p>\n

session.messages.push({ role<\/span>: ‘user’<\/span>, content<\/span>: message });<\/p>\n

console<\/span>.log(`[\/chat] received:`<\/span>, { message, sessionId });<\/p>\n

res.setHeader(‘Content-Type’<\/span>, ‘text\/event-stream’<\/span>);<\/p>\n

res.setHeader(‘Cache-Control’<\/span>, ‘no-cache’<\/span>);<\/p>\n

res.setHeader(‘Connection’<\/span>, ‘keep-alive’<\/span>);<\/p>\n

res.flushHeaders();<\/p>\n

const<\/span> controller = new<\/span> AbortController();<\/p>\n

res.on(‘close’<\/span>, () =><\/span> {<\/p>\n

if<\/span> (!res.writableEnded) controller.abort();<\/p>\n

});<\/p>\n

let<\/span> fullResponse = »<\/span>;<\/p>\n

try<\/span> {<\/p>\n

const<\/span> stream = await<\/span> openai.chat.completions.create(<\/p>\n

{<\/p>\n

model<\/span>: ‘gpt-4o-mini’<\/span>,<\/p>\n

messages<\/span>: [<\/p>\n

{ role<\/span>: ‘system’<\/span>, content<\/span>: ‘You are a helpful assistant. Be concise and clear.’<\/span> },<\/p>\n

…session.messages,<\/p>\n

],<\/p>\n

stream<\/span>: true<\/span>,<\/p>\n

max_tokens<\/span>: 800<\/span>,<\/p>\n

},<\/p>\n

{ signal<\/span>: controller.signal }<\/p>\n

);<\/p>\n

for<\/span> await<\/span> (const<\/span> chunk of<\/span> stream) {<\/p>\n

if<\/span> (req.socket.destroyed) break<\/span>;<\/p>\n

const<\/span> token = chunk.choices[0<\/span>]?.delta?.content;<\/p>\n

if<\/span> (token) {<\/p>\n

fullResponse += token;<\/p>\n

res.write(`data: ${JSON<\/span>.stringify({ token })}<\/span>nn`<\/span>);<\/p>\n

}<\/p>\n

}<\/p>\n

if<\/span> (fullResponse) {<\/p>\n

session.messages.push({ role<\/span>: ‘assistant’<\/span>, content<\/span>: fullResponse });<\/p>\n

console<\/span>.log(`[stream complete] ${fullResponse.length}<\/span> chars`<\/span>);<\/p>\n

}<\/p>\n

res.write(`data: ${JSON<\/span>.stringify({ done: true<\/span> })}<\/span>nn`<\/span>);<\/p>\n

res.end();<\/p>\n

} catch<\/span> (err) {<\/p>\n

if<\/span> (err.name === ‘AbortError’<\/span> || err.name === ‘APIUserAbortError’<\/span> || err.message === ‘Request was aborted.’<\/span>) {<\/p>\n

res.end();<\/p>\n

return<\/span>;<\/p>\n

}<\/p>\n

console<\/span>.error(‘Stream error:’<\/span>, err.message);<\/p>\n

res.write(`data: ${JSON<\/span>.stringify({ error: err.message })}<\/span>nn`<\/span>);<\/p>\n

res.end();<\/p>\n

}<\/p>\n

});<\/p>\n

const<\/span> PORT = process.env.PORT || 3000<\/span>;<\/p>\n

app.listen(PORT, () =><\/span> console<\/span>.log(`Server running at http:\/\/localhost:${PORT}<\/span>`<\/span>));<\/p>\n

Building the Frontend<\/h2>\n

The client has two jobs: manage the session lifecycle and render the stream. The session lifecycle is simple (create on load, delete on «New Chat»), and the stream rendering is where most implementations get tripped up.<\/p>\n

<!DOCTYPE html<\/span>><\/span><\/p>\n

<html<\/span> lang<\/span>=«en»<\/span>><\/span><\/p>\n

<head<\/span>><\/span><\/p>\n

<meta<\/span> charset<\/span>=«UTF-8»<\/span>><\/span><\/p>\n

<meta<\/span> name<\/span>=«viewport»<\/span> content<\/span>=«width=device-width, initial-scale=1.0»<\/span>><\/span><\/p>\n

<title<\/span>><\/span>AI Streaming Chat<\/title<\/span>><\/span><\/p>\n

<style<\/span>><\/span><\/span><\/p>\n

* { box-sizing<\/span>: border-box; margin<\/span>: 0<\/span>; padding<\/span>: 0<\/span>; }<\/p>\n

body<\/span> {<\/p>\n

font-family<\/span>: -apple-system, BlinkMacSystemFont, ‘Segoe UI’<\/span>, Roboto, sans-serif;<\/p>\n

background<\/span>: #f0f2f5<\/span>;<\/p>\n

height<\/span>: 100vh<\/span>;<\/p>\n

display<\/span>: flex;<\/p>\n

flex-direction<\/span>: column;<\/p>\n

align-items<\/span>: center;<\/p>\n

padding<\/span>: 1.5rem<\/span> 1rem<\/span>;<\/p>\n

}<\/p>\n

.chat-wrapper<\/span> { width<\/span>: 100%<\/span>; max-width<\/span>: 700px<\/span>; display<\/span>: flex; flex-direction<\/span>: column; height<\/span>: 100%<\/span>; }<\/p>\n

h1<\/span> { font-size<\/span>: 1.2rem<\/span>; color<\/span>: #333<\/span>; margin-bottom<\/span>: 1rem<\/span>; }<\/p>\n

#messages<\/span> {<\/p>\n

flex<\/span>: 1<\/span>; overflow-y<\/span>: auto; background<\/span>: white; border<\/span>: 1px<\/span> solid #e0e0e0<\/span>;<\/p>\n

border-radius<\/span>: 10px<\/span>; padding<\/span>: 1rem<\/span>; display<\/span>: flex; flex-direction<\/span>: column; gap: 0.75rem<\/span>;<\/p>\n

}<\/p>\n

.message<\/span> {<\/p>\n

max-width<\/span>: 85%<\/span>; padding<\/span>: 0.65rem<\/span> 0.9rem<\/span>; border-radius<\/span>: 10px<\/span>;<\/p>\n

line-height<\/span>: 1.6<\/span>; font-size<\/span>: 0.93rem<\/span>; white-space<\/span>: pre-wrap; word-break<\/span>: break-word;<\/p>\n

}<\/p>\n

.user<\/span> { background<\/span>: #0070f3<\/span>; color<\/span>: white; align-self<\/span>: flex-end; }<\/p>\n

.assistant<\/span> { background<\/span>: #f1f1f1<\/span>; color<\/span>: #111<\/span>; align-self<\/span>: flex-start; }<\/p>\n

.input-row<\/span> { display<\/span>: flex; gap: 0.5rem<\/span>; margin-top<\/span>: 0.75rem<\/span>; align-items<\/span>: flex-end; }<\/p>\n

textarea<\/span> {<\/p>\n

flex<\/span>: 1<\/span>; padding<\/span>: 0.7rem<\/span>; border<\/span>: 1px<\/span> solid #ddd<\/span>; border-radius<\/span>: 8px<\/span>;<\/p>\n

font-size<\/span>: 0.93rem<\/span>; font-family<\/span>: inherit; resize<\/span>: none; height<\/span>: 52px<\/span>; overflow-y<\/span>: auto;<\/p>\n

}<\/p>\n

button<\/span> { padding<\/span>: 0.65rem<\/span> 1.2rem<\/span>; border<\/span>: none; border-radius<\/span>: 8px<\/span>; font-size<\/span>: 0.93rem<\/span>; cursor<\/span>: pointer; }<\/p>\n

#send<\/span> { background<\/span>: #0070f3<\/span>; color<\/span>: white; }<\/p>\n

#send<\/span>:disabled<\/span> { background<\/span>: #aaa<\/span>; cursor<\/span>: not-allowed; }<\/p>\n

#clear<\/span> { background<\/span>: #f1f1f1<\/span>; color<\/span>: #555<\/span>; }<\/p>\n

.status<\/span> { font-size<\/span>: 0.78rem<\/span>; color<\/span>: #aaa<\/span>; margin-top<\/span>: 0.3rem<\/span>; min-height<\/span>: 1rem<\/span>; }<\/p>\n

<\/style<\/span>><\/span><\/p>\n

<\/head<\/span>><\/span><\/p>\n

<body<\/span>><\/span><\/p>\n

<div<\/span> class<\/span>=«chat-wrapper»<\/span>><\/span><\/p>\n

<h1<\/span>><\/span>AI Chat<\/h1<\/span>><\/span><\/p>\n

<div<\/span> id<\/span>=«messages»<\/span>><\/span><\/div<\/span>><\/span><\/p>\n

<p<\/span> class<\/span>=«status»<\/span> id<\/span>=«status»<\/span>><\/span><\/p<\/span>><\/span><\/p>\n

<div<\/span> class<\/span>=«input-row»<\/span>><\/span><\/p>\n

<textarea<\/span> id<\/span>=«input»<\/span> placeholder<\/span>=«Ask something… (Ctrl+Enter to send)»<\/span>><\/span><\/textarea<\/span>><\/span><\/p>\n

<button<\/span> id<\/span>=«send»<\/span>><\/span>Send<\/button<\/span>><\/span><\/p>\n

<button<\/span> id<\/span>=«clear»<\/span>><\/span>New Chat<\/button<\/span>><\/span><\/p>\n

<\/div<\/span>><\/span><\/p>\n

<\/div<\/span>><\/span><\/p>\n

<script<\/span>><\/span><\/span><\/p>\n

let<\/span> sessionId = null<\/span>;<\/p>\n

async<\/span> function<\/span> initSession<\/span>(<\/span>) <\/span>{<\/p>\n

const<\/span> res = await<\/span> fetch(‘\/session’<\/span>, { method<\/span>: ‘POST’<\/span> });<\/p>\n

const<\/span> data = await<\/span> res.json();<\/p>\n

sessionId = data.sessionId;<\/p>\n

}<\/p>\n

function<\/span> appendMessage<\/span>(role, text<\/span>) <\/span>{<\/p>\n

const<\/span> div = document<\/span>.createElement(‘div’<\/span>);<\/p>\n

div.className = `message ${role}<\/span>`<\/span>;<\/p>\n

div.textContent = text;<\/p>\n

document<\/span>.getElementById(‘messages’<\/span>).appendChild(div);<\/p>\n

div.scrollIntoView({ behavior<\/span>: ‘smooth’<\/span> });<\/p>\n

return<\/span> div;<\/p>\n

}<\/p>\n

document<\/span>.getElementById(‘send’<\/span>).addEventListener(‘click’<\/span>, async<\/span> () => {<\/p>\n

const<\/span> input = document<\/span>.getElementById(‘input’<\/span>);<\/p>\n

const<\/span> message = input.value.trim();<\/p>\n

if<\/span> (!message || !sessionId) return<\/span>;<\/p>\n

const<\/span> sendBtn = document<\/span>.getElementById(‘send’<\/span>);<\/p>\n

const<\/span> status = document<\/span>.getElementById(‘status’<\/span>);<\/p>\n

input.value = »<\/span>;<\/p>\n

sendBtn.disabled = true<\/span>;<\/p>\n

status.textContent = ‘Generating…’<\/span>;<\/p>\n

appendMessage(‘user’<\/span>, message);<\/p>\n

const<\/span> assistantBubble = appendMessage(‘assistant’<\/span>, »<\/span>);<\/p>\n

try<\/span> {<\/p>\n

const<\/span> response = await<\/span> fetch(‘\/chat’<\/span>, {<\/p>\n

method<\/span>: ‘POST’<\/span>,<\/p>\n

headers<\/span>: { ‘Content-Type’<\/span>: ‘application\/json’<\/span> },<\/p>\n

body<\/span>: JSON<\/span>.stringify({ message, sessionId }),<\/p>\n

});<\/p>\n

if<\/span> (!response.ok) {<\/p>\n

const<\/span> err = await<\/span> response.json();<\/p>\n

assistantBubble.textContent = `Error: ${err.error}<\/span>`<\/span>;<\/p>\n

return<\/span>;<\/p>\n

}<\/p>\n

const<\/span> reader = response.body.getReader();<\/p>\n

const<\/span> decoder = new<\/span> TextDecoder();<\/p>\n

while<\/span> (true<\/span>) {<\/p>\n

const<\/span> { value, done } = await<\/span> reader.read();<\/p>\n

if<\/span> (done) break<\/span>;<\/p>\n

const<\/span> lines = decoder.decode(value, { stream<\/span>: true<\/span> }).split(‘\\n’<\/span>);<\/p>\n

for<\/span> (const<\/span> line of<\/span> lines) {<\/p>\n

if<\/span> (!line.startsWith(‘data: ‘<\/span>)) continue<\/span>;<\/p>\n

try<\/span> {<\/p>\n

const<\/span> data = JSON<\/span>.parse(line.slice(6<\/span>));<\/p>\n

if<\/span> (data.done) break<\/span>; \/\/ exits inner for loop; outer while loop ends on next read() \u2192 done: true<\/span><\/p>\n

if<\/span> (data.error) { assistantBubble.textContent += `\\n[Error: ${data.error}<\/span>]`<\/span>; break<\/span>; }<\/p>\n

if<\/span> (data.token) {<\/p>\n

assistantBubble.textContent += data.token;<\/p>\n

assistantBubble.scrollIntoView({ behavior<\/span>: ‘smooth’<\/span> });<\/p>\n

}<\/p>\n

} catch<\/span> { \/* split chunk; wait for next read *\/<\/span> }<\/p>\n

}<\/p>\n

}<\/p>\n

status.textContent = »<\/span>;<\/p>\n

} catch<\/span> (err) {<\/p>\n

assistantBubble.textContent = `Request failed: ${err.message}<\/span>`<\/span>;<\/p>\n

} finally<\/span> {<\/p>\n

sendBtn.disabled = false<\/span>;<\/p>\n

status.textContent = »<\/span>;<\/p>\n

input.focus();<\/p>\n

}<\/p>\n

});<\/p>\n

document<\/span>.getElementById(‘clear’<\/span>).addEventListener(‘click’<\/span>, async<\/span> () => {<\/p>\n

if<\/span> (!sessionId) return<\/span>;<\/p>\n

await<\/span> fetch(`\/session\/${sessionId}<\/span>`<\/span>, { method<\/span>: ‘DELETE’<\/span> });<\/p>\n

await<\/span> initSession();<\/p>\n

document<\/span>.getElementById(‘messages’<\/span>).innerHTML = »<\/span>;<\/p>\n

document<\/span>.getElementById(‘status’<\/span>).textContent = »<\/span>;<\/p>\n

});<\/p>\n

document<\/span>.getElementById(‘input’<\/span>).addEventListener(‘keydown’<\/span>, (e<\/span>) =><\/span> {<\/p>\n

if<\/span> (e.key === ‘Enter’<\/span> && e.ctrlKey) document<\/span>.getElementById(‘send’<\/span>).click();<\/p>\n

});<\/p>\n

initSession();<\/p>\n

<\/script<\/span>><\/span><\/p>\n

<\/body<\/span>><\/span><\/p>\n

<\/html<\/span>><\/span><\/p>\n

The try\/catch inside the inner loop handles a real edge case: SSE events occasionally span two network reads. When that happens, JSON.parse throws on the incomplete fragment, and catching it so the next read can deliver the rest keeps the stream alive. Without the catch, the whole stream crashes on what is essentially normal network behavior.<\/p>\n

Running It<\/h2>\n

npm run dev<\/p>\n

The terminal confirms the server started:<\/p>\n

Server running at http:\/\/localhost:3000<\/p>\n

Hit http:\/\/localhost:3000 in your browser and type something. The terminal should print:<\/p>\n

[\/session] created: 2dfb2586-4601-4781-932d-cb3dcca5d5a5<\/p>\n

[\/chat] received: {<\/p>\n

message: ‘Do you know about StackAbuse?’<\/span>,<\/p>\n

sessionId: ‘2dfb2586-4601-4781-932d-cb3dcca5d5a5’<\/span><\/p>\n

}<\/p>\n

[stream complete] 304 chars<\/p>\n

In the browser, you will see the response build out token by token, which means the streaming is working. The interface looks like this with a live response:<\/p>\n

Send a follow-up that references the first answer and the model will remember it. The session history is included in every request, so context carries across turns. Click «New Chat» and the server deletes the session and creates a fresh one. The model starts over with no memory of the previous conversation.<\/p>\n

To inspect the raw stream, open DevTools, go to Network, find the \/chat request, and look at the EventStream tab. You will see each data: event arrive individually, which is the most useful thing I know of for debugging streaming behavior that looks correct in the UI but is not working the way you think.<\/p>\n

Production Gaps to Address Before Launch<\/h2>\n

Before you ship this, know what the implementation does not handle. I have seen developers take tutorial code and deploy it unchanged.<\/p>\n

Context overflow.<\/strong> Every request sends the full session history. gpt-4o-mini has a 128,000-token context window, which is large enough that most chat sessions will never hit it. But a session running for hours will eventually approach the limit. When it does, the API returns a context_length_exceeded error. The standard fix is a sliding window: keep the system prompt, trim old messages from the front of the history, optionally run a summarization call to compress old context before it falls off. This is a real engineering problem and there is no one-size answer.<\/p>\n

Rate limiting.<\/strong> There is none here, so one client can exhaust your monthly OpenAI quota. Add express-rate-limit before this goes anywhere near the internet.<\/p>\n

Authentication.<\/strong> The \/chat endpoint accepts requests from anyone. In a real deployment, you need session-based auth or JWT validation before the route handler runs. Without it, anyone who knows your URL is spending your API credits.<\/p>\n

Multi-process sessions.<\/strong> Covered above. Redis or equivalent before you scale horizontally.<\/p>\n

Conclusion<\/h2>\n

I have shipped versions of this exact setup to production twice now, across different companies and products, with the same core pattern: session store on the server, SSE for transport, and AbortController to stop wasting money on disconnected clients. The gaps I listed are not hypothetical warnings; I hit most of them myself before figuring out the fixes.<\/p>\n

The part that trips most developers is not the OpenAI API itself, but the plumbing around it: getting tokens to the browser the moment they arrive, keeping the connection alive without burning credits on disconnected clients, and maintaining conversation context without letting it grow unbounded. That is where the real work sits. Hopefully this article saved you the afternoon I spent learning most of it the hard way.<\/p>\n

The full code is at github.com\/ziaongit\/nodejs-openai-streaming<\/a>. Clone it, break it, and make it your own.<\/p>","protected":false},"excerpt":{"rendered":"

I almost shipped a bad AI integration once. The feature worked technically: the model answered questions correctly, the UI looked clean, and everyone nodded along in our internal demo. Then we put real users on it, and within 48 hours a product manager Slack’d me asking why 60% of sessions … <\/p>\n

Read More<\/a><\/div>\n","protected":false},"author":0,"featured_media":900,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-899","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-news"],"_links":{"self":[{"href":"https:\/\/imcodinggenius.com\/index.php?rest_route=\/wp\/v2\/posts\/899"}],"collection":[{"href":"https:\/\/imcodinggenius.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/imcodinggenius.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/imcodinggenius.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=899"}],"version-history":[{"count":0,"href":"https:\/\/imcodinggenius.com\/index.php?rest_route=\/wp\/v2\/posts\/899\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/imcodinggenius.com\/index.php?rest_route=\/wp\/v2\/media\/900"}],"wp:attachment":[{"href":"https:\/\/imcodinggenius.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=899"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/imcodinggenius.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=899"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/imcodinggenius.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=899"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}