Technical Guide · Stadium Assistant

Solution for responses that do not meet the required parameters

Walking Skeleton correction: identify the source of the 503s, fix the measurement, reduce latency, stabilize the OpenAI Responses API, and validate the p95 criterion again.

Walking Skeleton AWS us-east-1 OpenAI Responses API

11solution steps

20validation requests

≤1500ms p95 target

Original prompt

Exact point in the chat

“Okay, now tell me step by step and in detail how to solve the problem of the responses that do not meet the required parameters.”

Work objective

Fix availability and latency before closing the Walking Skeleton

The objective is not only to get all 20 requests to finish. For the test to be considered passed, the report must correctly distinguish between successful and failed requests and evaluate latency only with valid data.

The exit criterion we must achieve is:

Expected final criterion

total = 20
successful = 20
failed = 0
successRatePct = 100
successfulP95Ms <= 1500
overallTargetMet = true

Important: a 503 must never be counted as if it were a valid response when calculating successful p95. First eliminate the cause of the 503; then optimize latency.

Problem 1 — Availability

HTTP 503 responses appeared during the burst. We need to determine whether the source is API Gateway, Lambda, or the call to OpenAI.

Problem 2 — Latency

Complete responses on the direct OpenAI route can exceed the 1500 ms target. Optimization must be measured separately from errors.

Step 1

Identify exactly where the 503s are coming from

Do not change the model or performance code yet. First confirm the source of the error.

Completed

Open PowerShell in the DEV environment and first confirm that the Lambda function is not being limited by concurrency.

Lambda reserved concurrency

aws lambda get-function-concurrency `
  --function-name sa-dev-orchestrator-chat `
  --region us-east-1 `
  --profile sa-dev

Then review the function configuration:

Lambda configuration

aws lambda get-function-configuration `
  --function-name sa-dev-orchestrator-chat `
  --region us-east-1 `
  --profile sa-dev `
  --query "{State:State,LastUpdateStatus:LastUpdateStatus,Timeout:Timeout,MemorySize:MemorySize,Runtime:Runtime}"

Review the Lambda logs for the period when you ran the test:

CloudWatch · Lambda

aws logs tail "/aws/lambda/sa-dev-orchestrator-chat" `
  --since 30m `
  --region us-east-1 `
  --profile sa-dev

And review the API Gateway logs:

CloudWatch · API Gateway

aws logs tail "/aws/apigateway/sa-dev-http-api" `
  --since 30m `
  --region us-east-1 `
  --profile sa-dev

✓

What we are looking for

Throttling messages, timeouts, integration 5xx errors, or an error coming from the OpenAI call. We must not assume the source without evidence.

Step 2

Fix the report for `scripts/test-api-burst.mjs`

The script must separate successes and failures and calculate p50/p95 over successful responses.

Completed

Replace the contents of:

File

scripts/test-api-burst.mjs

with a version that explicitly reports successful, failed, successRatePct, successfulP50Ms, successfulP95Ms and overallTargetMet.

scripts/test-api-burst.mjs

const args = process.argv.slice(2);

function getArg(name, fallback) {
  const index = args.indexOf(`--${name}`);
  return index >= 0 && args[index + 1] !== undefined
    ? args[index + 1]
    : fallback;
}

const url = getArg(
  "url",
  process.env.CHAT_URL ||
    "https://w7jtrco599.execute-api.us-east-1.amazonaws.com/dev/chat"
);

const total = Number(getArg("total", "20"));
const concurrency = Number(getArg("concurrency", "5"));
const delayMs = Number(getArg("delay-ms", "200"));
const message = getArg("message", "Hola");
const targetP95Ms = Number(getArg("target-p95-ms", "1500"));

function percentile(values, p) {
  if (!values.length) return null;
  const sorted = [...values].sort((a, b) => a - b);
  const index = Math.ceil((p / 100) * sorted.length) - 1;
  return sorted[Math.max(0, index)];
}

const results = [];
let nextIndex = 0;

async function sendOne(index) {
  if (delayMs > 0 && index > 0) {
    await new Promise((resolve) => setTimeout(resolve, delayMs));
  }

  const startedAt = performance.now();
  let status = 0;
  let body = "";
  let error = null;

  try {
    const response = await fetch(url, {
      method: "POST",
      headers: { "content-type": "application/json" },
      body: JSON.stringify({
        message,
        session_id: `burst-${Date.now()}-${index}`,
        locale: "es"
      })
    });

    status = response.status;
    body = await response.text();
  } catch (err) {
    error = err instanceof Error ? err.message : String(err);
  }

  const latencyMs = Math.round(performance.now() - startedAt);
  const ok = status >= 200 && status < 300 && !error;

  results.push({
    index: index + 1,
    ok,
    status,
    latencyMs,
    error,
    body: body.slice(0, 300)
  });

  console.log(
    `[${index + 1}/${total}] status=${status || "ERR"} ` +
    `latency_ms=${latencyMs} ok=${ok}`
  );
}

async function worker() {
  while (true) {
    const index = nextIndex++;
    if (index >= total) return;
    await sendOne(index);
  }
}

await Promise.all(
  Array.from({ length: Math.min(concurrency, total) }, () => worker())
);

const successfulResults = results.filter((item) => item.ok);
const failedResults = results.filter((item) => !item.ok);
const successfulLatencies = successfulResults.map((item) => item.latencyMs);

const successful = successfulResults.length;
const failed = failedResults.length;
const successRatePct = Number(((successful / total) * 100).toFixed(2));
const successfulP50Ms = percentile(successfulLatencies, 50);
const successfulP95Ms = percentile(successfulLatencies, 95);
const successfulMaxMs = successfulLatencies.length
  ? Math.max(...successfulLatencies)
  : null;

const overallTargetMet =
  total === 20 &&
  successful === 20 &&
  failed === 0 &&
  successRatePct === 100 &&
  successfulP95Ms !== null &&
  successfulP95Ms <= targetP95Ms;

const statusCounts = results.reduce((acc, item) => {
  const key = String(item.status || "ERR");
  acc[key] = (acc[key] || 0) + 1;
  return acc;
}, {});

console.log("\n=== RESULT ===");
console.log(JSON.stringify({
  url,
  message,
  total,
  concurrency,
  delayMs,
  successful,
  failed,
  successRatePct,
  successfulP50Ms,
  successfulP95Ms,
  successfulMaxMs,
  targetP95Ms,
  overallTargetMet,
  statusCounts,
  failures: failedResults
}, null, 2));

process.exit(overallTargetMet ? 0 : 1);

The important correction: the response times of 503 responses are not mixed with the latency distribution of successful responses. The test fails if there is a single unsuccessful response or if the successful p95 exceeds the target.

Step 3

Configure `reasoning.effort = "none"`

For “Hola” you do not need additional reasoning. That work increases latency without adding value.

Completed

In the OpenAI Responses API call, configure:

Responses API

reasoning: {
  effort: "none"
}

The idea is that a simple Walking Skeleton request should not consume reasoning time it does not need.

Step 4

Configure `text.verbosity = "low"`

The Walking Skeleton response should be short. We do not want “Hola” to produce a long response.

Completed

Responses API

text: {
  verbosity: "low"
}

For this test, the response should remain one or two sentences. The objective here is to validate the end-to-end pipeline, not generate long content.

Step 5

Reduce `max_output_tokens`

Limit generation so a simple response has no room to grow unnecessarily.

Completed

Output limit

max_output_tokens: 80

For the Walking Skeleton, 80 tokens are enough for a very short response and reduce generation time compared with much higher limits.

Step 6

Avoid hidden retries and control the timeout

An automatic retry can turn a fast error into an extremely slow response.

Completed

Configure the OpenAI client explicitly:

OpenAI client

const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
  maxRetries: 0,
  timeout: 12000
});

For this phase we want to observe the real error. If there is a 429, 5xx, or OpenAI timeout, it must remain visible in the logs and not be hidden behind multiple SDK retries.

Step 7

Improve logging for the OpenAI call

We need to know whether a Lambda failure really comes from OpenAI and what type of failure it was.

Completed

Do not log the API key or sensitive content. Log only the operational information needed for diagnosis.

Safe logging example

try {
  const startedOpenAI = Date.now();

  const response = await openai.responses.create({
    model: process.env.OPENAI_MODEL,
    reasoning: { effort: "none" },
    text: { verbosity: "low" },
    max_output_tokens: 80,
    instructions:
      "Responde en el idioma del usuario. " +
      "Para un saludo, responde brevemente en una o dos frases.",
    input: message
  });

  console.log(JSON.stringify({
    event: "openai_call_ok",
    trace_id,
    model: process.env.OPENAI_MODEL,
    openai_latency_ms: Date.now() - startedOpenAI,
    response_id: response.id,
    route: "direct"
  }));

  return response.output_text;

} catch (error) {
  console.error(JSON.stringify({
    event: "openai_call_error",
    trace_id,
    name: error?.name,
    status: error?.status,
    code: error?.code,
    type: error?.type,
    message: error?.message,
    request_id: error?.request_id,
    route: "direct"
  }));

  throw error;
}

✓

What you must be able to distinguish

Timeout, rate limit, OpenAI 5xx error, application error, or integration error. Without this separation, we will not know what to optimize.

Step 8

Create a `fast_path` for the Walking Skeleton greeting

The original criterion tests “Hola”. You do not need to call OpenAI to resolve a completely deterministic greeting.

Completed

Before executing the direct OpenAI route, normalize the message and respond locally to simple greetings.

Fast path

function normalizeMessage(value = "") {
  return value
    .trim()
    .toLocaleLowerCase("es")
    .replace(/[¡!¿?.,]/g, "");
}

function getFastPathResponse(message, locale = "es") {
  const normalized = normalizeMessage(message);

  const greetings = new Set([
    "hola",
    "hello",
    "hi",
    "buenas",
    "buen dia",
    "buenos dias",
    "buenas tardes",
    "buenas noches"
  ]);

  if (!greetings.has(normalized)) {
    return null;
  }

  if (locale === "en") {
    return "Hello! I’m the Stadium Assistant. How can I help you?";
  }

  return "¡Hola! Soy el Stadium Assistant. ¿En qué puedo ayudarte?";
}

In the handler:

Use fast_path before OpenAI

const fastPathResponse = getFastPathResponse(message, locale);

if (fastPathResponse) {
  const latencyMs = Date.now() - startedAt;

  console.log(JSON.stringify({
    trace_id,
    latency_ms: latencyMs,
    status: 200,
    route: "fast_path"
  }));

  return {
    statusCode: 200,
    headers: { "content-type": "application/json" },
    body: JSON.stringify({
      userMessage: message,
      assistantResponse: fastPathResponse,
      trace_id
    })
  };
}

Why it applies: the objective of the Walking Skeleton is to validate that the Web/API Gateway/Lambda/response/logs path works within the target. A fixed greeting is deterministic and does not require model inference.

Step 9

Package again and deploy to Lambda

After modifying the code, you must build deployment.zip again and upload it to the DEV function.

Completed

First run the existing local tests:

Local tests

npm run test:health
npm run test:injection
npm run test:local

Generate the deployment package again using the project procedure. If your packaging script is already configured:

Packaging

npm run package

Then update the Lambda function:

ZIP deployment

aws lambda update-function-code `
  --function-name sa-dev-orchestrator-chat `
  --zip-file fileb://deployment.zip `
  --region us-east-1 `
  --profile sa-dev

Wait for the update to finish:

Wait for deployment

aws lambda wait function-updated `
  --function-name sa-dev-orchestrator-chat `
  --region us-east-1 `
  --profile sa-dev

deployment.zip: do not unzip it to upload it to Lambda. Lambda receives the ZIP as the code package.

Step 10

Repeat the tests by concurrency level

Do not jump directly to maximum load. Increase concurrency gradually to see where the problem appears.

Completed

Test A — concurrency 1

20 requests · 1 concurrent

node .\scripts\test-api-burst.mjs `
  --url "https://w7jtrco599.execute-api.us-east-1.amazonaws.com/dev/chat" `
  --total 20 `
  --concurrency 1 `
  --delay-ms 200 `
  --message "Hola"

Test B — concurrency 5

20 requests · 5 concurrent

node .\scripts\test-api-burst.mjs `
  --url "https://w7jtrco599.execute-api.us-east-1.amazonaws.com/dev/chat" `
  --total 20 `
  --concurrency 5 `
  --delay-ms 200 `
  --message "Hola"

Test C — concurrency 10

20 requests · 10 concurrent

node .\scripts\test-api-burst.mjs `
  --url "https://w7jtrco599.execute-api.us-east-1.amazonaws.com/dev/chat" `
  --total 20 `
  --concurrency 10 `
  --delay-ms 200 `
  --message "Hola"

Test D — concurrency 20

20 requests · 20 concurrent

node .\scripts\test-api-burst.mjs `
  --url "https://w7jtrco599.execute-api.us-east-1.amazonaws.com/dev/chat" `
  --total 20 `
  --concurrency 20 `
  --delay-ms 200 `
  --message "Hola"

Record the following in each test:

successful.
failed.
successRatePct.
successfulP50Ms.
successfulP95Ms.
successfulMaxMs.
overallTargetMet.
HTTP status code distribution.

✓

Result required to close the original criterion

20/20 successful, 0 failed, successRatePct 100, successful p95 ≤ 1500 ms, and overallTargetMet=true.

Step 11

If it still fails: review OpenAI limits and Fast mode / Priority

This step is performed only after you have evidence that AWS and the code are working correctly.

Completed

If the logs show 429, rate limit, or OpenAI saturation, review your project limits and concurrency pattern. Do not increase concurrency blindly.

If the direct model route remains stable but does not meet the latency target, treat that measurement as a metric separate from the greeting criterion. For the Walking Skeleton, the fast_path allows the pipeline to be validated without spending a full inference on a deterministic input.

Optional: if your account/project has a priority or fast processing mode, it can be evaluated later. It must not replace correcting the code, logs, or measurement.

Closeout

Evidence you must save

Lambda logs with no throttling/unexplained 503 errors. API Gateway logs with no integration 5xx errors. New report from test-api-burst.mjs. successful = 20 and failed = 0. successRatePct = 100. successfulP95Ms <= 1500 for “Hola”. overallTargetMet = true. Final screenshot/record for Walking Skeleton evidence.

Solution progress

Complete the checklist to confirm that the solution has been validated.

Exact work order

Do not mix diagnosis and optimization

Identify the source of the 503s.
Fix the script report.
Configure reasoning.effort = "none".
Configure text.verbosity = "low".
Reduce max_output_tokens.
Improve OpenAI logs.
Package and deploy again.
Repeat the tests and save evidence.

Expected result

Solution for responses that do not meet the required parameters

Exact point in the chat

Fix availability and latency before closing the Walking Skeleton

Problem 1 — Availability

Problem 2 — Latency

Identify exactly where the 503s are coming from

Fix the report for scripts/test-api-burst.mjs

Configure reasoning.effort = "none"

Configure text.verbosity = "low"

Reduce max_output_tokens

Avoid hidden retries and control the timeout

Improve logging for the OpenAI call

Create a fast_path for the Walking Skeleton greeting

Package again and deploy to Lambda

Repeat the tests by concurrency level

Test A — concurrency 1

Test B — concurrency 5

Test C — concurrency 10

Test D — concurrency 20

If it still fails: review OpenAI limits and Fast mode / Priority

Evidence you must save

Do not mix diagnosis and optimization

The Walking Skeleton is stable, measurable, and has enough evidence to separate availability, latency, and execution path.

Fix the report for `scripts/test-api-burst.mjs`

Configure `reasoning.effort = "none"`

Configure `text.verbosity = "low"`

Reduce `max_output_tokens`

Create a `fast_path` for the Walking Skeleton greeting