# GenAI Engineer Interview Questions: RAG, Evals and Cost
GenAI engineering interviews reward the candidate who measures before fixing. Every question below has a wrong answer that sounds right.
11 min read · Advanced · Lesson 5 of 7 in Interviews and careers · Updated 2026-10-02 · By Akash Das
Canonical: https://akash-coded.github.io/aws-bedrock-agentcore-strands/learn/genai-engineer-interview-questions/
> **In short.** GenAI engineer interviews test whether you can make a model-backed system
> measurably right, fast and affordable. You do it by separating retrieval failures from generation
> failures, evaluating against your own data, splitting latency and cost into the parts you control, and
> designing for the model being tricked. These ten questions each come with the framework, a strong
> answer, the follow-up that finds the limit of your experience, and the red flag.
**In this lesson** you'll practise:
- the diagnostic questions that separate a GenAI engineer from someone who has used an API;
- a framework and a strong answer for each, with the follow-up that tests it;
- the research behind three of the answers, so you can cite it rather than assert it.
## Sound familiar?
- Your fix for every quality problem is a bigger model or a longer prompt.
- You know the latency, and not which part of the system it comes from.
- Your evaluation set was written by the same model it evaluates.
Each question below has an answer that sounds right and is wrong; the strong answer measures first.
## What does a GenAI engineer interview test?
**Whether you measure before you fix.** Expect coding (often an LLM call with tools or retrieval), a
system design for an LLM feature, and detailed questions on your own systems. Those are where
candidates separate: the strong ones can say what their system costs per task, how they know it is right,
and which failure they found in production and what now prevents it.
## Quality
### Q1 · "Your RAG system gives wrong answers. How do you find out whether retrieval or generation is at fault?"
**Tests:** diagnosis before remedy · **Framework:** the grounding triangle: retrieved, cited, verified
What a strong answer covers
- **Build a labelled set with gold passages**, then measure the two halves separately: **retrieval recall
at k** (is the needed passage in the top k?) and, for cases where it was, **faithfulness** (does the
answer follow from the passage?).
- **Bucket the failures**, because the fixes differ: chunking by meaning rather than by size, hybrid
keyword-plus-vector search, a reranker, query rewriting, or, for generation, citations per claim and
abstaining when support is missing.
- **The insight:** raising k often lowers accuracy. Models use information in the middle of a long context
less reliably than at the edges, so extra passages dilute the right one. Measure accuracy against k; it
peaks and then falls.
**The follow-up:** "Recall is 95% and answers are still wrong." → a generation failure: check faithfulness,
position of the passage, conflicting passages, and whether the prompt allows the model to decline.
**Red flag:** "use a bigger model" before measuring either half.
What a strong answer covers
- **Before launch**: a golden set from real inputs, labelled by people, sliced by kind of case, including
cases the system fails and cases it should decline; a checker per kind of step; a CI gate per slice
reported with lower bounds.
- **Model judges, calibrated**: a judge from a different model family, a rubric, and a measured agreement
rate with human labels. Judges are known to favour the first answer shown, longer answers, and their own
model family's outputs: randomise order and control for length.
- **After launch**: a shadow run against the current process, then a canary, then a sampled human review
and a watch on the mix of outputs for drift.
- **The insight:** a golden set built from cases the system already passes is a mirror; it can only go down.
**The follow-up:** "How big should the set be?" → sized per slice by how close the score is to its bar:
the cases needed grow with the square of the gap.
**Red flag:** public benchmarks as the main evidence.
What a strong answer covers
- **Retrieval for knowledge** that changes or must be cited.
- **Prompting and examples for behaviour** you can describe.
- **Fine-tuning for consistency at scale** (format, style, a narrow task) or to distil a large model's
behaviour into a smaller, cheaper, faster one, once you have enough good examples and a gap that
prompting did not close.
- **Not for facts**: fine-tuned knowledge goes stale silently and cannot be cited.
- **The insight:** fine-tuning moves cost from inference to data and operations: every base-model update
means re-training and re-evaluating. Measure the gap it closes before paying that tax.
**The follow-up:** "The fine-tuned model is better on the benchmark and worse in production." → the
benchmark is not your distribution; evaluate on your own slices.
**Red flag:** "fine-tune it on our documents so it knows our policies."
What a strong answer covers
- **Measure the split first.** Model time: cap output tokens, stream, use a faster model on easy routes,
cache a long stable prefix. Tool time: parallelise independent calls, cache, set timeouts. Orchestration:
fewer turns; route known paths to a workflow.
- **Ask which clock the requirement is about**: time to first token or total time. Streaming the first
token in 400 ms can make an eight-second answer feel fast.
- **The insight:** most latency programmes optimise the model clock because it is visible, while the
orchestration clock (turns multiplied by round trips) is the one that dominates agentic systems.
**The follow-up:** "Streaming is not allowed; the output is JSON." → then fewer turns, smaller outputs,
parallel tools, and a faster model per slice where its bar still holds.
**Red flag:** "use a faster model" without a measurement.
What a strong answer covers
- **Calls per task first**: it multiplies the instruction, schema, history and retrieval tokens on every turn.
- **Then the taxes**: retrieval (top-k set by habit), schema (tool definitions sent on every call), history
(unbounded buffers), reasoning tokens after a prompt change.
- **Then the four signatures**: tokens per call, share on the expensive tier, cache hit ratio, attempts per case.
- **The insight:** a timestamp, request ID or user name near the top of a system prompt silently destroys
prompt caching, because caches match an exact prefix. Put stable content first and variables last.
**The follow-up:** "Caching is on and the hit rate is 3%." → find what varies in the prefix, and check the
provider's minimum cacheable length and cache lifetime.
**Red flag:** "negotiate a discount."
What a strong answer covers
- **Define slices and bars first**, then test two or three candidates on the golden set.
- **Route by slice**: the cheapest model that clears each slice's bar, a stronger one for the rest.
- **Pin versions**, log which model answered every response, and plan the fallback: a fallback to a larger
model is a cost cliff, and a fallback to a smaller one is a silent quality drop.
- **Re-evaluate on every model update**, as a change like any other.
- **The insight:** routing by difficulty usually saves more than any single model switch, and logging the
answering model is the only way to detect silent failover.
**The follow-up:** "A new model tops the leaderboard." → run it through the harness per slice; adopt it where
it clears the bar more cheaply.
**Red flag:** "the best model on the leaderboard."
What a strong answer covers
- **Treat every retrieved document and tool output as untrusted data**, and mark it as such in the prompt.
- **Limit what a tricked model can do**: fewer tools while untrusted content is in context, least privilege
per tool, caps in signatures, confirmation tokens the model cannot create for consequential actions,
egress allowlists.
- **Filter outputs** for data exfiltration patterns, and log every tool call.
- **Test continuously**: an attack suite on every prompt, tool or context change, from every entry point.
- **The insight:** the question is not whether the model can be tricked (it can) but what is the worst it
can do when it is. Injection is a permissions problem more than a prompt problem.
**The follow-up:** "A partner API adds a free-text field." → a new entry point: extend the attack suite to it
before it reaches production.
**Red flag:** "tell the model to ignore instructions in documents."
What a strong answer covers
- **Constrain**: use native structured output or tool-use schemas, small and well described.
- **Validate** with a schema parser, then **validate the meaning** with exact checks: dates in range, IDs that
exist, totals that add up.
- **Repair** with a bounded retry that feeds back the validation error.
- **Monitor** the parse-failure and repair rates per slice.
- **The insight:** well-formed is not correct. A perfectly valid JSON object with the wrong account number is
the dangerous failure, and only a semantic check catches it.
**The follow-up:** "Repairs succeed 99% of the time." → and the 1% goes where? Route it to a person, and
count repairs as a cost.
**Red flag:** regular expressions over free text.
What a strong answer covers
- **Short-term**: the current conversation, with a summary once it passes a threshold.
- **Long-term**: extracted facts and preferences with their provenance, a retention period, and user controls
to view and delete them; retrieved by relevance, not replayed wholesale.
- **Never**: raw personal data, payment details, full records. Nothing stored is the best leak control.
- **The insight:** memory can make an assistant worse. A wrong or stale memory is a persistent hallucination
that the user cannot see; show what is remembered and let people correct it.
**The follow-up:** "Extraction runs asynchronously and lags." → use short-term memory for the current session
and design for the gap.
**Red flag:** "store everything in a vector database."
What a strong answer covers
- **The numbers**: cost per task, latency split, golden-set size and slices, lower bounds.
- **The failure**: how it was found, which signal, and the root cause.
- **The change**: the guard, test or check that now makes that failure impossible or visible.
- **The insight:** interviewers use this question to check everything above. A candidate who shipped
something real can answer it with numbers without pausing.
**The follow-up:** "What would you build differently?" → a specific design decision, and why.
**Red flag:** no numbers, or a failure that was "the model's fault".
Show the answer
**Which model answered, what changed in the inputs, and what changed in the retrieved data.** First the
per-call log: did traffic fail over to a different model, or did the provider update the model behind an
alias? Then the inputs: a new customer segment or a new question type shifts the mix. Then the data: new or
changed documents in the index, or a broken ingestion job. Only then the prompt, and nobody changed it.