What is a Large Language Model (LLM) and how does it work?
An LLM is a transformer-based model trained on massive text corpora to predict the next token. Architecture: decoder-only transformer (GPT family) with billions of parameters. Training: causal language modeling — given tokens 1..n, predict token n+1; trained on trillions of tokens from internet text, books, code. Emergent capabilities appear at scale: reasoning, coding, few-shot learning, instruction following. Inference: autoregressive generation — sample one token at a time, append to context, repeat. Temperature controls randomness (0=greedy, 1=sampling, >1=creative). Context window = max tokens the model can attend to simultaneously.
What is prompt engineering and what are the key techniques?
Prompt engineering crafts input text to elicit desired LLM behavior without changing model weights. Key techniques: (1) Zero-shot: direct instruction — "Classify this review as positive/negative:", (2) Few-shot: provide examples before the task — 2–5 labeled examples improve accuracy significantly, (3) Chain-of-Thought (CoT): "Think step by step" — forces LLM to reason explicitly before answering, improves complex reasoning by 10–40%, (4) Self-consistency: sample multiple CoT paths, take majority vote, (5) Role prompting: "You are an expert Python developer...", (6) Structured output: "Respond only in JSON format: {field: value}". For production: always version and test prompts — they degrade with model updates.
Explain Retrieval-Augmented Generation (RAG) architecture.
RAG grounds LLM responses in retrieved documents, reducing hallucination. Components: (1) Indexing — chunk documents, embed with sentence-transformers, store in vector DB (Pinecone/Chroma/Weaviate), (2) Retrieval — embed query, find top-k similar chunks (cosine similarity), optionally rerank, (3) Augmentation — inject retrieved chunks into prompt: "Use only the following context: {chunks} Question: {query}", (4) Generation — LLM generates answer grounded in retrieved text. Why over fine-tuning: (1) Knowledge updates without retraining, (2) Source attribution, (3) Lower cost. Production challenges: chunking strategy, retrieval quality, context window limits, hallucination on borderline retrievals.
What is fine-tuning vs RLHF vs prompting — when to use each?
Prompting: modify input, no training — fastest, cheapest. Best when: task works well zero/few-shot, no labeled data, need flexibility. Full fine-tuning: update all weights on task-specific data. Best when: you have 10k+ labeled examples, need maximum performance, can afford GPU compute. LoRA/QLoRA: update only low-rank adapter weights (0.1% of params) — 100x cheaper than full fine-tuning, 90% of the performance. Best for: adapting large models cheaply. RLHF (Reinforcement Learning from Human Feedback): align model to human preferences using reward model trained on comparisons. Used by: ChatGPT, Claude. Need large amounts of human preference data. For most production use cases: start with prompting, use LoRA if prompting insufficient.
What is hallucination in LLMs and how do you reduce it?
Hallucination: LLMs confidently state false information. Types: (1) Factual — wrong facts ("The Eiffel Tower is in Berlin"), (2) Faithfulness — answer contradicts the provided context, (3) Temporal — outdated knowledge. Reduction strategies: (1) RAG — ground responses in retrieved documents, (2) Structured prompts: "If you don't know, say 'I don't know'" — but models often ignore this, (3) Temperature 0 for factual tasks — removes sampling randomness, (4) Self-consistency — multiple samples + voting, (5) Citation forcing — "Quote the exact passage that supports your answer", (6) Evaluation: use FactScore, RAGAS faithfulness metric to detect hallucinations automatically.
What is an AI agent and how do you build one?
An AI agent uses an LLM as a reasoning engine that can take actions via tools to complete multi-step tasks. Architecture: LLM + tools + memory + planning loop. ReAct pattern: Thought (reason about next action) → Action (call tool) → Observation (tool result) → repeat until done. Tools: web search, code execution, API calls, database queries. Memory: short-term (conversation history), long-term (vector store). Frameworks: LangChain Agents, LlamaIndex agents, LangGraph (for complex multi-step). Production challenges: (1) Agents can get stuck in loops, (2) Unpredictable tool call sequences, (3) Cost from multiple LLM calls per task, (4) Hallucinated tool calls. Use structured outputs + explicit stopping conditions.
What are the key differences between GPT-4, Claude, Gemini, and Llama?
GPT-4 (OpenAI): strongest reasoning, best function calling, industry standard for agentic tasks, expensive, closed-source. Claude (Anthropic): longest context window (200K), best for document analysis, strong at following complex instructions, designed for safety. Gemini (Google): multimodal-first, integrated with Google ecosystem, competitive on reasoning. Llama 3 (Meta): open-source, can self-host, 70B rivals GPT-3.5 class. Mistral/Mixtral: efficient open-source models, MoE architecture. Selection criteria: (1) Task requirements — function calling (GPT-4), long documents (Claude), self-hosting (Llama), (2) Cost — Llama hosted on Groq is 10x cheaper than GPT-4, (3) Latency — Groq inference is fastest, (4) Compliance — self-hosted for data sovereignty.
How do you handle context window limits in production LLM applications?
Strategies: (1) Chunking + retrieval (RAG) — don't send everything, retrieve relevant parts, (2) Sliding window — for long conversations, keep last N turns + summary of earlier context, (3) Summarization — periodically compress old context with the LLM itself, (4) Hierarchical processing — map-reduce: process chunks independently, then synthesize, (5) Context compression — LLMLingua removes unimportant tokens from long contexts (3–10x compression), (6) Long-context models — Claude 200K, Gemini 1M for document-level tasks where retrieval is impractical. Monitor average context utilization — if consistently >80% of limit, implement compression proactively.
What is function calling / tool use in LLMs and how do you implement it?
Function calling lets LLMs output structured JSON to invoke external functions instead of free text. OpenAI API: pass tools=[{type: "function", function: {name, description, parameters: JSON Schema}}]. If model decides to call a function: response.choices[0].message.tool_calls contains name + arguments as JSON string. Execute the function, send result back as role:"tool" message, let model incorporate result into final response. Best practices: (1) Write clear, specific descriptions — the model uses them to decide when to call, (2) Use strict JSON Schema validation on inputs, (3) Handle errors gracefully (tool execution can fail), (4) Parallel tool calling for independent operations. Basis of all LLM agents.
What is the difference between streaming and non-streaming LLM responses?
Non-streaming: wait for the complete response before displaying — adds full generation latency to time-to-first-token. For a 500 token response at 50 tokens/sec, user waits 10 seconds to see anything. Streaming: server-sent events (SSE) or WebSocket stream tokens as generated — user sees output immediately, time-to-first-token ~200ms. Implementation with OpenAI: stream=True, iterate over chunks: for chunk in client.chat.completions.create(..., stream=True): yield chunk.choices[0].delta.content. FastAPI: return StreamingResponse with async generator. Frontend: EventSource API or fetch with ReadableStream. Always use streaming for chat interfaces — perceived latency improvement is massive even if total time is identical.
Know your weak spots before the interview
10-question AI readiness assessment → personalized study plan.
Take Free Assessment →Unlock 3,000+ Interview Questions
Get full access to all interview questions with detailed answers, explanations, and real company context
Enroll GenAI Pack →