3.5. Reflection Checkpoint: Applications of Foundation Models

Key Takeaways

• Application design decisions follow a decision tree: standard task → use a managed AI service; needs domain knowledge or behavior change → RAG for a knowledge gap, fine-tuning for a behavior gap.
• RAG grounds an LLM's answers in retrieved, verified documents (retrieve → augment → generate) — it separates the "knowledge source" (your private data) from the "language skill" (the LLM), which is the core defense against hallucination for enterprise applications.
• Vector databases (OpenSearch k-NN, Aurora/RDS with pgvector, DocumentDB, Neptune, Bedrock Knowledge Bases) power the "retrieve" step of RAG by finding semantically similar content, not just keyword matches.
• Agents for Amazon Bedrock go a step beyond RAG: RAG answers a question from retrieved context, while an agent plans and executes a multi-step goal by calling APIs and tools — the exam tests recognizing when a requirement needs multi-step orchestration versus a single retrieval-and-generate call.
• Prompt engineering technique scales with how much guidance the model needs: zero-shot (no examples) → single-shot (one example) → few-shot (multiple examples) — the fix for an inconsistent zero-shot output is almost always to add few-shot examples before reaching for fine-tuning.
• Pre-training builds a general Foundation Model (expensive, done by AI labs); fine-tuning specializes an existing model on a small, high-quality, task-specific dataset — this is the customization method nearly every organization actually uses.
• Evaluating generative AI output requires combining approaches: benchmark datasets (MMLU, HumanEval, GLUE) compare models on standardized tests but don't guarantee performance on your specific data; human evaluation is the gold standard for real quality but slow and expensive; ROUGE (summarization) and BLEU (translation) are cheap automated proxies that measure word overlap only — none of them detect hallucinations on their own.

Connecting Forward

You now know how to design, ground, customize, and evaluate a Foundation Model application. Phase 4 shifts focus from "does it work well" to "does it work responsibly" — the principles and practices of Responsible AI.

Self-Check Questions

• Your team needs a chatbot that can look up a customer's order status via an internal API and then draft a personalized refund email — not just answer a question from documents. Is this a RAG use case or an agent use case, and what's the deciding signal?
• A summarization model scores a strong ROUGE-1 of 0.75, but human reviewers find factual errors in 1 in 4 summaries. What does this reveal about what ROUGE actually measures, and what additional evaluation step was missing?