Prompt, RAG or fine-tuning: which fix matches your AI's error?

An online store's chatbot gets the return policy wrong and sounds curt. The team has 3 ideas: rewrite the AI's instructions, feed it documents to look up, or retrain the model.

This article compares 3 approaches: prompting (the instructions sent to the AI), RAG (letting the AI look up documents at the moment it answers) and fine-tuning (training a model further on your own examples). The main sources are documentation from OpenAI, Anthropic, Google Cloud and Amazon Bedrock plus arXiv studies, checked on October 24, 2026.

It answers: which error each approach fixes, how to tell which error you have, and when to stop. Click Star already has articles on RAG, on evaluating AI chatbots and on token optimization; this one does not repeat them.

Key points

• There are 2 kinds of error. Missing knowledge (private documents, outdated facts) needs context supplied. Wrong behavior (format, tone) needs behavior taught (per OpenAI).
• Start with a prompt and test questions with correct answers (OpenAI mentions 20 or more) to see where errors are.
• RAG fits when knowledge changes often or is private. Fine-tuning fits when you need stable behavior and have quality examples: OpenAI suggests 50 to start, Google Cloud about 100.
• Fine-tuning is not a way to load knowledge. Gekhman and co-authors (EMNLP 2024) found models learn new facts slowly this way, and once learned, they tend to hallucinate more.
• Combining approaches is not always better. In OpenAI's test, adding RAG with examples from a lookup store to a fine-tuned model lowered the BLEU score (out of 100, closeness to a reference answer) from 87 to 83.
• Fine-tuning depends on the provider: OpenAI is winding down its self-serve platform; Google Cloud and Amazon Bedrock still describe the feature.

How the 3 ways of making AI "know the job" differ

Picture the AI as a new employee. A prompt is the instruction sheet in front of them. RAG is the filing cabinet they check when a customer asks. Fine-tuning is a long training course so they work by habit without instructions. This analogy is Click Star's, close to OpenAI's exam metaphor.

With prompting and RAG the model stays the same and only its input changes: RAG finds relevant document passages and adds them to the prompt before the model answers.

Fine-tuning means training an existing model further on a small dataset for 1 specific job. According to Google Cloud (updated October 22, 2026), this adjusts the model's parameters to your data.

Start from the type of error, not from the tool

Many guides present the 3 approaches as a ladder: prompt, then RAG, then fine-tuning. OpenAI says it is often not a straight line: they are levers for different problems, and you must pull the right one.

Optimize context when the model lacks knowledge, is outdated, or needs proprietary information. Optimize behavior when results are inconsistent, badly formatted or in the wrong tone (OpenAI, the guide "Optimizing LLM Accuracy", no date shown).

So the first step is to measure, not pick a tool. Anthropic asks for specific, measurable success criteria and testing on a held-out set (the page "Define success criteria and build evaluations", no date shown).

OpenAI suggests 20 or more test questions with correct answers. For each wrong answer ask: did the model lack information, or have it and act wrongly?

Prompting: enough for more jobs than you think

For summarization, translation and code generation, OpenAI says a prompt is often enough for production. Its reason for starting with a prompt: writing a prompt forces you to define what "correct" means.

An OpenAI example: correcting Icelandic sentences with GPT-4 scored 62 BLEU with no examples and 70 with 3 example pairs. An older model and Icelandic, shown only to illustrate.

Anthropic recommends 3 to 5 realistic, diverse examples (Anthropic's prompting guide for Claude). It also advises letting the model say "I don't have enough information", and says these techniques reduce hallucinations but do not eliminate them (the page "Reduce hallucinations").

The limit: stuffing a whole document store into the prompt may not solve it. Liu and co-authors (arXiv, July 6, 2023) found accuracy drops significantly when the needed information sits in the middle of a long context.

OpenAI does not say to drop long context, but to measure at several context sizes.

RAG: when knowledge changes often or is private

Choose RAG when the error is "the AI does not know". A weekly price list or internal documents are knowledge the model lacks or has outdated.

The big advantage: you fix knowledge by fixing documents. The original RAG paper (Lewis and co-authors, arXiv, May 22, 2020) names 2 open problems for models that rely only on what they have learned: updating knowledge and showing where an answer came from.

RAG adds retrieval to tune. According to OpenAI, wrong passages or too many irrelevant ones drown out the real information.

Fine-tuning: teaching how to work, not what to know

Choose fine-tuning when the model knows enough but performs unevenly. OpenAI names 2 reasons: higher accuracy on 1 task, or higher efficiency (fewer tokens, or a smaller model).

Google Cloud lists jobs suited to supervised fine-tuning: classification, sentiment analysis, entity extraction and summarizing content that is not complex. A side benefit is shorter prompts, lowering run-time cost and latency.

On data, OpenAI advises 50 or more examples, quality over quantity, matching real inputs. Google Cloud says "about 100 examples or more", while its supervised fine-tuning section speaks of hundreds of labeled examples. Each provider gives its own mark; neither is a standard threshold.

A common misunderstanding is using fine-tuning to make an AI "memorize" company documents. Gekhman and co-authors (arXiv, May 9, 2024, version 3 on October 1, 2024) found examples with new knowledge are learned much more slowly than those matching existing knowledge, and once learned they linearly increase the tendency to hallucinate.

They conclude models mostly acquire facts in pre-training (the initial learning stage), while fine-tuning teaches them to use those facts. Limit: it is a closed-book question-answering setup, so it does not speak for every current model.

OpenAI's test shows both sides. Fine-tuning with 1,000 examples raised gpt-3.5-turbo to 78 BLEU and gpt-4 to 87, against 70 with only 3 example pairs. Adding RAG (1,000 more examples from a lookup store) to the fine-tuned model dropped the score to 83: OpenAI says this is a behavior problem and the extra context was noise.

OpenAI also says the 2 can stack when errors include both missing knowledge and wrong behavior.

Fine-tuning also depends on the provider and the model lifecycle

Check that the provider still offers fine-tuning. OpenAI's "Model optimization" page says it is winding down its fine-tuning platform: closed to new users, with existing users able to create jobs (new training runs) for the coming months.

The "Deprecations" page, in its self-serve fine-tuning section, gives dates: from May 7, 2026, organizations that never fine-tuned cannot create jobs; from July 2, 2026, those with no fine-tuned inference in 60 days cannot either; from January 6, 2027, existing customers cannot create new jobs.

A fine-tuned model runs until its base model is retired. The page lists GPT-4.1 as removed on January 14, 2027, fine-tuned versions on the same schedule; older ones such as ft-gpt-4 were shut down on October 23, 2026.

Fine-tuning is not finished. Google Cloud's "Introduction to tuning" page (updated October 22, 2026) lists Gemini models supporting supervised fine-tuning, such as Gemini 3.5 Flash. The Amazon Bedrock page describes supervised fine-tuning, reinforcement fine-tuning and distillation.

On Claude: Anthropic's glossary says the Claude API does not currently offer fine-tuning. Amazon Bedrock's fine-tuning page lists only Claude 3 Haiku (us-west-2).

Comparing by criteria

The table summarizes the documents above, arranged by Click Star (a reference framework, not a measurement).

According to OpenAI, RAG requires tuning retrieval, while fine-tuning must rerun training and manage the dataset. When content changes, RAG is lighter because only documents are edited (Click Star's view).

OpenAI also warns that both can cause regressions (formerly good results turning bad) as an application grows more complex.

Put a price on errors before investing

"Accurate enough" depends on the damage of 1 error. OpenAI's illustrative support example: a case the AI solves saves 20 USD, a case wrongly sent to a human costs 40 USD, and a case that makes the customer leave costs 1,000 USD.

Assuming 5% of failed cases end in a lost customer, OpenAI computes break-even accuracy of 81.5% (an illustration, not an industry standard). For very costly cases such as fraud, it keeps a human in charge and the AI only assists.

Click Star suggests (a reference framework, not from outside sources) a similar table with your own numbers: the value of 1 correct case and the damage of 1 minor and 1 serious error. If you are already above break-even, you may not need to invest more urgently, but weigh the extra gain against the cost.

What this means for Vietnamese businesses

Click Star found no public measurement comparing these 3 approaches on Vietnamese data. Every figure here comes from foreign sources; test with real Vietnamese customer questions, with and without diacritics.

Both RAG and fine-tuning touch company data, which may include customers' personal information. Ask your legal team first; Click Star already has an article on bringing AI into a business with safe data.

Click Star's view. For most business chatbots, a sensible order is: build a test set, tune the prompt, then add RAG if the error is missing knowledge. Fine-tuning comes later, only for behavior, once there are many quality examples and the provider still supports it.

The reason is the cost of backing out: editing a prompt or document is much faster than redoing a fine-tuned version. Click Star sells AI integration services, so this view may lean toward ready-made solutions; check it against your own test set.

What to do: a 14-day plan

The plan below is a Click Star reference framework, not taken from outside sources.

• Days 1 to 2: collect 20 or more real customer questions with correct answers; set measurable success criteria.
• Days 3 to 4: write a basic prompt, run every question, record the wrong answers.
• Days 5 to 6: label each wrong answer: missing knowledge or wrong behavior.
• Days 7 to 9: add 3 to 5 example pairs and a "say you don't know" rule.
• Days 10 to 12: if knowledge is still missing, add RAG with 1 small group of documents; measure again.
• Days 13 to 14: build the damage table, compare it with current accuracy; decide to stop, widen RAG, or gather examples for fine-tuning.

Common mistakes

1. Choosing the tool before measuring: "fine-tuning" sounds professional, so you start at once with no test set.
2. Using fine-tuning to load price lists or policies, when research shows models struggle to learn new facts this way.
3. Stuffing a whole document store into the prompt and trusting the model to read it all.
4. Combining RAG and fine-tuning without measuring again, although OpenAI's test shows the score can drop.
5. Training on examples unlike real input, such as clean data when customers type without diacritics.
6. Planning fine-tuning without checking the provider and the base model's retirement schedule.
7. Putting customer data into training sets or document stores without asking legal.

What is not known

Click Star has no public measurement comparing the 3 approaches on Vietnamese data. OpenAI's and Anthropic's guides show no date on the page; the Icelandic test used models that are now retired.

Neither Claude page above shows a date, so how long this status lasts is unclear. This article also uses no official fine-tuning price list.