Why did I use RAG rather than fine tuning for SudoChat?

Author: Mustafa Siddiqui · Part: 3 of 3 · Recruiter question: Why did Mustafa use RAG rather than fine tuning for SudoChat?

Evidence rule: This is my first-person authored source material. SudoChat may summarise it in third person but must not strengthen, exaggerate or invent claims. Limitations and Do not claim sections are hard factual boundaries.

## Relevance to responsible AI
The RAG decision also reflects several of my responsible AI principles.

**Transparency:**
The answer can be linked back to evidence.

**Accountability:**
Knowledge owners remain responsible for authoritative source material.

**Safety:**
The system can be restricted to approved evidence.

**Privacy:**
Information can remain in governed knowledge sources rather than being unnecessarily incorporated into model training.

**Reliability:**
Knowledge can be updated independently of the model.

For me, that separation is valuable.

## Limitations or gaps
RAG is not inherently safe or accurate.

Poor retrieval can result in poor answers.

An outdated knowledge base can still generate outdated responses.

A model can misinterpret retrieved evidence.

Incorrect metadata can cause the wrong source to be prioritised.

Prompt injection can potentially enter through retrieved content.

RAG systems therefore still require:

* evaluation
* source governance
* access control
* security testing
* monitoring
* citation checking
* safe refusal behaviour

I also do not claim that fine tuning would never be appropriate for SudoChat.

If the project later developed requirements that were better addressed through model adaptation, I would evaluate that option.

The current decision was based on the requirements of the present proof of concept.

## Useful links
SudoChat project and repository

My portfolio:
https://mustafa-siddiqui.com/

GitHub:
https://github.com/sudoqui

Responsible AI and privacy research

Federal Court AI resources

Research on Retrieval Augmented Generation

## Do not claim
Do not claim RAG is always superior to fine tuning.

Do not claim RAG eliminates hallucination.

Do not claim fine tuning is inherently unsafe.

Do not claim information stored in a RAG system is automatically private.

Do not claim retrieval automatically respects permissions unless the architecture explicitly enforces them.

Do not claim SudoChat uses model fine tuning.

Do not claim SudoChat's knowledge is permanently embedded inside the underlying LLM.

Do not claim a RAG system automatically knows which document is authoritative.

Do not claim the hypothetical Federal Court architecture described here represents the Court's current implementation.

The accurate representation is that I chose RAG because **SudoChat's primary challenge is controlled, current and traceable knowledge rather than changing the fundamental behaviour of the language model**. RAG allows the evidence to remain separate from the model, making it easier to update, audit, cite, govern and remove while supporting the responsible AI boundaries I wanted the system to demonstrate.

← Previous part · Knowledge Base