Retrieval-Augmented Generation (RAG)

Retrieval-Augmented Generation (RAG) is a system architecture in which a large language model does not rely only on patterns stored in its parameters. At runtime, the application retrieves relevant evidence from external information sources and places that evidence into the model context. The real value of RAG is not the creation of a chat window. It is the ability to connect an existing website, document archive, database, transcript collection, event store, or relationship graph to generative AI without turning the entire information system into a newly trained model.

This course is the natural continuation of Artificial Intelligence: Philosophy, Theory and Practice, Natural Language Processing, and Large Language Models.

Artificial Intelligence
        |
        v
Natural Language Processing
        |
        v
Large Language Models
        |
        v
Retrieval-Augmented Generation

The AI course covers the broader problem space; NLP covers language representation and processing; the LLM course covers the generative model. This course focuses on the engineering layer that connects models to external information systems.

The central principle is:

useful answer
   =
appropriate model
 + correct evidence
 + correct authorization
 + correct context
 + validated output

A powerful model cannot repair a retrieval system that consistently returns the wrong evidence. Correct retrieval is still unsafe if unauthorized data enters the context window.

Unit 1: Parametric and External Knowledge

A large language model learns distributed patterns in its parameters. This can be viewed as parametric knowledge. It is not a database in which every fact maps cleanly to a document row and source identifier.

RAG adds a separate runtime knowledge path.

                 +--------------------+
                 | model parameters   |
                 | parametric memory  |
                 +---------+----------+
                           |
Query -------------------->| LLM
                           ^
                           |
                 +---------+----------+
                 | retrieved evidence |
                 | external knowledge |
                 +--------------------+

A document can change today without the model weights changing. In a RAG system, the document can be re-indexed and the updated evidence can become available at query time.

The original RAG formulation combined parametric and non-parametric memory. Modern production systems use the term more broadly: lexical search, dense retrieval, reranking, SQL, graph queries, temporal filters, and tool calls can all participate in the retrieval layer.

Unit 2: What RAG Is Not

RAG solves a different problem from fine-tuning, long-context inference, or tool execution.

Fine-tuning

Fine-tuning modifies model parameters to change behavior or representation. Re-training a model whenever a document changes is usually the wrong architecture for frequently changing knowledge.

knowledge changed
   |
   +--> fine-tuning:
   |      train
   |      new weights
   |      evaluate
   |      redeploy
   |
   +--> RAG:
          reprocess document
          update index
          retrieve on next query

Long context

A very large context window does not make retrieval unnecessary. Sending the entire archive with every query increases token cost, latency, authorization complexity, and the risk that relevant evidence is lost in a large context.

Long context and retrieval are complementary. Retrieval narrows the evidence set; long context can preserve more surrounding material for selected sources.

Tool use

RAG retrieves information. Tools can perform actions.

RAG:
"What do these records say?"

Tool:
"Run this query."
"Open this file."
"Produce this report."

A production system can combine both.

Unit 3: Turning a Traditional Website into an AI-Aware Website

A website may have years of useful material even if it was never designed around AI. Articles, lecture notes, projects, wiki entries, and technical documents already form a valuable corpus.

The safest first step is not to redesign the site. Build a parallel information pipeline.

Existing Website
   |
   +--> HTML / Markdown
   +--> database content
   +--> wiki
   +--> documents
   |
   v
Content Extraction
   |
   v
Cleaning + Metadata
   |
   v
Chunking
   |
   +--> lexical index
   |
   +--> vector index
   |
   v
RAG API
   |
   v
semantic search / conversational UI

The AI layer becomes a read-oriented knowledge interface above the existing system.

From search to synthesis

Traditional search:

user -> "real-time systems"
     -> matching pages

RAG:

user
  |
  v
"What trade-offs between latency and throughput
are discussed across this site?"
  |
  v
relevant pages + sections + passages
  |
  v
source-grounded synthesis

RAG does not replace search. It uses retrieval as the evidence acquisition stage for generation.

A sensible first production version

Do not begin with an agent or a complex graph.

Query
  |
  v
BM25 + Dense Search
  |
  v
Reciprocal Rank Fusion
  |
  v
Cross-Encoder Reranker
  |
  v
Top evidence
  |
  v
LLM
  |
  v
Cited answer

A measurable baseline is more valuable than an impressive but opaque pipeline.

Unit 4: The Ingestion Pipeline

RAG quality begins before query time.

Source
  |
  v
Extraction
  |
  v
Normalization
  |
  v
Metadata
  |
  v
Chunking
  |
  v
Embedding / lexical indexing
  |
  v
Versioning
  |
  v
Published index

Sources can include HTML, Markdown, PDF, DOCX, database records, transcripts, technical notes, code, event descriptions, and graph node or edge descriptions.

Different sources should not be forced through one generic splitter.

Canonical documents

If the same content exists as HTML, PDF, and a database row, indexing all copies as independent truth creates duplicate retrieval.

Maintain a canonical identity:

canonical_document_id
source_type
source_uri
language
version
created_at
modified_at
access_scope
content_hash

An index entry should always map back to the authoritative source.

Unit 5: Chunking

Chunking is one of the most consequential RAG design choices.

Fixed-length splitting is easy, but can break semantic units.

"... the database transaction boundary ..."
                         | CUT
"... begins and commit behavior ..."

Structure-aware chunking

For technical notes:

Course
 |
 +--> Unit
       |
       +--> Heading
             |
             +--> Subheading
                   |
                   +--> Paragraph

The search unit can be small while the context unit is larger.

retrieval granularity
        !=
context granularity

Parent-child retrieval is especially useful for long structured documents.

Content-aware splitting

Code, transcripts, event streams, and prose require different boundaries.

A method boundary may be better for code; a speaker turn or timestamp for transcripts; an event window for time-series narratives; a heading hierarchy for a course note.

Overlap

Overlap can preserve local context, but excessive overlap expands the index, increases duplicate hits, pollutes reranking, and wastes context tokens.

It must be measured, not guessed.

Unit 6: Embeddings and Vector Search

An embedding model maps text into a dense vector space in which semantically related texts are intended to be close.

"database connection pool"
          |
          v
      embedding
          |
          v
[0.12, -0.31, ..., 0.07]

The query is encoded in the same space and nearest neighbors are retrieved.

Similarity is not truth

Vector similarity does not mean:

• factual correctness,
• causality,
• identity,
• relationship type,
• reliability.

This distinction becomes critical when RAG is connected to human or relationship data.

Approximate nearest neighbors

Exact comparison against millions of vectors can be expensive. ANN structures such as HNSW and FAISS indexes trade a controlled amount of recall for lower latency and better scalability.

The engineering trade-off is:

latency
  vs
recall
  vs
memory

Unit 7: Sparse, Dense, and Hybrid Retrieval

Dense retrieval should not be the only retrieval method in many enterprise systems.

BM25 and related lexical methods remain strong for exact names, identifiers, rare technical terms, codes, and abbreviations.

Dense retrieval is useful for paraphrase and semantic equivalence.

Hybrid retrieval combines both:

                  +--> BM25 ---------+
Query ------------|                  |
                  +--> Dense --------+--> Fusion --> Candidates
                                     |
Metadata filters --------------------+

Score distributions from sparse and dense retrievers are not directly comparable. Rank-based fusion such as Reciprocal Rank Fusion is a robust baseline.

Turkish

Turkish morphology increases the value of both semantic and lexical layers. Dense retrieval can reduce vocabulary mismatch, while lexical retrieval remains important for proper names, codes, and exact technical terminology.

The 2026 RAGTurk study is particularly relevant because it evaluates multiple RAG stages for Turkish and reports that stacking more generative transformations does not automatically improve quality. Strong reranking with a simpler pipeline can provide a better cost-quality trade-off. The result should be treated as evidence from a specific benchmark, not a universal law.

Unit 8: Reranking

First-stage retrieval should emphasize recall. A smaller candidate set can then be reranked by a more expensive model.

10 million chunks
      |
      v
BM25 + ANN
      |
    top 100
      |
      v
Cross-Encoder
      |
    top 10
      |
      v
LLM

A bi-encoder allows document representations to be precomputed. A cross-encoder jointly processes query and candidate and can model richer interactions, but is too expensive for the entire corpus.

bi-encoder = candidate generation
cross-encoder = precision refinement

Unit 9: Query Understanding and Routing

Not every natural-language question should go to vector search.

"How many events occurred in the last seven days?" is an aggregation query.

"Explain changes in communication patterns and provide textual evidence" may require SQL, temporal analysis, graph retrieval, and text retrieval.

                 User Query
                    |
                    v
               Query Router
                    |
      +-------------+-------------+
      |             |             |
  SQL / Filter   Text RAG      Graph Query
      |             |             |
      +-------------+-------------+
                    |
                 Evidence
                    |
                    v
                   LLM

The router can be model-assisted, but high-impact systems should constrain available tools and validate generated parameters.

Unit 10: Feeding RAG from Big Data

The naive design is to convert every structured row into a sentence and embed everything. This is often wasteful.

A structured event such as:

event_id
actor_a
actor_b
timestamp
duration
channel
location_observation

is already optimized for filters, aggregation, joins, and temporal queries.

The proper architecture is heterogeneous:

                    Big Data
                       |
       +---------------+---------------+
       |               |               |
 structured data    text / ASR       relations
       |               |               |
 SQL / analytics    hybrid IR          graph
       |               |               |
       +---------------+---------------+
                       |
                 Evidence Builder
                       |
                       v
                      LLM

What should be embedded?

Good candidates include document passages, transcript segments, notes, narrative descriptions, titles, and entity descriptions.

Poor candidates include values that are already deterministically searchable by SQL, such as counters, normalized identifiers, or exact time-range predicates.

Derived text

Structured events can be transformed into deterministic textual summaries for semantic retrieval, but that summary must remain linked to the source event identifier.

Unit 11: Incremental Indexing and Change Capture

Rebuilding the full index every night does not scale to large corpora.

change
  |
  v
CDC / timestamp / queue
  |
  v
reprocess changed source
  |
  v
invalidate previous chunks
  |
  v
new chunks + embeddings
  |
  v
publish new index version

Idempotency requires stable document keys, source versions, and hashes.

Deletion must propagate to vector indexes, lexical indexes, caches, summaries, and graph structures.

Unit 12: Hot, Warm, and Cold Knowledge

Not all data has the same query frequency.

HOT
recent and frequently queried
fast ANN / memory-heavy

WARM
medium-term history
disk-oriented index

COLD
long archive
slower, cheaper storage

A query about the last week may stay in hot storage. A five-year trend query may activate colder tiers.

Unit 13: Metadata as a First-Class Retrieval Layer

Metadata can reduce the candidate set before semantic search.

language = tr
date >= 2026-09-01
content_type = transcript
access_scope IN user_scopes
source_type = communication

Then run dense or hybrid retrieval inside that permitted subset.

metadata = exact constraint
embedding = approximate semantic relevance

They solve different problems.

Unit 14: Conversational Systems over Private Data

Conversation memory and institutional knowledge should be separate.

Conversation Memory
  = what has been said in this session?

Knowledge Memory
  = what does the corpus actually contain?

A production flow:

User
 |
 v
Session Context
 |
 v
Query Rewrite
 |
 v
ACL
 |
 v
Retriever Router
 |
 +--> SQL
 +--> BM25
 +--> Dense
 +--> Graph
 |
 v
Reranker
 |
 v
Context Builder
 |
 v
Local LLM
 |
 v
Cited Answer

When evidence is insufficient, the system should say so rather than generate an unsupported answer.

Unit 15: Provenance and Evidence

Each chunk should map back to the source.

chunk_id
document_id
source_id
section
offset / page / event_id
version
hash
access_scope

Citations are useful only when they actually support the generated claim.

Question
 |
 v
Retrieved Candidates
 |
 v
Reranked Evidence
 |
 +--> source 1
 +--> source 2
 +--> source 3
 |
 v
LLM Answer
 |
 v
Claim-Evidence Validation

In audit-oriented domains, a summary must never replace the source evidence.

Unit 16: Graph-Based RAG

Flat text retrieval is weak when evidence is distributed across relationships.

A -> B
B -> D
D -> C

A question about the connection between A and C may require multi-hop traversal rather than one semantically similar chunk.

Knowledge graphs and event graphs

Knowledge graph:

PERSON ---- WORKS_AT ----> ORGANIZATION
ORGANIZATION ---- LOCATED_IN ----> CITY

Event graph:

ENTITY_A
   |
   v
[EVENT_123]
   ^
   |
ENTITY_B

For high-impact communication analysis, event-centered graphs can be safer because semantic labels such as "friend" or "associate" are often interpretations rather than directly observed facts.

Observation, computation, interpretation

Observed:
14 communication events between A and B.

Computed:
frequency increased during the last 30 days.

Interpretation:
the nature of the relationship is not established by this fact alone.

RAG should preserve these epistemic layers.

Unit 17: Local, Global, and DRIFT-Style Graph Retrieval

Graph retrieval has different query modes.

Local search starts from a specific entity and expands to neighbors, relationships, relevant text units, and possibly community summaries.

Global search addresses whole-corpus questions such as major themes. Baseline top-k vector retrieval is weak for such questions because no single passage necessarily represents the corpus-level pattern.

A DRIFT-style approach combines global community context with iterative local follow-up retrieval.

community context
       |
       v
initial answer / questions
       |
       v
local retrieval
       |
       v
new subqueries
       |
       v
refined evidence

These modes are more expensive than baseline RAG and should be selected according to query type.

Unit 18: Generalized RAG for Communication and Relationship Analysis

Consider an anonymized closed-network system containing high-volume communication events, transcripts, notes, devices, location observations, and relationship graphs.

The correct architecture is not one giant vector database.

                         Query
                           |
                           v
                         Router
          +----------------+----------------+
          |                |                |
   Deterministic         Text             Graph
      Query              RAG              Search
          |                |                |
   SQL / time         BM25 + Dense       Subgraph
   / filters              |                |
          +----------------+----------------+
                           |
                         Fusion
                           |
                       Reranker
                           |
                    Evidence Builder
                           |
                       Local LLM
                           |
                     Cited Result

Question classes:

• exact counts and time ranges -> SQL,
• semantic topics in transcripts -> hybrid RAG,
• multi-hop relationships -> graph query,
• relationship changes plus textual context -> temporal analysis + graph + RAG.

CDR-like event data

Traffic records are evidence, not interpretations. A cell observation is not automatically an exact physical position, and communication frequency is not automatically a social relationship label.

The system should preserve:

source event
 -> evidence

aggregation
 -> computed feature

LLM explanation
 -> interpretive presentation

Unit 19: Time-Aware RAG

Time is often a primary dimension of event data.

"latest contact", "first contact", "peak period", and "increase relative to last month" require different temporal logic.

A universal recency boost is therefore wrong.

ranking =
semantic relevance
+ lexical relevance
+ task-specific temporal score

The temporal score should follow query intent.

Unit 20: Human-Relationship Inference Boundaries

Edges should distinguish provenance classes.

OBSERVED
direct source event exists

DERIVED
computed from observed events

INFERRED
generated as a hypothesis

A system can support:

A --[42 observed events]--> B

without claiming:

A --[close friend]--> B

unless independent evidence supports that interpretation.

Unit 21: Hierarchical Retrieval

Very large archives can use multiple retrieval levels.

Archive
  |
  +--> Year
        |
        +--> Month
              |
              +--> Community / topic
                    |
                    +--> Event group
                          |
                          +--> raw document / transcript

Higher-level summaries are navigation structures, not primary evidence.

summary = routing aid
raw source = evidence

Unit 22: Query Decomposition

A complex analytical question may require several tools.

Example:

"Explain relationship pairs whose communication intensity increased during the last three months and where a new technical topic appeared in the associated transcripts."

Possible plan:

1. resolve time interval
2. compute intensity change
3. select candidate pairs
4. retrieve related transcripts
5. search for newly emerging topic evidence
6. merge source evidence
7. generate explanation

This is controlled agentic RAG: different subqueries can invoke different retrievers.

Unit 23: Agentic RAG

An agent can dynamically choose retrieval methods, reformulate the query, or retrieve again when evidence is insufficient.

Question
 |
 v
Plan
 |
 v
Retrieve
 |
 v
Enough evidence?
 |             |
 no            yes
 |             |
 v             v
re-query      answer

Production systems need bounded execution:

• maximum steps,
• maximum tokens,
• timeout,
• allow-listed tools,
• cost budget.

An agent must never expand its authorization scope.

Unit 24: Offline and On-Premises RAG

Sensitive environments can run RAG entirely offline.

Local Sources
    |
    v
Local Embedding
    |
    v
Local Index
    |
    v
Local Reranker
    |
    v
Local LLM

Local embedding is valuable for privacy, deterministic versioning, and low external dependency.

Version together:

• model license,
• model hash,
• embedding version,
• tokenizer,
• index format,
• CPU/GPU requirements,
• update bundle,
• rollback procedure.

Changing the embedding model usually requires rebuilding its vector index.

Unit 25: Performance and Capacity Engineering

RAG latency is a pipeline, not a single model call.

T_total =
query_parse
+ authentication
+ filtering
+ sparse_search
+ vector_search
+ reranking
+ context_build
+ LLM_prefill
+ LLM_decode

Measure p95 and p99, not only averages.

Increasing top-k can increase retrieval recall while also increasing reranker cost, prompt tokens, prefill time, and attention work.

Ingestion throughput

Large-scale indexing can use a bounded pipeline:

reader
 -> parse queue
 -> chunk workers
 -> embedding batches
 -> index writer

Backpressure is required. Unbounded queues convert throughput mismatch into memory growth.

Unit 26: Caching

Possible caches include:

query normalization
query embeddings
retrieval results
reranker results
final response

Deterministic stages are easiest to cache safely.

A final-answer cache key must include authorization and relevant versions. Otherwise responses can leak across users or become stale after index changes.

Unit 27: Security

RAG moves external text into the model context and therefore creates a new attack surface.

Prompt injection

A retrieved document may contain text such as "ignore previous instructions." That text is data, not an instruction from the system owner.

Retrieved content should be treated as untrusted input and separated from system instructions.

Vector and embedding weaknesses

Risks include unauthorized retrieval, cross-tenant leakage, embedding inversion, poisoned documents, stale-knowledge conflicts, and sensitive metadata exposure.

Embeddings are not anonymization.

Authorization before retrieval

Wrong:

all data
 -> retrieval
 -> LLM context
 -> authorization afterward

Correct:

user identity
   |
   v
authorized corpus
   |
   v
retrieval
   |
   v
LLM

Unauthorized evidence should never enter the context window.

Data poisoning

Ingestion should retain source trust, hashes, approvals, document class, and modification provenance.

Unit 28: RAG Evaluation

At least three layers should be measured.

Retrieval:

• Recall@K,
• Precision@K,
• MRR,
• nDCG,
• hit rate,
• source diversity.

Generation:

• answer relevance,
• factual correctness,
• faithfulness,
• citation support,
• unsupported-claim rate.

System:

• p50/p95/p99,
• throughput,
• index freshness,
• zero-result rate,
• authorization failures,
• cache hit ratio.

RAG quality
   |
   +--> retrieval quality
   +--> generation quality
   +--> system quality

A single score can hide root causes.

Unit 29: Reducing Hallucination, Not Eliminating It

RAG can reduce unsupported generation but cannot guarantee truth.

Failure modes include:

wrong retrieval
correct retrieval + wrong interpretation
insufficient retrieval
conflicting sources
model prior overriding evidence
prompt injection
stale index

A trustworthy system should support explicit abstention and contradiction reporting.

Unit 30: Versioning and Reproducibility

A RAG release is more than an LLM checkpoint.

RAG_RELEASE =
    parser_version
  + chunker_version
  + embedding_version
  + lexical_index_version
  + vector_index_version
  + graph_version
  + reranker_version
  + prompt_version
  + llm_version
  + acl_policy_version

Without component-level versioning, regressions are difficult to diagnose.

Unit 31: Production Rollout

A safe rollout can progress in stages.

Stage 1:

Query -> Hybrid Search -> source list

No generation.

Stage 2:

Query -> Retrieval -> Reranker -> LLM -> cited summary

Stage 3 adds conversation state.

Stage 4 adds read-only structured tools.

Stage 5 adds an agent only if the earlier pipeline is insufficient.

This prevents retrieval defects from being hidden behind agent complexity.

Unit 32: Recommended General Architecture

For an anonymized high-volume, closed-network, multi-source system:

                            User
                             |
                     Identity / Session
                             |
                        Authorization
                             |
                         Query Router
                             |
        +--------------------+--------------------+
        |                    |                    |
    SQL / Analytics       Hybrid Text           Graph
        |                BM25 + Dense            Query
        |                    |                    |
        |                 Fusion               Subgraph
        |                    |                    |
        +--------------------+--------------------+
                             |
                          Reranker
                             |
                      Evidence Builder
                             |
                     Provenance / Scope
                             |
                         Local LLM
                             |
                     Output Validator
                             |
                        Cited Answer

The architecture follows three rules:

1. deterministic questions go to deterministic engines,
2. semantic questions go to retrieval,
3. relationship questions go to graph structures.

The LLM provides synthesis and natural-language interaction at the final layer.

Unit 33: Application Pattern — Existing Content System

Suppose a legacy content system has:

CONTENT
id
language
slug
title
summary
body
published_at
modified_at

The source schema does not need to be changed.

A derived retrieval document can contain:

document_id = content.id
language
title
section_path
chunk_text
modified_at
content_hash
source_url

When a content record changes, only that record needs to be reprocessed.

Unit 34: Application Pattern — Structured Events + Text + Graph

Consider a system containing an event store, transcripts, and an interaction graph.

EVENT
  |
  +--> time
  +--> endpoint A / B
  +--> duration
  +--> channel
  +--> source id

TRANSCRIPT
  |
  +--> event_id
  +--> segment
  +--> text

GRAPH
  |
  +--> node
  +--> edge / event reference

Question:

"Which relationships increased in activity during the last 60 days, and what topics appear in the associated transcripts?"

Processing:

SQL
 -> compute intensity change
 -> candidate pairs

Graph
 -> retrieve candidate subgraphs

Hybrid RAG
 -> retrieve transcript evidence

Reranker
 -> prioritize evidence

LLM
 -> summarize only retrieved evidence

This is more auditable than embedding every raw event row.

Unit 35: Turkish RAG Engineering

Turkish needs dedicated evaluation.

Normalization

I/İ/ı/i, Unicode normalization, proper-name suffixes, and ASCII Turkish can change lexical retrieval.

Auxiliary fields can be indexed:

original
lower_tr
ascii_aux
morph_aux

The canonical text remains unchanged.

Morphology

Different inflected forms can share a root while appearing lexically distant. A Turkish-aware analyzer can help sparse retrieval, while dense retrieval can reduce some vocabulary mismatch.

Multilingual corpora

A Turkish query may need to retrieve English technical material. Multilingual embedding can bridge languages, but language metadata should still control scope when necessary.

Unit 36: When Not to Use Graph RAG

Graph RAG should not be selected simply because graphs are fashionable.

Delay it when:

• most questions are answered by single passages,
• entity/relation extraction is unreliable,
• graph indexing cost is too high,
• the corpus changes too rapidly for graph maintenance,
• users mostly need exact search.

Start with:

hybrid retrieval
 -> reranking
 -> citations

Add graph retrieval only when measured query classes require multi-hop or corpus-level reasoning.

Unit 37: Combining RAG and Fine-Tuning

Fine-tuning and RAG are complementary.

Fine-tuning can specialize output style, task behavior, or classification ability.

RAG is better suited to frequently changing, access-controlled, source-cited knowledge.

fine-tuned behavior
      +
retrieved evidence
      =
domain behavior + current knowledge

Embedding confidential documents into a controlled retrieval layer is operationally different from teaching the same material into model weights.

Unit 38: Knowledge and Action Boundaries

Retrieved evidence can support an answer or recommendation. Executing an action is a separate authorization problem.

Question
 |
 v
RAG
 |
 v
Proposal
 |
 v
Policy / Human / Deterministic Rule
 |
 v
Action

In high-impact systems, natural-language output should never become an action merely because it is fluent.

Unit 39: Retriever Design by Data Type

One retriever is not optimal for every source type. The data model should influence retrieval.

Technical documentation

Technical documents depend heavily on headings, version metadata, product names, API symbols, and error codes.

query
 |
 +--> exact / lexical search
 +--> dense semantic search
 +--> section metadata
 |
 v
reranker

Rare symbols may be poorly represented by a general embedding model while remaining easy to retrieve lexically.

Transcripts

A transcript chunk should preserve metadata such as record identifier, speaker role, start and end time, language, and segment order.

A narrow semantic hit can be expanded with neighboring segments:

hit
 |
 +--> previous
 +--> current
 +--> next

This restores conversational context without embedding the entire recording as one document.

Structured events

Structured events should first use SQL or metadata filters. Dense retrieval can operate over associated descriptions or transcripts.

Source code

Code retrieval can combine symbol search, lexical search, code embeddings, file paths, and call graphs.

Graph data

Graph retrieval uses neighborhood, path, community, and temporal constraints. Text embeddings support graph retrieval; they do not replace graph traversal.

Unit 40: Context Building Is Not Concatenation

Joining top-k passages verbatim is not sufficient context engineering.

The context builder should address:

• duplicate passages,
• conflicting sources,
• source timestamps,
• authorization,
• neighboring chunks,
• evidence confidence,
• token budget.

Duplicate collapse

Overlap and document versions can create near-duplicate candidates.

candidate set
  |
  v
near-duplicate collapse
  |
  v
source diversity
  |
  v
token-budget packing

Contradictions

If two sources disagree, the system should expose the disagreement rather than silently choosing one.

Citation mapping

Claims should retain evidence identifiers during generation.

Evidence E17
Evidence E42
    |
    v
Claim C3
    |
    +--> E17
    +--> E42

This makes post-generation validation possible.

Unit 41: A General Event Graph for Communication Data

For relationship analysis, events should remain first-class data.

Entity
  |
  +--> participates_in --> CommunicationEvent
                              |
                              +--> timestamp
                              +--> duration
                              +--> direction
                              +--> channel
                              +--> source_record_id
                              +--> transcript_id

A pairwise relationship edge can be derived for query and visualization.

Its attributes can include event count, total duration, first/last observation, directional counts, channel distribution, and source event identifiers.

The edge is not raw evidence; provenance must point back to events.

Time windows

A relationship graph should be read as:

G(t1,t2)

rather than one timeless structure.

Direction should also be preserved even when the UI renders an undirected summary.

Unit 42: Communities and Global Sensemaking

Community detection can reduce a very large graph into structurally dense groups.

Full Graph
   |
   +--> Community A
   +--> Community B
   +--> Community C

A graph community is not automatically a real-world organization or social group.

graph community
   !=
real-world organization

Community summaries can include dominant nodes, strong edges, time distributions, related text themes, and provenance coverage.

They should guide retrieval, not replace source evidence.

Global questions can use a map/reduce pattern over community reports.

Unit 43: A RAG Decision Tree

Not every problem needs RAG.

Exact filter or aggregation?
  +--> SQL / analytics

Need textual evidence?
  +--> hybrid retrieval

Need relationship / multi-hop reasoning?
  +--> graph retrieval

Need different model behavior?
  +--> prompt / SFT / fine-tuning

A disciplined RAG architecture begins by refusing to embed data that already has a better deterministic query model.

Unit 44: Building an Evaluation Dataset

A production benchmark should reflect real workload classes:

exact lookup
semantic lookup
multi-document synthesis
temporal
graph local
graph multi-hop
global corpus
no-answer
authorization
adversarial

Each question can define expected sources, acceptable answer properties, forbidden claims, and required filters.

No-answer cases

The system should explicitly abstain when the corpus does not support an answer.

Authorization cases

The same query should be tested under different access scopes. Evidence that is invisible to one scope must not leak through generated text.

Regression

Every severe production retrieval or grounding failure should become a permanent regression case.

Unit 45: Phased Implementation for a High-Volume Closed Network

For a closed-network system containing large event data, transcripts, and relationship analytics, I would not begin with Graph RAG.

Phase 1: Text evidence retrieval

Use a Turkish-aware lexical index plus a multilingual or Turkish-capable embedding model, fuse results, and rerank with a cross-encoder.

Benchmark retrieval before adding an LLM.

Phase 2: Cited question answering

Add a local model with strict grounding rules:

use only provided evidence
abstain when evidence is insufficient
preserve source identifiers
do not invent relationship labels

Phase 3: SQL and graph router

Route exact counts and filters to SQL, semantic questions to text RAG, relationships to graph queries, and mixed questions to controlled orchestration.

Phase 4: Temporal subgraphs

Preserve:

node -> edge -> event ids -> transcript ids

so every graph answer can return to source evidence.

Phase 5: Community and Graph RAG

Add community/global methods only when measured workloads actually require multi-hop or whole-corpus reasoning.

Unit 46: A Framework-Neutral Service Contract

A RAG service can expose conceptual operations:

search(query, scope, filters)
 -> Evidence[]

answer(query, evidence)
 -> Answer

graph(query, scope, timeRange)
 -> GraphEvidence[]

aggregate(metric, filters)
 -> StructuredEvidence

Evidence can carry source identity, text, score, timestamp, access scope, and provenance.

The application contract should not depend on one orchestration library.

application
   |
RAG contracts
   |
+-- lexical engine
+-- vector engine
+-- graph engine
+-- LLM runtime

Unit 47: Cost and Scaling

RAG cost includes more than generated tokens.

Local deployments still consume embedding compute, index memory, disk, reranker capacity, GPU memory, queue time, and re-indexing windows.

Raw vector storage roughly depends on vector count, dimensions, and bytes per dimension, with additional overhead from ANN structures and metadata.

At scale, consider partitioning, quantization, cold storage, smaller embedding dimensions, and selective embedding.

A cascaded reranker can reduce expensive inference:

top 100 -> lightweight reranker -> top 30
top 30  -> stronger reranker -> top 8

Every stage must justify its additional latency through measured quality improvement.

Relationship to Artificial Intelligence, NLP, and LLMs

RAG is not all of artificial intelligence.

Artificial Intelligence: Philosophy, Theory and Practice provides the broader framework of representation, learning, reasoning, search, and decision making.

Natural Language Processing provides the foundations for text representation, tokenization, retrieval, query understanding, and linguistic normalization.

Large Language Models explains the generative model that consumes retrieved evidence.

NLP
  -> representation of language and queries

LLM
  -> generation and contextual synthesis

RAG
  -> connecting the right external evidence at runtime

Graph-based RAG also connects to Discrete Mathematics: Sets, Logic, Relations and Graphs.

The relationship with Database Management Systems is equally important. RAG does not replace databases. Structured integrity, transactions, deterministic filtering, and exact aggregation remain database responsibilities. RAG is an upper-layer interface that can combine structured and unstructured evidence.

For communication and relationship data, the source-versus-interpretation boundary in HTS Analysis should remain explicit. Graph edges, derived scores, and LLM explanations do not have the same evidentiary status.

Conclusion

Retrieval-Augmented Generation is not simply the act of passing more text to an LLM. It is the engineering problem of selecting the correct source, retrieval method, authorization boundary, context budget, and evidence chain.

A mature RAG system combines:

Search
  +
Databases
  +
Graph Analytics
  +
NLP
  +
LLMs
  +
Authorization
  +
Evaluation

The best architecture is not the one with the largest model, the most agents, or the most elaborate graph. It is the one that provides measurable retrieval quality, predictable latency, provenance, access control, and traceability back to source evidence.

References

• Patrick Lewis et al. “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks.” NeurIPS, 2020. https://arxiv.org/abs/2005.11401

• Vladimir Karpukhin et al. “Dense Passage Retrieval for Open-Domain Question Answering.” EMNLP, 2020. https://arxiv.org/abs/2004.04906

• Nils Reimers; Iryna Gurevych. “Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks.” EMNLP-IJCNLP, 2019. https://arxiv.org/abs/1908.10084

• Jeff Johnson; Matthijs Douze; Hervé Jégou. “Billion-scale Similarity Search with GPUs.” 2017. https://arxiv.org/abs/1702.08734

• Yu. A. Malkov; D. A. Yashunin. “Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs.” IEEE TPAMI, 2020; preprint 2016. https://arxiv.org/abs/1603.09320

• Omar Khattab; Matei Zaharia. “ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT.” SIGIR, 2020. https://arxiv.org/abs/2004.12832

• Darren Edge et al. “From Local to Global: A Graph RAG Approach to Query-Focused Summarization.” 2024. https://arxiv.org/abs/2404.16130

• Zirui Guo et al. “LightRAG: Simple and Fast Retrieval-Augmented Generation.” 2024. https://arxiv.org/abs/2410.05779

• Hao Yu et al. “Evaluation of Retrieval-Augmented Generation: A Survey.” 2024. https://arxiv.org/abs/2405.07437

• Dongyu Ru et al. “RAGChecker: A Fine-grained Framework for Diagnosing Retrieval-Augmented Generation.” 2024. https://arxiv.org/abs/2408.08067

• Süha Kağan Köse et al. “RAGTurk: Best Practices for Retrieval Augmented Generation in Turkish.” SIGTURK 2026, 2026. DOI: https://doi.org/10.18653/v1/2026.sigturk-1.15

• Microsoft Research. GraphRAG Documentation: Basic, Local, Global and DRIFT Search. https://microsoft.github.io/graphrag/

• OWASP GenAI Security Project. LLM01:2025 Prompt Injection. https://genai.owasp.org/llmrisk/llm01-prompt-injection/

• OWASP GenAI Security Project. LLM08:2025 Vector and Embedding Weaknesses. https://genai.owasp.org/llmrisk/llm082025-vector-and-embedding-weaknesses/

• Deepak Dhyani. RAG with Python Cookbook: Learn Principles of RAG with LLM and Agentic AI, with 120+ Recipes. BPB Publications, 2026. ISBN 978-93-65895-735.

• Julie Smith. RAG Generative AI: A Practical Guide to Building Custom Retrieval-Augmented Pipelines and Enhancing AI Systems. 2024.