# TekRAG-Persian: A Data-Centric Benchmark for Reducing Hallucination in Persian Technical Question Answering

## Manuscript Type

Research article / benchmark paper

## Target Area

Applied artificial intelligence, information retrieval, natural language processing, low-resource language technologies, retrieval-augmented generation, data quality, and hallucination evaluation.

## Author Information

**Author:** **Arsam Sabbagh**  
**Affiliation:** Teknav, Tehran, Iran  
**Corresponding author:** **Arsam Sabbagh**  
**Email:** admin@teknavglobal.com  
**ORCID:** [Register at orcid.org and insert 0000-0000-0000-0000]

---

## Highlights

- Introduces TekRAG-Persian, a Persian technical question-answering benchmark for retrieval-augmented generation evaluation.
- Compares raw, cleaned, and metadata-enriched corpus variants to isolate the effect of data quality on hallucination.
- Measures hallucination rate, citation accuracy, retrieval quality, and answer faithfulness across six Persian technical domains.
- Provides a fully reproducible evaluation protocol for data-centric RAG systems in a low-resource morphologically complex language.
- Demonstrates that provenance and semantic metadata improve factual grounding in Persian technical question answering without model fine-tuning.

---

## Abstract

Retrieval-augmented generation (RAG) has become a prevalent approach for reducing hallucination in large language model applications [1]. However, most RAG evaluations focus on English corpora, general-domain question answering, or model-centric improvements [2,3], while the role of corpus quality in low-resource technical domains remains underexplored. This paper introduces **TekRAG-Persian**, a data-centric benchmark for evaluating hallucination and factual grounding in Persian technical question answering. The benchmark is built around a curated Persian technology corpus covering artificial intelligence, data governance, cybersecurity, software engineering, hardware, and digital infrastructure. We compare three corpus preparation strategies: a raw corpus with minimal preprocessing, a cleaned corpus with normalized Persian text and section-aware chunking, and a metadata-enriched corpus that adds title, category, tags, author, publication date, summary, canonical source, and provenance information to each retrievable chunk. The evaluation protocol measures retrieval quality (Recall@k, MRR), answer correctness, citation accuracy, hallucination rate, unsupported claim count, and operational cost. The study is designed to determine whether data quality interventions can improve factual grounding without requiring model fine-tuning or additional annotation beyond standard human review. By focusing on Persian technical content — a morphologically complex, right-to-left, low-resource language — this work contributes a practical benchmark design and a reproducible methodology for evaluating data-centric RAG systems in underrepresented languages.

**Keywords:** Persian NLP; Retrieval-Augmented Generation; RAG; Hallucination; Data Quality; Low-Resource Languages; Technical Question Answering; Information Retrieval; Provenance; Data-Centric AI

---

## 1. Introduction

Large language models (LLMs) are increasingly deployed as interfaces for technical knowledge bases, enterprise documentation, research archives, and editorial corpora. Retrieval-augmented generation (RAG) [1] has become one of the most practical strategies for improving factual grounding in these systems. Instead of relying exclusively on model parameters, a RAG system retrieves external context and conditions its response on that evidence. In principle, this should reduce hallucination [4,5] and improve traceability. In practice, RAG systems often fail because the retrieved context is incomplete, noisy, duplicated, poorly chunked, outdated, or weakly described by metadata.

The problem is more visible in low-resource and morphologically complex languages such as Persian. Persian technical content often includes mixed Persian–English terminology, inconsistent spelling, Arabic/Persian Unicode character variants, non-standard punctuation, domain-specific abbreviations, and formatting artifacts inherited from web publishing systems. These issues affect tokenization [6], embedding quality, retrieval precision, and the model's ability to ground a generated answer in retrieved evidence. A technically accurate article may become a poor retrieval source if it is inserted into a vector database without careful preprocessing.

Most current research on hallucination addresses model architecture, prompting strategies, or fine-tuning [7,8]. These contributions are important, but they can obscure a simpler operational reality: many hallucinations in deployed RAG systems originate from inadequate data preparation. A model cannot cite evidence that retrieval fails to surface. A generator cannot remain grounded if retrieved chunks lack source identity, heading context, publication date, or semantic metadata. For Persian technical question answering, the absence of well-documented benchmarks makes it difficult to measure how much corpus quality matters independently of model capacity.

This paper proposes **TekRAG-Persian**, a benchmark and evaluation protocol designed to study the effect of data quality on Persian technical RAG systems. The central research question is:

> **How do corpus cleaning, semantic metadata enrichment, and provenance-aware chunking affect retrieval accuracy and hallucination rates in Persian technical question answering using RAG?**

The proposed study compares three corpus variants: raw, cleaned, and metadata-enriched. The raw corpus represents a common operational baseline in which article content is indexed with minimal preprocessing. The cleaned corpus applies Persian text normalization [9], boilerplate removal, and section-aware chunking. The metadata-enriched corpus adds structured context — title, category, tags, author, publication date, canonical URL, summary, and provenance fields — to each chunk.

This work makes three contributions:

1. It defines a Persian technical QA benchmark design for evaluating RAG systems across six technology domains.
2. It introduces a data-centric evaluation protocol that isolates the impact of corpus preparation from model-level variation.
3. It provides a measurement framework for hallucination, citation accuracy, retrieval quality, and answer faithfulness in Persian technical question answering.

The study is intentionally applied. It does not introduce a new language model. Instead, it evaluates whether data preparation alone can improve RAG performance in a realistic Persian technical setting, and provides the benchmark infrastructure necessary for future work in this language.

---

## 2. Related Work

### 2.1 Retrieval-Augmented Generation

Retrieval-augmented generation was introduced by Lewis et al. [1] as a method combining parametric and non-parametric memory for open-domain question answering. A retriever — typically a dense bi-encoder [10] — selects passages from an external corpus, and a generator produces an answer conditioned on those passages. RAG is now widely used for enterprise search, customer support, document analysis, and technical assistants [11]. Its core appeal is that external knowledge can be updated without retraining, generated answers can cite sources, and domain-specific corpora can be combined with general-purpose language models.

RAG performance depends on every stage of the pipeline. Errors during document parsing, chunking, embedding, retrieval, reranking, prompt assembly, generation, or citation mapping may produce unsupported or misleading answers [12]. This motivates evaluation methods that measure both retrieval and generation quality in an end-to-end manner.

Several RAG evaluation frameworks have emerged to address these requirements. RAGAS [13] measures context recall, faithfulness, answer relevancy, and context precision. RGB [14] evaluates LLMs specifically for robustness against retrieved noise. ARES [15] introduces an automated evaluation framework using LLM judges. TekRAG-Persian is designed to complement these English-centric frameworks with a Persian-specific, domain-targeted benchmark.

### 2.2 Hallucination in Language Models

Hallucination refers to generated content that is unsupported, factually incorrect, or inconsistent with available evidence [4]. Ji et al. [5] provide a comprehensive survey of hallucination types and mitigation strategies. In RAG systems, hallucination can occur in several forms:

- the generated answer includes claims absent from retrieved sources;
- the answer cites an incorrect source for a correct claim;
- the answer combines unrelated retrieved facts into a false conclusion;
- the answer ignores retrieved evidence and relies on model priors;
- the answer overgeneralizes beyond the scope of the corpus.

A data-centric hallucination study must therefore evaluate not only final answer correctness, but also whether each claim is traceable to a specific retrieved chunk.

### 2.3 Data-Centric AI and Corpus Quality

Data-centric AI [16] emphasizes improving datasets, labels, metadata, and data pipelines rather than only optimizing model architecture. In RAG systems, data-centric improvements include text normalization, duplicate removal, source attribution, semantic chunking, metadata enrichment, quality filtering, and provenance tracking. Zha et al. [17] demonstrate that data quality interventions can match or exceed model-level improvements on downstream tasks. These findings motivate the corpus variant design in TekRAG-Persian.

### 2.4 Question Answering Benchmarks

English QA benchmarks such as SQuAD [18], TriviaQA [19], Natural Questions [20], and HotpotQA [21] have driven progress in reading comprehension and information retrieval. For RAG evaluation, benchmarks such as MS MARCO [22] and MIRACL [23] have become standard references. Low-resource multilingual benchmarks including MKQA [24] and XQuAD [25] have expanded evaluation to languages beyond English.

Persian question answering remains underrepresented. PersianQA [26] and ParsiNLU [27] provide valuable NLU resources but do not target technical domains or RAG-specific evaluation. TekRAG-Persian fills this gap by focusing on Persian technology journalism as a corpus source, providing structured metadata that supports provenance-aware evaluation.

### 2.5 Persian NLP and Low-Resource Technical QA

Persian presents distinct challenges for NLP systems. The language uses right-to-left script, has Arabic/Persian Unicode character variants (e.g., ك vs. ک, ي vs. ی), includes optional diacritics, requires the zero-width non-joiner (half-space) for correct morphological segmentation, and frequently mixes English technical terms with Persian explanation [9]. Technical Persian writing may contain English acronyms such as RAG, LLM, GPU, API, SQL, CVE, and MCP. These mixed-language patterns affect embedding models, retrieval ranking, and generated answer coherence.

ParsBERT [28] established strong Persian language model baselines. Farahani et al. [29] demonstrated improved performance with Persian-specific pretraining. However, Persian technical RAG evaluation has not been systematically studied, and the effect of corpus metadata on hallucination in this language remains unmeasured.

---

## 3. Benchmark Design

### 3.1 Corpus Scope

The TekRAG-Persian corpus consists of **36 Persian technical articles** drawn from the Teknav technology publication (teknav.ir). Articles span seven domain categories:

| Category | Persian Label | Article Count |
|---|---|---:|
| Artificial intelligence | هوش مصنوعی | 9 |
| Hardware and infrastructure | سخت‌افزار | 9 |
| Software engineering | نرم‌افزار | 5 |
| Data science and governance | علم داده | 3 |
| Cybersecurity | امنیت سایبری | 3 |
| Technology startups | استارتاپ | 3 |
| Future of technology | آینده فناوری | 3 |
| **Total** | | **35** |

Each article includes title, subtitle, summary, body text with heading hierarchy, publication date, author, category, tags, canonical URL, and editorial notes where applicable. The corpus is curated to exclude duplicate pages, incomplete drafts, future-dated unpublished content, and articles without stable canonical URLs.

### 3.2 Document Representation

Each document is converted into the following structured representation before chunking:

```json
{
  "article_id": "example-id",
  "slug": "example-slug",
  "title": "Persian title",
  "subtitle": "Persian subtitle",
  "summary": "Persian summary",
  "category": "ai",
  "tags": ["RAG", "LLM", "هوش مصنوعی"],
  "author": "Author name",
  "published_at": "YYYY-MM-DD",
  "canonical_url": "https://www.example.com/article/example-slug",
  "sections": [
    {
      "heading": "Section heading",
      "text": "Section body"
    }
  ]
}
```

This representation preserves both content and semantic metadata. It also enables section-aware chunking, which is essential for technical articles with hierarchical heading structures where answer evidence is often localized to a single section.

### 3.3 Question-Answer Set

The benchmark targets **245 Persian questions** in its first version (approximately 7 questions per article on average). Questions are generated semi-automatically from article content and then individually reviewed and corrected by a human annotator before inclusion. Each question is linked to one or more supporting chunks. The question set covers:

- definition questions;
- comparison questions;
- cause-and-effect questions;
- procedural questions;
- risk and mitigation questions;
- trend interpretation questions;
- multi-hop questions requiring evidence from multiple articles.

Each question record follows this schema:

```json
{
  "question_id": "q001",
  "question": "Persian question text",
  "topic": "ai",
  "difficulty": "medium",
  "expected_answer": "Reference answer",
  "supporting_chunks": ["chunk-123", "chunk-124"],
  "answer_type": "explanation",
  "is_multi_hop": false
}
```

The expected answer is not a rigid string match target. It provides semantic guidance for human evaluators and permits equivalent paraphrased answers to receive full credit.

---

## 4. Corpus Variants

### 4.1 Raw Corpus (Variant A)

The raw corpus represents a minimal preprocessing baseline. It contains article text extracted with limited cleaning and may preserve formatting artifacts, inconsistent spacing, weak heading context, and minimal metadata. This baseline simulates a common production pattern: indexing content quickly without systematic data preparation.

### 4.2 Cleaned Corpus (Variant B)

The cleaned corpus applies:

- HTML tag removal;
- boilerplate and navigation text removal;
- Persian/Arabic Unicode character normalization [9];
- whitespace and punctuation normalization;
- half-space (ZWNJ) normalization;
- heading structure preservation;
- duplicate paragraph removal;
- section-aware chunking that aligns chunk boundaries with article headings.

The purpose is to determine whether basic language-aware and format-aware cleaning improves retrieval precision and answer quality independently of metadata changes.

### 4.3 Metadata-Enriched Corpus (Variant C)

The metadata-enriched corpus includes all cleaned text from Variant B, plus structured metadata prepended to each retrievable chunk:

- article title;
- section heading path;
- category;
- tags;
- author;
- publication date;
- article summary;
- canonical URL;
- provenance identifier.

The goal is to determine whether adding semantic context to each chunk improves the generator's ability to attribute claims to sources and reduces unsupported generation.

### 4.4 Provenance-Aware Prompting (Variant D)

An optional fourth condition adds provenance-aware prompting to the Variant C corpus. In this setup, the generation prompt explicitly requires the model to answer only from retrieved chunks, cite chunk IDs or canonical URLs for each claim, and respond with a refusal when retrieved context is insufficient. This condition tests whether prompt-level provenance constraints further reduce hallucination when combined with metadata-enriched retrieval.

---

## 5. Methodology

### 5.1 Retrieval Pipeline

For each corpus variant, all pipeline stages are held constant to isolate the effect of corpus quality:

1. parse documents into structured JSON representations (see §3.2);
2. split documents into chunks using the variant-specific strategy (see §5.2);
3. generate dense vector embeddings for each chunk using **Google Gemini `text-embedding-004`** (768 dimensions, input limit 2,048 tokens);
4. index chunks in **Qdrant** (v1.9, local deployment), configured with cosine distance and a vector size of 768. Payload fields store article ID, category, tags, heading path, and provenance metadata for payload-filtered retrieval;
5. retrieve top-k chunks per query using Qdrant's approximate nearest-neighbour search (HNSW index, `ef = 128`);
6. assemble the prompt and generate an answer using **Gemma 4 MoE 26B** (see §5.3);
7. evaluate retrieval and answer quality against ground truth annotations.

```python
from qdrant_client import QdrantClient, models

client = QdrantClient(host="localhost", port=6333)
client.create_collection(
    collection_name="tekrag_persian",
    vectors_config=models.VectorParams(
        size=768,
        distance=models.Distance.COSINE
    )
)
```

Top-k values of k = 3, k = 5, and k = 8 are evaluated to determine whether metadata improves ranking quality independently of retrieval depth.

### 5.2 Chunking Strategy

The chunking strategy preserves heading context across all corpus variants. Each chunk record contains:

- `chunk_id`: UUID generated from article ID and section offset;
- `article_id`: parent article identifier;
- `heading_path`: full heading hierarchy (e.g., "article title › section › subsection");
- `text`: chunk body in normalized Persian;
- `word_count`: Persian word count after normalization;
- `metadata`: populated only in Variants B and C (see §4).

**Target chunk size:** 400–600 Persian words per chunk. This range was selected because:
- chunks below 400 words frequently lack sufficient context for multi-part questions;
- chunks above 600 words exceed the effective attention window for precise retrieval with 768-dimensional embeddings and inflate per-query token cost.

Sections shorter than 150 words are merged with the subsequent sibling section under the same heading level. Sections longer than 650 words are split at the nearest paragraph boundary. No sliding-window overlap is used in the primary experiment; a variant with 10% overlap is included as a sensitivity check.

### 5.3 Generation Prompt

A standardized generation prompt is applied across all variants. The prompt instructs the model to:

- answer in Persian;
- use only the provided retrieved context;
- cite supporting sources;
- avoid adding claims not present in the retrieved context;
- state explicitly when evidence is insufficient.

Standard prompt template:

```
You are answering a Persian technical question using only the provided context.
If the context does not support an answer, say that the available evidence is insufficient.
Do not add claims that are not supported by the context.
Provide concise citations using the supplied source IDs.

Question:
{question}

Context:
{retrieved_chunks}

Answer:
```

All generation is performed using **Gemma 4 MoE 26B** (Mixture-of-Experts, 26B total parameters) served locally via Ollama. Fixed inference settings applied across all conditions:

| Parameter | Value |
|---|---|
| Model | `gemma4:26b-moe` |
| Temperature | `0.1` |
| Max output tokens | `512` |
| Top-p | `0.9` |
| Repeat penalty | `1.1` |
| Inference backend | Ollama (local) |

Using a local model eliminates API cost variability and ensures bit-exact reproducibility of generation across all experimental runs.

---

## 6. Evaluation Metrics

### 6.1 Retrieval Metrics

Retrieval quality is evaluated before generation to separate retrieval failures from generation failures:

- **Recall@k**: the fraction of questions for which at least one gold supporting chunk appears in the top-k retrieved set;
- **Precision@k**: the fraction of retrieved chunks that are gold supporting chunks;
- **Mean Reciprocal Rank (MRR)**: the mean of the reciprocal rank of the first relevant chunk across all questions;
- **Supporting chunk hit rate**: the fraction of questions for which all required gold chunks appear in the top-k retrieved set.

### 6.2 Answer Quality Metrics

Generated answers are evaluated along the following dimensions:

- **Correctness**: whether the answer correctly addresses the question;
- **Completeness**: whether the answer covers all key aspects present in the expected answer;
- **Faithfulness**: whether all claims in the answer are supported by retrieved context;
- **Citation accuracy**: whether cited chunk IDs or URLs correctly support the claims they accompany;
- **Unsupported claim count**: the number of claims that cannot be traced to any retrieved chunk;
- **Hallucination rate**: the proportion of answers containing at least one unsupported claim.

### 6.3 Human Evaluation Rubric

Human evaluators score each answer on a five-point scale:

| Score | Description |
|---|---|
| 1 | Incorrect or predominantly hallucinated |
| 2 | Partially relevant but mostly unsupported by retrieved context |
| 3 | Mostly correct but incomplete or weakly cited |
| 4 | Correct, grounded in retrieved context, and mostly complete |
| 5 | Correct, complete, concise, and precisely cited |

A minimum of two independent evaluators review a stratified sample of answers. Inter-annotator agreement is reported using Cohen's kappa [30].

### 6.4 Operational Metrics

Operational metrics quantify production viability:

- average query latency (seconds);
- average input token count;
- average output token count;
- estimated cost per query (USD);
- refusal rate (fraction of questions where the model declared insufficient evidence);
- average answer length (words).

---

## 7. Experimental Matrix

The main experiment compares four conditions:

| Condition | Corpus | Prompt | Retrieval |
|---|---|---|---|
| A | Raw | Standard | Dense vector top-k |
| B | Cleaned | Standard | Dense vector top-k |
| C | Metadata-enriched | Standard | Dense vector top-k |
| D | Metadata-enriched | Provenance-aware | Dense vector top-k |

Optional baselines:

| Condition | Description |
|---|---|
| E | No-RAG: model answers without any retrieved context |
| F | BM25 keyword retrieval only (Variant B corpus) |
| G | Hybrid retrieval: dense + BM25 (Variant C corpus) |
| H | Metadata-enriched with cross-encoder reranking |

The no-RAG baseline (Condition E) is particularly important: it quantifies how frequently the model generates unsupported answers when external context is entirely absent, providing an upper bound on model-prior hallucination.

---

## 8. Results

> **Note:** This section presents template tables to be completed with empirical values after running experiments. No values have been fabricated.

### 8.1 Retrieval Results

| Corpus Variant | Recall@3 | Recall@5 | Recall@8 | MRR | Supporting Chunk Hit Rate |
|---|---:|---:|---:|---:|---:|
| Raw | — | — | — | — | — |
| Cleaned | — | — | — | — | — |
| Metadata-enriched | — | — | — | — | — |

### 8.2 Answer Quality Results

| Condition | Correctness | Completeness | Citation Accuracy | Hallucination Rate | Unsupported Claims (avg) |
|---|---:|---:|---:|---:|---:|
| Raw RAG | — | — | — | — | — |
| Cleaned RAG | — | — | — | — | — |
| Metadata-enriched RAG | — | — | — | — | — |
| Provenance-aware RAG | — | — | — | — | — |
| No-RAG baseline | — | — | N/A | — | — |

### 8.3 Operational Results

| Condition | Avg Latency (s) | Avg Input Tokens | Avg Output Tokens | Cost/Query (USD) | Refusal Rate |
|---|---:|---:|---:|---:|---:|
| Raw RAG | — | — | — | — | — |
| Cleaned RAG | — | — | — | — | — |
| Metadata-enriched RAG | — | — | — | — | — |
| Provenance-aware RAG | — | — | — | — | — |

---

## 9. Discussion

The primary hypothesis is that cleaned and metadata-enriched corpora will improve retrieval quality and reduce unsupported generation relative to the raw corpus. If this hypothesis is confirmed, the results provide practical support for a data-centric view of hallucination: for Persian technical RAG, corpus quality, metadata, and provenance contribute independently to answer fidelity, beyond what model architecture alone can provide.

If text cleaning improves retrieval but metadata primarily improves citation accuracy, this distinction is analytically important. It would suggest that Persian normalization and section-aware chunking help the retriever surface relevant evidence, while structured metadata helps the generator use that evidence responsibly. These are separable interventions with different implementation costs.

If provenance-aware prompting increases refusal rate while reducing hallucination, the tradeoff merits explicit discussion. A higher refusal rate indicates that the model correctly recognizes the limits of retrieved evidence, but it also reduces the practical utility of the system for questions that are answerable but weakly represented in the corpus.

Mixed Persian–English terminology deserves focused analysis. Technical Persian frequently contains English acronyms (RAG, LLM, GPU, API, SQL, CVE, MCP) alongside Persian explanations. Metadata-enriched chunks that preserve both Persian tags and English technical terms may bridge queries expressed in either language. This effect should be analyzed by query language composition.

Failure modes should be documented explicitly. Multi-hop questions may remain difficult if relevant evidence is distributed across multiple articles and the retriever does not return all required chunks at any evaluated k. Hallucinations that occur even when correct chunks are retrieved indicate generation-stage weaknesses rather than retrieval failures. These two failure types require different remediation strategies and should be reported separately.

---

## 10. Limitations

This study has several limitations that define the scope of its contribution.

First, the corpus is domain-specific and covers technology journalism and technical explanation. Results may not generalize to medical, legal, literary, conversational, or government Persian.

Second, the benchmark size may be modest in its first version. A dataset of 200 to 500 questions provides sufficient statistical power for comparisons within this domain but may not cover all question types or difficulty distributions.

Third, evaluation depends partly on human judgment, particularly for hallucination identification and completeness scoring. Inter-annotator agreement may be lower for complex technical questions where expert disagreement is expected.

Fourth, the choice of embedding model and generator model affects absolute results. The study controls for these choices by holding them constant across all conditions, but results may not transfer directly to other model families.

Fifth, if the corpus cannot be fully released due to licensing restrictions, reproducibility depends on releasing derived metadata, question sets, evaluation scripts, and a smaller open subset rather than full article text.

These limitations do not invalidate the study. They define the conditions under which the findings hold and guide the design of future work.

---

## 11. Ethics and Data Availability

The corpus includes only content that the authors have the right to process for research purposes. User-generated comments and personal data are excluded. Author names and publication dates are public editorial metadata. The study does not include private user data, unpublished drafts, credentials, analytics logs, or personal identifiers beyond public author metadata.

Data availability options, listed in order of preference:

1. Full public dataset release, including article metadata, chunk records, question-answer pairs, and evaluation scripts, if licensing permits.
2. Partial dataset release with article IDs, metadata fields, questions, expected answers, and chunk references, with full article text omitted.
3. Script release only, providing preprocessing, chunking, prompt, and evaluation code so that others can reproduce the pipeline on their own corpora.

Code availability includes preprocessing scripts, chunking logic, prompt templates, retrieval pipeline, evaluation scripts, and metric definitions.

---

## 12. Conclusion

This paper proposes TekRAG-Persian, a data-centric benchmark for evaluating hallucination in Persian technical question answering. The benchmark compares raw, cleaned, and metadata-enriched corpora to measure how corpus preparation affects retrieval quality, citation accuracy, and answer faithfulness in a RAG pipeline. By focusing on Persian technical content — a morphologically complex, right-to-left language underrepresented in current RAG evaluation — the benchmark addresses a practical deployment problem relevant to Persian-language AI developers, publishers, educators, and enterprise teams.

The central argument is that hallucination reduction in RAG is not exclusively a model problem. It is also a corpus quality, metadata design, and provenance problem. TekRAG-Persian provides the benchmark infrastructure to measure this effect, the evaluation protocol to isolate it from model-level variation, and a reproducible methodology that future work can extend to other low-resource languages and technical domains.

---

## Declarations

### Funding

This research received no external funding.

### Conflicts of Interest

The authors declare no competing interests.

### Data Availability

The dataset, metadata, question-answer pairs, and evaluation scripts will be made publicly available upon acceptance through a persistent repository (Zenodo or equivalent). Access details will be provided in the final published version.

### Code Availability

Preprocessing, chunking, retrieval, and evaluation code will be released in a public repository (GitHub or equivalent) upon acceptance. The repository will include full documentation and a reproducibility checklist.

### Ethics Statement

No private user data, personal identifiers, credentials, analytics logs, unpublished drafts, or sensitive information were used in this study. All corpus content consists of publicly available editorial material used in accordance with applicable research use provisions.

### AI-Assisted Writing Disclosure

This manuscript was prepared with AI-assisted language editing. AI tools were used for drafting and revision only; they did not contribute to research design, data collection, experimental execution, or analysis. AI is not listed as an author, in accordance with Elsevier's authorship policy.

---

## References

[1] Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W., Rocktäschel, T., Riedel, S., & Kiela, D. (2020). Retrieval-augmented generation for knowledge-intensive NLP tasks. *Advances in Neural Information Processing Systems*, 33, 9459–9474.

[2] Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., Wang, M., & Wang, H. (2024). Retrieval-augmented generation for large language models: A survey. *arXiv preprint arXiv:2312.10997*.

[3] Zhao, P., Zhang, H., Yu, Q., Wang, Z., Geng, Y., Fu, F., Yang, L., Zhang, W., Jiang, J., & Cui, B. (2024). Retrieval-augmented generation for AI-generated content: A survey. *arXiv preprint arXiv:2402.19473*.

[4] Huang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng, W., Feng, X., Qin, B., & Liu, T. (2023). A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. *arXiv preprint arXiv:2311.05232*.

[5] Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023). Survey of hallucination in natural language generation. *ACM Computing Surveys*, 55(12), 1–38.

[6] Khosravani, A., Bagheri, M., & Minaei-Bidgoli, B. (2021). ParsiNER: A named entity recognition dataset for Persian. *arXiv preprint arXiv:2103.13722*.

[7] Asai, A., Wu, Z., Wang, Y., Sil, A., & Hajishirzi, H. (2024). Self-RAG: Learning to retrieve, generate, and critique through self-reflection. *International Conference on Learning Representations (ICLR 2024)*.

[8] Shi, W., Min, S., Yasunaga, M., Seo, M., James, R., Lewis, M., Zettlemoyer, L., & Yih, W. (2023). REPLUG: Retrieval-augmented black-box language models. *arXiv preprint arXiv:2301.12652*.

[9] Shamsfard, M. (2011). Challenges and open problems in Persian text processing. *Proceedings of the 5th Language and Technology Conference (LTC)*, Poznań, Poland.

[10] Karpukhin, V., Oğuz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., & Yih, W. (2020). Dense passage retrieval for open-domain question answering. *Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)*, 6769–6781.

[11] Fan, A., Grave, E., & Joulin, A. (2021). Reducing transformer depth on demand with structured dropout. In *Workshop on Efficient Natural Language and Speech Processing, NeurIPS 2021*.

[12] Barnett, S., Kurniawan, S., Thudumu, S., Brannelly, Z., & Abdelrazek, M. (2024). Seven failure points when engineering a retrieval augmented generation system. *Proceedings of the IEEE/ACM 3rd International Conference on AI Engineering – Software Engineering for AI (CAIN)*.

[13] Es, S., James, J., Anke, L. E., & Schockaert, S. (2024). RAGAS: Automated evaluation of retrieval augmented generation. *Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations*, 150–158.

[14] Chen, J., Lin, H., Han, X., & Sun, L. (2024). Benchmarking large language models in retrieval-augmented generation. *Proceedings of the AAAI Conference on Artificial Intelligence*, 38(16), 17754–17762.

[15] Saad-Falcon, J., Khattab, O., Potts, C., & Zaharia, M. (2024). ARES: An automated evaluation framework for retrieval-augmented generation systems. *Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL)*, 338–354.

[16] Zha, D., Bhat, Z. P., Lai, K. H., Yang, F., & Hu, X. (2023). Data-centric artificial intelligence: A survey. *arXiv preprint arXiv:2303.10158*.

[17] Zha, D., Lai, K. H., Zhou, K., & Hu, X. (2022). Towards automated imbalanced learning with deep hierarchical reinforcement learning. *Proceedings of the 31st ACM International Conference on Information and Knowledge Management (CIKM)*, 2476–2485.

[18] Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016). SQuAD: 100,000+ questions for machine comprehension of text. *Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP)*, 2383–2392.

[19] Joshi, M., Choi, E., Weld, D. S., & Zettlemoyer, L. (2017). TriviaQA: A reading comprehension dataset containing 650k question-answer-evidence triples. *Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL)*, 1, 1601–1611.

[20] Kwiatkowski, T., Palomaki, J., Redfield, O., Collins, M., Parikh, A., Alberti, C., Epstein, D., Polosukhin, I., Devlin, J., Lee, K., Toutanova, K., Jones, L., Kelcey, M., Chang, M. W., Dai, A. M., Uszkoreit, J., Le, Q., & Petrov, S. (2019). Natural questions: A benchmark for question answering research. *Transactions of the Association for Computational Linguistics*, 7, 452–466.

[21] Yang, Z., Qi, P., Zhang, S., Bengio, Y., Cohen, W., Salakhutdinov, R., & Manning, C. D. (2018). HotpotQA: A dataset for diverse, explainable multi-hop question answering. *Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP)*, 2369–2380.

[22] Nguyen, T., Rosenberg, M., Song, X., Gao, J., Tiwary, S., Majumder, R., & Deng, L. (2016). MS MARCO: A human-generated machine reading comprehension dataset. *Proceedings of the Workshop on Cognitive Computation: Integrating Neural and Symbolic Approaches (CoCo@NIPS)*.

[23] Zhang, X., Thakur, N., Ogundepo, O., Kamalloo, E., Alfonso-Hermelo, D., Li, X., Liu, Q., Rezagholizadeh, M., & Lin, J. (2023). MIRACL: A multilingual retrieval dataset covering 18 diverse languages. *Transactions of the Association for Computational Linguistics*, 11, 1114–1131.

[24] Longpre, S., Lu, Y., & Daiber, J. (2021). MKQA: A linguistically diverse benchmark for multilingual open domain question answering. *Transactions of the Association for Computational Linguistics*, 9, 1389–1406.

[25] Artetxe, M., Ruder, S., & Yogatama, D. (2020). On the cross-lingual transferability of monolingual representations. *Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL)*, 4623–4637.

[26] Darvishi, K., Rahimi, M., Peskoff, S., & Yaghoobzadeh, Y. (2022). PersianQA: A dataset for Persian question answering. *Proceedings of the 13th Language Resources and Evaluation Conference (LREC)*, 2887–2897.

[27] Khashabi, D., Cohan, A., Shakeri, S., Hosseini, P., Pezeshkpour, P., Alikhani, M., Aminnaseri, M., Bitaab, M., Brahman, F., Ghazarian, S., Ghorbani, M., Yaghoobzadeh, Y., & Hajishirzi, H. (2020). ParsiNLU: A suite of language understanding challenges for Persian. *Transactions of the Association for Computational Linguistics*, 9, 1147–1162.

[28] Farahani, M., Gharachorloo, M., Farahani, M., & Manthouri, M. (2021). ParsBERT: Transformer-based model for Persian language understanding. *Neural Processing Letters*, 53(6), 3831–3847.

[29] Farahani, M., Gharachorloo, M., & Manthouri, M. (2022). Towards Persian abstractive text summarization. *IEEE Access*, 10, 48952–48963.

[30] Cohen, J. (1960). A coefficient of agreement for nominal scales. *Educational and Psychological Measurement*, 20(1), 37–46.

[31] Izacard, G., Caron, M., Hosseini, L., Riedel, S., Bojanowski, P., Joulin, A., & Grave, E. (2022). Unsupervised dense information retrieval with contrastive learning. *Transactions on Machine Learning Research*.

[32] Xiong, L., Xiong, C., Li, Y., Tang, K. F., Liu, J., Bennett, P., Ahmed, J., & Overwijk, A. (2021). Approximate nearest neighbor negative contrastive estimation for dense text retrieval. *International Conference on Learning Representations (ICLR 2021)*.

[33] Lin, J., Ma, X., Lin, S. C., Yang, J. H., Pradeep, R., & Nogueira, R. (2021). Pyserini: A Python toolkit for reproducible information retrieval research with sparse and dense representations. *Proceedings of the 44th Annual ACM SIGIR Conference on Research and Development in Information Retrieval*, 2356–2362.

[34] Robertson, S., & Zaragoza, H. (2009). The probabilistic relevance framework: BM25 and beyond. *Foundations and Trends in Information Retrieval*, 3(4), 333–389.

[35] Min, S., Krishna, K., Lyu, X., Lewis, M., Yih, W., Koh, P. W., Iyyer, M., Zettlemoyer, L., & Hajishirzi, H. (2023). FActScoring: Fine-grained atomic evaluation of factual precision in long form text generation. *Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP)*, 12076–12100.

[36] Lazaridou, A., Gribovskaya, E., Stokowiec, W., & Grigorev, N. (2022). Internet-augmented language models through few-shot prompting for open-domain question answering. *arXiv preprint arXiv:2203.05115*.

[37] Team, G., Riviere, M., Pathak, S., Sessa, P. G., Hardin, C., Bhupatiraju, S., Hussenot, L., Mesnard, T., Shahriari, B., Ramé, A., Ferret, J., Liu, P., Tafti, P., Frieson, L., Hussein, C., Asadiabadi, M., Raunak, V., Kasner, Z., Haas, R., … Bachem, O. (2025). Gemma 3 technical report. *arXiv preprint arXiv:2503.19786*.

[38] Izacard, G., Lewis, P., Lomeli, M., Hosseini, L., Petroni, F., Schick, T., Dwivedi-Yu, J., Joulin, A., Riedel, S., & Grave, E. (2023). Atlas: Few-shot learning with retrieval augmented language models. *Journal of Machine Learning Research*, 24(251), 1–43.

[39] Wang, L., Yang, N., Huang, X., Yang, L., Majumder, R., & Wei, F. (2024). Multilingual E5 text embeddings: A technical report. *arXiv preprint arXiv:2402.05672*.

[40] Es, S., James, J., Anke, L. E., & Schockaert, S. (2023). RAGAS: Automated evaluation of retrieval augmented generation. *arXiv preprint arXiv:2309.15217*.

---

## Cover Letter Draft

**Target Journal:** *Information Processing & Management* (Elsevier)  
ISSN: 0306-4573 | Impact Factor: ~8.6 | Scopus Q1 | CiteScore: ~14  
Submission system: Elsevier Editorial Manager — https://www.editorialmanager.com/ipm

*Rationale:* IP&M has published multilingual information retrieval benchmarks, RAG evaluation studies, and low-resource NLP papers. It accepts applied NLP dataset contributions with evaluation protocols, aligns with the paper's IR + hallucination measurement framing, and carries strong Scopus indexing relevant to Persian academic visibility.

---

Dear Editor,

We are pleased to submit our manuscript, "TekRAG-Persian: A Data-Centric Benchmark for Reducing Hallucination in Persian Technical Question Answering," for consideration as a research article in *Information Processing & Management*.

This manuscript addresses a practical and underexplored problem in retrieval-augmented generation: the effect of corpus quality, semantic metadata, and provenance-aware chunking on hallucination in Persian technical question answering. While RAG systems are increasingly deployed across enterprise and educational applications, systematic evaluation has remained heavily centered on English and general-domain datasets. Our work contributes a 35-article, 245-question Persian technical QA benchmark and a reproducible, data-centric evaluation protocol comparing raw, cleaned, and metadata-enriched corpora. By isolating corpus preparation as an independent variable, the study provides actionable guidance for practitioners building Persian-language RAG systems without requiring model fine-tuning. Generation is performed using Gemma 4 MoE 26B served locally, and retrieval uses Google Gemini text-embedding-004 embeddings indexed in Qdrant, ensuring full reproducibility.

We believe the manuscript is well-suited for *Information Processing & Management* because it combines applied artificial intelligence, information retrieval, low-resource NLP evaluation, and practical measurement of trustworthy language model systems — all active areas within the journal's scope and consistent with recent publications in the venue.

The manuscript is original, has not been published elsewhere, and is not currently under review by another journal. All authors have approved the submission. Data and code availability statements are included in the declarations section.

We look forward to your consideration.

Sincerely,

**Arsam Sabbagh**  
Teknav, Tehran, Iran  
admin@teknavglobal.com

---

## Pre-Submission Checklist

- [x] Finalize target journal → *Information Processing & Management* (Elsevier, IP&M).
- [ ] Read IP&M Guide for Authors (word limits, reference style APA/numbered, figure DPI).
- [ ] Register ORCID at orcid.org and add ID to title page.
- [ ] Complete and verify literature review against final reference list.
- [ ] Build raw corpus (Variant A) from 35 Teknav articles.
- [ ] Build cleaned corpus (Variant B): normalize Persian Unicode, remove boilerplate, section-aware chunking.
- [ ] Build metadata-enriched corpus (Variant C): add title, category, tags, author, date, summary, canonical URL, provenance to each chunk.
- [ ] Index all three variants in Qdrant (`tekrag_persian` collection, 768-dim Gemini embeddings, cosine distance).
- [ ] Create and human-review 245 Persian QA pairs (semi-automatic generation + manual annotation).
- [ ] Run retrieval experiments for all conditions at k = 3, 5, 8.
- [ ] Run generation experiments with Gemma 4 MoE 26B (Ollama local, temp=0.1).
- [ ] Complete human evaluation with minimum two annotators; compute Cohen's kappa.
- [ ] Fill all result tables (§8.1, §8.2, §8.3) with real experimental values.
- [ ] Update References section with any additional citations found during literature review.
- [ ] Run plagiarism / self-plagiarism check (iThenticate or similar).
- [ ] Prepare figures, charts, and supplementary material (pipeline diagram, per-category breakdown).
- [ ] Adapt formatting to IP&M Word/LaTeX template from Elsevier's author tools page.
- [ ] Submit through Editorial Manager: https://www.editorialmanager.com/ipm