Turkish Text Normalization and Phonetic Matching Methods

A technical guide separating deasciification, character normalization, Numeric Soundex, edit distance, and syllabification by the problem each method actually solves.

Several different Turkish text problems are often collapsed into the label “fuzzy matching.” Restoring sahin to şahin, placing Şahin and Sahin in the same candidate bucket, computing edit distance, and finding syllable boundaries are different tasks. Applying one algorithm to all of them makes the output difficult to interpret.

Classify the problem first

restore Turkish characters from ASCII  → deasciification
make Turkish/ASCII variants share a key → character normalization
build phonetic candidate buckets        → Numeric Soundex-like key
measure character-level proximity       → edit distance
find syllable boundaries                → syllabifier
perform morphological analysis          → a separate NLP problem

Deasciification is an inference problem

Turning cagri into çağrı is not just a character table. ASCII letters such as c, g, i, o, s, and u can map differently depending on context. Full deasciification therefore attempts to infer the appropriate Turkish form.

Treating Ş and S as the same initial symbol in an index is different. That is a normalization policy, not a reconstruction inference.

A phonetic key generates candidates

A key such as Numeric Soundex does not prove that two names identify the same person or word. Its job is to reduce the search space before a more expensive comparison. Different names can share a key; collisions are expected.

A typical pipeline is therefore:

normalize
→ phonetic key
→ candidate bucket
→ secondary comparison
→ application decision

The secondary comparison can be edit distance, another name-similarity measure, or a domain rule.

Edit distance is not phonetics

Levenshtein distance counts insertions, deletions, and substitutions needed to transform one sequence into another. It sees characters, not Turkish phonology. Relationships such as ASCII/Turkish variants or phonetic substitutions require preprocessing or a custom cost model.

Phonetic keys and edit distance therefore need not compete. One can perform candidate generation while the other ranks candidates.

Syllabification is not identity matching

Syllable boundaries are useful for speech, teaching, generation, and linguistic preprocessing. They do not by themselves form a meaningful identity metric. A finite-state Turkish syllabifier is therefore not a more sophisticated Soundex; it solves a different problem.

Measure the pipeline, not only the key

For name matching, at least two stages should be evaluated:

candidate recall: did the true match enter the candidate set?
candidate reduction: how much of the corpus was eliminated?
final precision/recall: what errors remained after secondary comparison?

A key with very few collisions can still be poor if it separates genuine variants and destroys candidate recall.

Split the evaluation set by error class

A single random train/test split does not show which Turkish name-matching problem a method actually solves. Evaluation pairs should be separated into classes such as ASCII/Turkish-character variation, typographical error, whitespace or punctuation variation, phonetic proximity, genuine homophone or homograph conflicts, and clearly unrelated negatives. This exposes cases where an apparently better aggregate score hides a regression in one error class.

At the candidate-generation stage, the primary question is not precision but whether the true match entered the candidate set. Candidate recall should therefore be reported together with average and percentile bucket size. A very broad bucket can preserve recall while making the second stage prohibitively expensive; an overly selective key can look efficient while dropping the true match before ranking even begins.

The second-stage threshold should be fixed on a validation set rather than tuned against the final test set. In identity-matching applications where false positives can be costly, ROC/AUC is not sufficient by itself; precision, recall, and the absolute false-positive count at the intended operating point should also be reported under a class balance that resembles production use.

Choosing Methods for Turkish Text Normalization and Phonetic Matching