Turkish Text Normalization and Phonetic Matching Methods

A technical guide separating deasciification, character normalization, Numeric Soundex, edit distance, and syllabification by the problem each method actually solves.

Several different Turkish text problems are often collapsed into the label “fuzzy matching.” Restoring sahin to şahin, placing Şahin and Sahin in the same candidate bucket, computing edit distance, and finding syllable boundaries are different tasks. Applying one algorithm to all of them makes the output difficult to interpret.

Classify the problem first

restore Turkish characters from ASCII  → deasciification
make Turkish/ASCII variants share a key → character normalization
build phonetic candidate buckets        → Numeric Soundex-like key
measure character-level proximity       → edit distance
find syllable boundaries                → syllabifier
perform morphological analysis          → a separate NLP problem

Deasciification is an inference problem

Turning cagri into çağrı is not just a character table. ASCII letters such as c, g, i, o, s, and u can map differently depending on context. Full deasciification therefore attempts to infer the appropriate Turkish form.

Treating Ş and S as the same initial symbol in an index is different. That is a normalization policy, not a reconstruction inference.

A phonetic key generates candidates

A key such as Numeric Soundex does not prove that two names identify the same person or word. Its job is to reduce the search space before a more expensive comparison. Different names can share a key; collisions are expected.

A typical pipeline is therefore:

normalize
→ phonetic key
→ candidate bucket
→ secondary comparison
→ application decision

The secondary comparison can be edit distance, another name-similarity measure, or a domain rule.

Edit distance is not phonetics

Levenshtein distance counts insertions, deletions, and substitutions needed to transform one sequence into another. It sees characters, not Turkish phonology. Relationships such as ASCII/Turkish variants or phonetic substitutions require preprocessing or a custom cost model.

Phonetic keys and edit distance therefore need not compete. One can perform candidate generation while the other ranks candidates.

Syllabification is not identity matching

Syllable boundaries are useful for speech, teaching, generation, and linguistic preprocessing. They do not by themselves form a meaningful identity metric. A finite-state Turkish syllabifier is therefore not a more sophisticated Soundex; it solves a different problem.

Measure the pipeline, not only the key

For name matching, at least two stages should be evaluated:

candidate recall: did the true match enter the candidate set?
candidate reduction: how much of the corpus was eliminated?
final precision/recall: what errors remained after secondary comparison?

A key with very few collisions can still be poor if it separates genuine variants and destroys candidate recall.