# ONNX and CTranslate2 for ArcFace Models > An ArcFace-trained ONNX face-embedding model cannot be moved directly to CTranslate2 merely because its graph is available in ONNX; the supported model architecture and inference graph must also match. - Author: Muhammet Ali Köker - Language: en - Canonical: https://alikoker.com.tr/en/onnx-ctranslate2-arcface-architectural-comparison - Translation: https://alikoker.com.tr/arcface-onnx-ctranslate2 - Published: 2026-08-04T12:00:00+03:00 - Modified: 2026-09-08T02:30:00+03:00 - Verified: 2026-09-08T02:30:00+03:00 - Type: article ## ArcFace loss and embedding inference [ArcFace](/en/wiki/arcface) is not the name of the convolutional backbone that processes facial images, but an additive angular margin loss used during training. Its objective is to bring facial representations of the same person closer in angular space while increasing angular separation among different classes. A fixed margin is added to the angle of the true class over normalized features and class weights. While working with Dlib- and InsightFace-based face-recognition models and [ONNX](/en/wiki/onnx) inference pipelines, I repeatedly encountered the assumption that ArcFace denotes the model backbone itself. This article starts from that practical distinction and separates the training loss, embedding backbone, model interchange format, and runtime support. A simplified ArcFace loss is: ```text exp(s · cos(θyi + m)) Lᵢ = -log ------------------------------------------------ exp(s · cos(θyi + m)) + Σj≠yi exp(s · cos θj) ``` Where: - `θyi` is the angle between, for example, the true class weight and the feature vector. - `m` is the additive angular margin. - `s` is the coefficient that scales normalized logits. - `yi` is the true identity class, for example. After training, the classification head is not used in many face-verification systems. The required component during inference is the backbone that transforms an aligned facial image into a fixed-dimensional embedding vector: ```text x ∈ R^(3×112×112) → f(x) ∈ R^512 ``` The embedding is commonly normalized with the L2 norm: ```text f(x) f̂(x) = ------------- ||f(x)||₂ ``` Similarity between two faces can be computed as the inner product of normalized embedding vectors: ```text similarity(a, b) = f̂(a)ᵀ f̂(b) ``` The object intended for conversion is not an "ArcFace layer." It is an inference graph of an image backbone, such as ResNet, iResNet, MobileFaceNet, or a similar architecture, with weights trained using the ArcFace loss. ## ONNX format does not guarantee conversion A model being available in ONNX format does not mean that it can be converted to any [inference engine](/en/wiki/inference-engine). ONNX carries a computation graph and weights in a common representation. The target runtime must additionally be able to parse the graph, support its nodes, and satisfy its execution semantics. The following distinctions must therefore be maintained: - Model format is not model architecture. - Operator support is not full graph support. - Weight quantization is not model conversion. - Output dimensionality is not runtime compatibility. An ArcFace-based InsightFace model may be a valid `.onnx` file and may produce a 512-dimensional facial representation. Nevertheless, it cannot be directly converted into a CTranslate2 model using the current CTranslate2 conversion tools. General-purpose ONNX runtimes use a different execution approach: ```text read the ONNX graph ↓ resolve each node's operator and attributes ↓ schedule the graph in topological order ↓ dispatch to an appropriate kernel or hardware provider ↓ produce intermediate tensors ``` ONNX Runtime can load supported graphs, optimize them, and distribute subgraphs to CPU, CUDA, TensorRT, OpenVINO, or other Execution Provider components. However, the opset version, custom operators, dynamic shapes, and the operator coverage of a hardware provider may still prevent execution. ## The architectural boundary of CTranslate2 CTranslate2 is not a general-purpose ONNX graph interpreter. It is a specialized C++ and Python inference library developed for efficient inference of [Transformer](/en/wiki/transformer)-based models. Its optimizations, such as weight quantization, layer fusion, and batch reordering, target Transformer execution patterns. The current converters do not provide a general entry point that accepts arbitrary ONNX graphs. Architecture-aware converters exist for specific model families in the OpenNMT, Fairseq, and Hugging Face Transformers ecosystems. Supported families include selected Transformer models such as BERT, T5, Llama, Whisper, and NLLB. General CNN image models are outside this scope. CTranslate2 conversion approximately follows this flow: ```text source framework model ↓ identify a known model class ↓ match expected parameters by name ↓ create a CTranslate2 model specification ↓ store weights in the target format ``` This approach knows the model family and inference flow in advance. No built-in `ArcFaceConverter`, `ResNetConverter`, or general `ONNXConverter` exists for an InsightFace face-recognition graph. It is not sufficient for a converter merely to read weights and map them to similar layers; the target must also provide a model specification and execution class representing the same graph. ## Operators and numerical validation Evaluating compatibility solely by operator names is misleading. The presence of matrix multiplication, activation, normalization, or certain convolution operations in a runtime does not demonstrate that an arbitrary CNN graph can be executed. ### Operator semantics Operations with the same name may use different data layouts, axes, padding, or broadcasting rules. A convolution node may, for example, include the following parameters: - `kernel_size` - `stride` - `padding` - `dilation` - `groups` - `input_layout` - `weight_layout` The common input layout in image models is `[N, C, H, W]`. The existence of a `Conv` operation in a codebase does not mean that all attribute combinations of the ONNX `Conv` operator are supported. ### Graph topology InsightFace backbones do not consist only of sequential convolution layers. They include residual connections, normalization layers, activations, and shape transformations: ```text x ────────────────┐ │ Conv → BN → PReLU → Conv → BN │ Add → output ``` Even if a target runtime can execute nodes individually, it must support branching, merging, and tensor lifetime rules. CTranslate2 works with defined structures of supported model types rather than free-form operator graphs. For a facial embedding backbone, a built-in model class would need to define the following questions: - What is the input image shape? - How are residual blocks ordered? - Where are feature-map dimensions reduced? - How is the final embedding layer constructed? - How are batch-normalization parameters applied? - Will the output be normalized? Loadability after conversion is also insufficient. Numerical equivalence must be tested: ```text yONNX = fONNX(x) ytarget = ftarget(x) max_abs_error = max |yONNX - ytarget| ``` Directional drift is additionally important in embedding systems: ```text cosine_drift = 1 - cosine_similarity(yONNX, ytarget) ``` If face-verification decisions are made for samples near a cosine-similarity threshold, small numerical differences can alter an accept-or-reject result. A working conversion does not mean that biometric decision behavior has been preserved. ## Inference engine and embedding contract For an ArcFace-based ONNX facial embedding model, ONNX Runtime is a natural starting point. Without changing the model format, Execution Provider options such as CPU, CUDA, TensorRT, and OpenVINO can be used under the same API. TensorRT can also be evaluated on NVIDIA GPUs with fixed input shapes and high batch loads. For Intel CPU, iGPU, or NPU targets, OpenVINO can load the model directly or convert it in advance to its own model format. On ARM-based edge devices, suitable ONNX Runtime CPU providers or mobile-oriented runtimes such as ncnn can be compared. ncnn provides a path for converting ONNX models to its own model format through `pnnx`. Engine selection should not be based only on single-image inference time. The following metrics should be measured: - Single-request latency - Sustained throughput - Batch scaling - p95 and p99 latency - Model loading time - Runtime memory consumption - Host-device copy cost - Multi-session behavior - [Embedding](/en/wiki/embedding) drift under precision changes Total latency in a face-recognition pipeline does not consist only of the embedding model: ```text Ttotal = Tdecode + Tdetect + Talign + Tcopy + Tembedding + Tsearch ``` An acceleration in embedding inference may have limited effect on the overall system if face detection or vector search is dominant. Moreover, two models producing 512-element vectors does not show that those vectors occupy the same embedding space: ```text f₁(x) ∈ R^512 f₂(x) ∈ R^512 ``` These expressions do not imply `f₁(x) ≈ f₂(x)`. The embedding space is formed jointly by training data, backbone, loss function, preprocessing, alignment, color-channel order, and normalization rules. The following contract must be preserved when changing the runtime: - Input size - RGB/BGR order - Pixel scaling - Mean and standard deviation - Face-alignment geometry - Output tensor name and order - L2 normalization - Similarity metric - Decision threshold With [INT8](/en/wiki/int8) quantization, model size or inference time alone should not be assessed. Similarity scores, decision thresholds, and ranking behavior in nearest-neighbor search must also be validated. Adding CNN facial embedding support to CTranslate2 is theoretically possible at the source-code level; however, it requires a new model specification, a converter, CPU and GPU kernel paths, memory planning, API extensions, and equivalence tests for different backbones. This cost would most likely exceed that of using an engine already optimized for CNNs and ONNX graphs. While ONNX carries a general computation graph, tensors, and operators, a CTranslate2 model represents a supported model family, a known inference flow, and an optimized weight layout. Unless a general translator exists between these abstraction levels, the `.onnx` extension does not provide CTranslate2 compatibility. ## Verify the Model, Format, and Runtime as Separate Claims The three technical objects in this article have different evidence sources. ArcFace is defined as an additive angular-margin training loss in the [CVPR 2019 paper](https://doi.org/10.1109/CVPR.2019.00482). The meaning of an ONNX file depends on the graph and operator contracts in the ONNX IR specification. CTranslate2 separately documents the model families, operators, and execution paths supported by its runtime. A successful export is therefore not proof of numerical equivalence. The converted pipeline should be evaluated on the same inputs using measurements such as `max_abs_error`, cosine similarity or drift, and the final decision thresholds that matter to the application. Those metrics are a verification method, not a claim that one universal tolerance is correct for every model; the acceptable bound depends on the use case and downstream decision rule. ## A Convertible Model Is Not Necessarily an Equivalent Model Successful export to ONNX does not prove that another inference runtime preserves the same embedding space. Preprocessing order, normalization, tensor layout, operator substitutions, and numerical precision can all shift score distributions, which matters directly in threshold-based face verification. A useful conversion test therefore goes beyond comparing one output tensor. Embedding norms, cosine-similarity distributions, and samples close to the operating threshold should be compared on the same dataset. [Embedding Drift](/en/wiki/embedding-drift) captures why production equivalence has to be validated above the model-file level. ## References - **[1]** Jiankang Deng; Jia Guo; Niannan Xue; Stefanos Zafeiriou. (2019). ArcFace: Additive Angular Margin Loss for Deep Face Recognition. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. [doi:10.1109/CVPR.2019.00482](https://doi.org/10.1109/CVPR.2019.00482) ### Sources and technical basis - **[2]** ONNX Project. (n.d.). Open Neural Network Exchange Intermediate Representation (ONNX IR) Specification. ONNX Project. [URL](https://onnx.ai/onnx/repo-docs/IR.html) - **[3]** OpenNMT. (n.d.). CTranslate2 Documentation. OpenNMT. [URL](https://opennmt.net/CTranslate2/) ## Cite This Work Köker, M. A. (2026). ONNX and CTranslate2 for ArcFace Models. alikoker.com.tr. https://alikoker.com.tr/en/onnx-ctranslate2-arcface-architectural-comparison - BibTeX: https://alikoker.com.tr/en/onnx-ctranslate2-arcface-architectural-comparison.bib - RIS: https://alikoker.com.tr/en/onnx-ctranslate2-arcface-architectural-comparison.ris - CSL-JSON: https://alikoker.com.tr/en/onnx-ctranslate2-arcface-architectural-comparison.csl.json