Multilingual and multimodal retrieval may use one shared embedding space or separate model-specific spaces. Never compare vectors across incompatible models. Represent each modality, language strategy, task, and model explicitly; fuse ranked results only after each retrieval path applies authorization. Evaluate every important language, modality, and cross-modal direction with representative labels.
Use separate tables when dimensions differ. role can distinguish title, body, image, summary, or query-specific representations. Do not average vectors automatically: averaging can erase a rare but decisive signal.
A multilingual model may place multiple languages in a shared space, enabling a query in one language to retrieve a document in another. Test it; model marketing is not domain evidence. Also preserve language metadata for filters, display, reranking, and fallback translation. Exact names and morphology may need language-specific full-text configurations.
Some models intentionally align text and images. If they do, a text query may search image vectors. Otherwise, retrieve in separate spaces and fuse lists. Store captions, OCR, timestamps, bounding boxes, page numbers, and content hashes for citation and reprocessing.
Build a matrix: query language/modality by document language/modality. Label positive, hard-negative, and no-result cases. Report metrics per cell, not only one average. Confirm authorization in every branch before fusion and citation.
Design a multilingual and multimodal retrieval experiment for [languages, modalities, use case]. Specify model spaces, schema, per-branch authorization, fusion, deduplication, provenance, and a complete evaluation matrix. Identify cells with insufficient labels.
Verification contract: No aggregate score may be reported without per-language and per-modality slices, and every fused result must trace back to an authorized source representation.