EVALUATING OPENAI AND VOYAGEAI EMBEDDING MODELS FOR E-COMMERCE SEMANTIC SEARCH: A PERFORMANCE AND USER PERCEPTION ANALYSIS

Abstract

The rapid evolution of e-commerce necessitates advanced search func-tionalities that can accurately capture user intent beyond traditional keyword matching. This research investigates e-commerce user experi-ence enhancement through embedding-based semantic search by compar-atively analyzing two state-of-the-art models: OpenAI's text-embedding-3-large (3072 dimensions, Model A) and VoyageAI's voyage-3-large (1024 dimensions, Model B). A functional e-commerce platform utiliz-ing Next.js, Supabase, and Pinecone was developed to facilitate this comparison. The methodology employed a blind comparative assess-ment framework with domain specialists (N=9) who evaluated search results for varied query typologies (short, medium, long, and ambigu-ous), supplemented by system log analysis and thematic analysis of qualitative feedback. Key findings indicate that while Model B (VoyageAI) generally produced higher cosine similarity scores, particu-larly for specific queries, a majority of users preferred Model A (Open-AI) for its ability to deliver more "nuanced" and contextually diverse results, especially for ambiguous or complex queries where the experi-mental dataset might lack exact product matches. This user preference is partly attributed to Model A's higher dimensionality potentially captur-ing a broader semantic field. A notable latency trade-off was observed, with Model B offering faster embedding generation and Model A providing quicker vector search times in the tested configuration. This study concludes that the selection of an embedding model presents criti-cal trade-offs for e-commerce platforms, balancing retrieval precision, user-perceived relevance in exploratory search scenarios with imperfect catalogs, and operational efficiency. The findings offer empirical in-sights for practitioners aiming to optimize product discovery mecha-nisms.