Algorithmic Confusion: A Transversal Study of Computational Trade Mark Similarity

T. Vandamme, Algorithmic Confusion: A Transversal Study of Computational Trade Mark Similarity, Doctoral Thesis in Engineering and Technology, ULB, 2025 Supervisors: Julien Cabay (JurisLab), Olivier Debeir (LISA) Jury : Jean-Marc Deltorn (Université de Strasbourg), Dev Gangjee (University of Oxford), Aniket Kesari (Fordham University, NYC), Matei Mancas (Université de Mons), Andrée Puttemans (ULB), Dimitris Sacharidis (ULB) Recent advancements in Artificial Intelligence (AI) have generated significant optimism across various fields, including Trade Mark (TM) Law. In particular, AI-driven TM retrieval systems promise to streamline the registration process by accurately identifying similar prior marks. Within the European Union, this process hinges on the complex legal test known as the Likelihood of Confusion (LoC), which includes a multifactorial assessment of the similarity of the signs. This thesis undertakes a comprehensive, transversal study to evaluate how effectively current AI systems can translate this central aspect of the legal standard into functional algorithms. By conducting a comparative analysis of two closed-source TM search engines provided by public Intellectual Property offices (namely the Benelux Office for Intellectual Property – BOIP and the European Union Intellectual Property Office – EUIPO), alongside an in-depth review of the state of the art algorithm, we assess the true capabilities of these systems and expose major methodological flaws in the development of those technologies. We curate and release two new benchmark datasets rooted in case law, aimed at enhancing the relevance and precision of these AI tools. This thesis raises the confusion around those algorithms and their ability to perform the ad-hoc similarity assessment at the heart of the confusion test, while also offering insights into the challenges and limitations of algorithmic regulation and the indispensable need for AI accountability.

(English version below)   La question de l’existence de similitudes pertinentes entre deux objets est centrale dans le droit de la propriété intellectuelle (PI). Extrêmement complexe, la réponse est entièrement laissée aux offices de PI et aux magistrats, respectivement dans le cadre de procédures administratives et judiciaires, sans véritables outils d’analyse. Des systèmes de décision algorithmique (ADS) ont toutefois vu le jour, principalement développés par des entreprises privées, et font désormais l’objet d’une utilisation croissante aux fins de l’application des droits de PI (surveillance des marchandises contrefaites en ligne, filtrage de contenus) et de l’enregistrement par les offices. Afin de limiter les biais et de sauvegarder les valeurs d’intérêt public associées à la PI, le développement de ces outils devrait idéalement être entrepris sous la supervision d’experts indépendants. Dans cette optique, ce projet de recherches financé dans le cadre Actions de Recherches Concertées (ARC) et avec le soutien de l’Office Benelux de la Propriété Intellectuelle (OBPI) vise à concevoir un modèle ouvert, supervisé et transparent d’analyse des similitudes en PI (IPSAM), en se concentrant sur les images 2D. A travers ce projet interdisciplinaire, des juristes (JurisLab) et ingénieurs (LISA) examineront les défis méthodologiques, techniques, juridiques et éthiques de la construction de tels outils, afin de fournir une étude critique des solutions technologiques actuellement disponibles. Les résultats du projet permettront de contribuer de manière inédite aux débats autour de la régulation algorithmique en général et dans le domaine de la PI en particulier. Promoteur : Julien Cabay   ***   The question whether two objects are similar in a relevant manner is core to intellectual property law (IP). The answer is extremely complex and entire left to IP Offices and judges, respectively in the frame of administrative and judicial proceedings, without any proper analytic tools. Yet, algorithmic decision systems (ADS) are currently being developed and used by private companies for the purposes of IP enforcement (monitoring infringing goods online, filtering out content) and registration by IP Offices, outside of public scrutiny. In order to limit biases and to safeguard public interest values associated with IP, ideally the development of such tools should be carried out under the supervision of independent experts. In that regard, this research project funded under the ‘Actions de Recherches Concertées’ framework (ARC) and supported by the Benelux Office Office for Intellectual Property (BOIP) aims at proposing an open source, supervised and transparent model for IP Similary Assesment (IPSAM) in relation to 2D images. Through this interdisciplinary project, lawyers (JurisLab) and engineers (LISA) will review the methodological, technical, legal, and ethical challenges of building such tools, in order to provide a critical study of the currently available technological solutions. Expected findings of the project will make an original contribution to the general debate on algorithmic regulation and its ramification in the specific field of IP. Promotor : Julien Cabay   ***   Publications : T. Vandamme, J. Cabay, O. Debeir, “A Quantitative Evaluation of Trademark Search Engines’ Performances through Large-Scale Statistical Analysis“, in X., Proceedings of the Nineteenth International Conference on Artificial Intelligence and Law (ICAIL 2023), June 19–23, 2023, Braga, Portugal, ACM, New York (NY, USA), 2023, pp. 343-350 J. Cabay, T. Vandamme, O. Debeir, “Looking through the Crack in the Black Box: A Comparative Case Law Benchmark for Auditing AI-Powered Trade Mark Search Engines“, 37 p., forthcoming (acceptance decision pending) T. Vandamme, Algorithmic Confusion: A Transversal Study of Computational Trade Mark Similarity, Doctoral Thesis in Engineering and Technology, Université libre de Bruxelles, 2025, 217 p., forthcoming (public defense on 17 April 2025) Conferences : J. Cabay, T. Vandamme, “Assessing IP Similarities Through Technology: A Trademark Exploration of Challenges and Avenues”, Webinar Artificial Intelligence Technology & Policy Talk organized by the Digital Law Center, University of Geneva, 4 November 2021 (video ; slides) J. Cabay, T. Vandamme,”Le recours à la technologie pour l’analyse des similitudes en matière de propriété intellectuelle: défis et perspectives en droit des marques”, Conférence du Centre de droit privé, ULB, 8 novembre 2021 (slides) J. Cabay, T. Vandamme, “AI-Powered Trademark Prior Art Search Tools: An Empirical Analysis ”, Conference  4th Waseda Brussels Conference – New Technologies and Regulation in Japan and Europe organized by the ULB and the Waseda University, Bip – House of the Capital Region, Brussels, 13 September 2022 (video ; slides) J. Cabay, T. Vandamme,  “Fundamental Rights, Copyright and Content Recognition (The Good, the Bad ant the Ugly?)”, Conférence lors de la Journée d’étude de l’Association belge pour le droit d’auteur (ABA) La transposition de la Directive 2019/790 sur le droit d’auteur et les droits voisins dans le marché unique numérique en droit belge, The Hotel, Bruxelles, 21 avril 2023 (slides) J. Cabay, T. Vandamme, “Experimenting TDM (I, Robot ?)”, Conférence lors de la Journée d’étude de l’Association belge pour le droit d’auteur (ABA) La transposition de la Directive 2019/790 sur le droit d’auteur et les droits voisins dans le marché unique numérique en droit belge, The Hotel, Bruxelles, 21 avril 2023 (slides) J. Cabay, “Artificial Intelligence, Intellectual Property and Fundamental Rights Intertwined – With some Lessons from IPSAM”, Webinar in the framework of the University Diploma Artificial Intelligence and Intellectual Property, Centre d’études internationales de la propriété intellectuelle (CEIPI), Université de Strasbourg,  23 January 2024 (slides) J. Cabay, T. Vandamme, “Legally Blind? Findings on Transparency and Explainability through a Study of AI-powered TM Search Engines”, DRAILS Workshop, Université Catholique de Louvain, March 26, 2024 (slides) J. Cabay, T. Vandamme, “Legally Blind? Findings on Transparency and Explainability through a Study of AI-powered TM Search Engines”, Midi du Centre de droit privé, Université libre de Bruxelles, March 27, 2024 (slides) J. Cabay, T. Vandamme, “Legally Blind? Findings on Transparency and Explainability through a Study of AI-powered TM Search Engines”, Seminar on Trademarks and New Digital Technologies, Sciences Po Paris, March 28, 2024 (slides) J. Cabay, “Computing Likelihood of Confusion? Lessons from IPSAM”, Trademark Law Institute Symposium on Proportitionality in Trademark Law, Maastricht University, 6-7 June 2024 (slides) J. Cabay, T. Vandamme, “Deep Learning for the Purpose of Trademark Similarity Assessment”, Journée d’études L’Ecole polytechnique de Bruxelles: Passé, Présent, Futur, ULB, 13 mars 2025 (slides) J. Cabay, “Computing Likelihood of Confusion? Lessons from IPSAM”, Webinar in the framework of the University Diploma Artificial Intelligence and Intellectual Property, Centre d’études internationales de la propriété intellectuelle (CEIPI), Université de Strasbourg, 24 March 2025 J. Cabay, T. Vandamme, “Deep Learning for the Purpose of Trademark Similarities Assessment in the EU and the Benelux”, Research Center for the Legal System of Intellectual Property (RCLIP) Evening Seminar, Waseda Univerity, Tokyo, 28 March 2025 J. Cabay, T. Vandamme, “Cross-Jurisdictional Development of AI-Powered Trademark Search Engines: A Comparison based on EU, Benelux and Japanese Trademarks”, 7th IP & Innovation Researchers of Asia (IPIRA) Conference, Waseda University, Tokyo, 5 April 2025 J. Cabay, T. Vandamme, “Computation of the Likelihood of Confusion Test”, AI & Trademarks workshop organized by City University of London, 9 June 2025  

Les candidatures pour l’année 2025-2026 sont ouvertes. Une séance d’information est organisée le vendredi 23 février 2024 de 13h à 14h en hybride au FabLab et sur Teams. Toutes les informations sont disponibles ici. L’échéance pour le dépôt des candidatures est fixée au mardi 21 février à midi.   The ULB is a member of the Pan-European Seal Traineeship Programme (Pan-European Seal). The Pan-European Seal is a comprehensive traineeship programme that bridges academia and the labour market in different fields (IP, law, finance, business, engineering, etc.) to promote and disseminate Intellectual Property among the Academic Community. It is promoted in partnership with two of the world’s largest IP offices, the European Union Intellectual Property Office (EUIPO) in Alicante, Spain and the European Patent Office (EPO) in Munich, Germany and their strategic University partners. This programme gives high-achieving, young university graduates access to a year-long (12 months) paid traineeship at either the EUIPO or the EPO, helping them get a foot in the door of the competitive world in a variety of fields through valuable, on-the-job, multicultural and professional work experience. Within the context of the ongoing fight against high levels of youth unemployment in Europe and the commitment thereof to social responsibility, the Pan-European Seal will offer traineeship posts every year to graduates of their university partners, administered by both of the abovementioned offices. The number of posts offered is determined on an annual basis. Application and selection of ULB prospective students is organized at JurisLab – FabLab ULB. Information are provided annually through institutional communication.