TY - M-10133
AU - Htet, Zaw Ye
AU - Aung, Tin Shine
TI - Implementation Approach for Duplicate Image Identification and Removal
T2 - Scientific Research Journal of Science, Engineering and Technology

PY - 2024
VL - 2
IS - 1
SN - 2584-0584

AB - This paper presents a systematic approach for identifying and removing duplicate images from various 3D image format collections. The identification process considers image structure, density, meta descriptions, and other properties. The system employs a preprocessing module to standardise and extract meta descriptions from diverse formats like STL, OBJ, FBX, and others. A vector database, utilising tools like FAISS or Milvus, stores the image vectors and meta descriptions for efficient similarity searches. Deep learning models, particularly Convolutional Neural Networks (CNNs), are trained to extract image features and compare vectors using cosine similarity or Euclidean distance. An integrated search engine allows users to find similar images by uploading an image and its meta description. A human validation interface is provided for manual confirmation of potential duplicates. This approach ensures efficient management and retrieval of 3D images while enhancing storage utilisation. Future work will further explore alternative models and similarity measures to improve system accuracy and efficiency.

KW - Duplicate image identification
KW -  3D image formats
KW -  image structure
KW -  image density
KW -  meta descriptions
KW -  preprocessing module
KW -  Vision Transformers
DO -