SHENZHEN UNIVERSITY · CHINA
Fine-grained Textual Inversion Network for Zero-Shot Composed Image Retrieval (FTI4CIR)
PhD · Taught in en
At a glance
—
Duration
—
IELTS
—
Tuition / yr
en
Language
PhD
Level
- Novel Fine-grained Textual Inversion Network (FTI4CIR)Methodology
- Addresses Zero-Shot Composed Image Retrieval (ZS-CIR)Task Focus
- Uses subject-oriented and attribute-oriented pseudo-word tokensRepresentation
- Incorporates caption-based semantic regularizationTraining Technique
- Demonstrated superiority on benchmark datasetsPerformance
This research introduces a novel method called the Fine-grained Textual Inversion Network (FTI4CIR) designed for Zero-Shot Composed Image Retrieval (ZS-CIR). Composed Image Retrieval allows users to search for images using a combination of a reference image and a text description that specifies desired modifications. ZS-CIR aims to achieve this without requiring pre-annotated image-text triplets, addressing the challenge of expensive data annotation. The FTI4CIR network addresses limitations in existing ZS-CIR approaches that use a single pseudo-word token to represent an image. This can be too coarse to capture an image's full content. FTI4CIR proposes a more comprehensive approach by mapping an image into multiple textual representations: a subject-oriented pseudo-word token and several attribute-oriented pseudo-word tokens. This fine-grained mapping aims to more accurately represent the image in a textual format. Furthermore, FTI4CIR incorporates a tri-wise caption-based semantic regularization. This component works to align these fine-grained pseudo-word tokens with real-word token embeddings, guided by image captions generated using the BLIP model. The effectiveness of this method has been demonstrated through extensive experiments on three benchmark datasets, showing superior performance.
Frequently asked questions
- What is the focus of the FTI4CIR program?
- The FTI4CIR program focuses on developing a novel Fine-grained Textual Inversion Network for Zero-Shot Composed Image Retrieval (ZS-CIR). It aims to improve image search by using a reference image and text modifications without needing expensive pre-annotated data.
- What innovative approach does FTI4CIR use for image representation?
- FTI4CIR maps an image into multiple textual representations: a subject-oriented pseudo-word token and several attribute-oriented pseudo-word tokens. This fine-grained mapping provides a more comprehensive textual representation of the image content.
- How does FTI4CIR ensure the accuracy of its textual representations?
- The program incorporates a tri-wise caption-based semantic regularization component. This aligns the fine-grained pseudo-word tokens with real-word token embeddings, guided by image captions generated using the BLIP model.
- Where was the FTI4CIR research presented?
- This research was presented at the prestigious IEEE/CVF International Conference on Computer Vision, demonstrating its academic recognition and impact in the field.
Accreditation
This research was presented at the IEEE/CVF International Conference on Computer Vision.
Frequently asked questions
What is the focus of the FTI4CIR program?
+
The FTI4CIR program focuses on developing a novel Fine-grained Textual Inversion Network for Zero-Shot Composed Image Retrieval (ZS-CIR). It aims to improve image search by using a reference image and text modifications without needing expensive pre-annotated data.
What innovative approach does FTI4CIR use for image representation?
+
FTI4CIR maps an image into multiple textual representations: a subject-oriented pseudo-word token and several attribute-oriented pseudo-word tokens. This fine-grained mapping provides a more comprehensive textual representation of the image content.
How does FTI4CIR ensure the accuracy of its textual representations?
+
The program incorporates a tri-wise caption-based semantic regularization component. This aligns the fine-grained pseudo-word tokens with real-word token embeddings, guided by image captions generated using the BLIP model.
Where was the FTI4CIR research presented?
+
This research was presented at the prestigious IEEE/CVF International Conference on Computer Vision, demonstrating its academic recognition and impact in the field.