An Adversarial Approach to Hard Triplet Generation

An overview of the paper “An Adversarial Approach to Hard Triplet Generation”. The paper brings to notice that the major challenge lies in distinguishing similar images from different categories. All images and tables in this post are from their paper.

The current state-of-the-art is to mine the most hard triplet examples from the mini-batch to train the network. However, mining-based methods tend to look into these triplets that are hard in terms of the current estimated network, rather than deliberately generating those hard triplets that really matter in globally optimizing the network. For this purpose, the authors propose an adversarial network for Hard Triplet Generation (HTG) to optimize the network ability in distinguishing similar examples of different categories as well as grouping varied examples of the same categories.

Hard Triplet Generation

We use triplet loss since, it is proved to perform better than contrastive loss. Here, $d(x_1,x_2)$ is the squared Euclidean distance between two l2-normalized feature vectors.

Adversarial Triplet Generator G

$G$ learns to produce challenging triplets of examples by pushing apart the vectors from the same category while pulling closer the vectors from different categories.The objective function of $G$ looks like: $\min(d(a,n)-d(a,p)+m,0)$ . Note, this isnt same as triplet loss, as the $d(x,y)-d(x,z)$ has been inverted here. The objective function to train $F$ looks like a simple triplet loss. $F$ is trained through hard triplets of examples generated by $G$ by pulling the positive pair closer and pushing the negative pair apart to meet the margin $m$ .

Multi-category Discriminator D

Without a discriminator, we might get random noisy triplets. The discriminator $D$ categories it into $K+1$ categories, where the first $K$ categories are real classes and the last is a fake class. We try to minimize $L = L_{\mathit{real}}+bL_{\mathit{fake}}$ . Here $L_{\mathit{real}}$ is average of softmax loss of $a$ , $a$ and $n$ falling in the right labels. We do the same for $L_{\mathit{fake}}$ . To enforce label preservation assumption in $G$ , we create a new loss $L_{\mathit{cls}}$ which is similar to $L_{\mathit{fake}}$ , but assuming that the generator was successful in generating a triplet. Hence, $G$ will now be minimizing $L_{\mathit{tri}} + bL_{\mathit{cls}}$ .

Written on August 3, 2020