100%(3)3 out of 3 people found this document helpful
This preview shows page 6 - 9 out of 12 pages.
model, for the first time, show that RBF models can achieve competitive accu-racy versus softmax models. To be specific, the last layer of this model is madeup of multiple weight matrix per class. This is because, the model learns eachclass centroid, and defines uncertainty as the distance to the closest centroid us-ing RBF. Considering loss function, it calculates binary cross entropy loss witheach class, and summate these losses. After loss is calculated, centroid of theclass are updated using and exponential moving average of the feature vectorsof data belonging to that class.3.1.2Evaluations and future investigationsI think that this method is notable in that it quantifies uncertainty using dis-tance function. OoD detection is one of the major drawback of classifier trainedin supervised manner. I had a chance to test few approaches based on sam-pling methods using Dropout layer.(Gal and Ghahramani,2016) But, there is alatency problem, since the model should run single input data multiple times.This method can solve this problem, because it measures uncertainty with single6
Figure3—A depiction of the architecture of DUQ. The input ismapped to feature space, where it is assigned to the closest cen-troid. The distance to that centroid is the uncertainty.inference in deterministic way.When quantifying uncertainty, uncertainty is separated out as "epistemic" and"aleatoric" uncertainty. Epistemic uncertainty is from the parameters of the model,and aleatoric uncertainty is uncertainty inherent in the data, for example,3lookssimilar to8. DUQ captures these uncertainties as a whole, so that future re-search should investigate how to distinguish uncertainties using RBF. And, theexperiment in this paper uses simple dataset such as CIFAR-10and SVHN inclassification task. I think ablation study with more scalable dataset is requiredto check which factors are critical in identifying OoD samples.3.2Paper2:FixMatch: Simplifying Semi-Supervised Learning with Consistencyand Confidence, NeurIPS’20(Sohn et al.,2020)3.2.1Summary and contributionsThis paper presents new semi-supervised learning(SSL) method called FixMatch,which is made up of two common methods, consistency regularization andpseudo-labeling. To be specific, as Figure4shows, FixMatch follows few steps,firstly, a weakly-augmented image is fed into the model to obtain its predic-tions. And if the model assigns a probability to any class which is above athreshold(dotted line in Figure4), the prediction is converted to a one hotpseudo-label. The pseudo-label simply means that it utilizes its own predictionon unlabeled data as true label. Then, the strongly-augmented version of thesame image is prepared, and the model is trained to make its prediction onthis image match the pseudo-label via a standard cross entropy loss. The back-7
ground of this method is from consistency regularization. It utilizes unlabeleddata by relying on the assumption that the model should output similar predic-tions when fed perturbed versions of the same image. Considering augmenta-