Model for the first time show that rbf models can

This preview shows page 6 - 9 out of 12 pages.

model, for the first time, show that RBF models can achieve competitive accu- racy versus softmax models. To be specific, the last layer of this model is made up of multiple weight matrix per class. This is because, the model learns each class centroid, and defines uncertainty as the distance to the closest centroid us- ing RBF. Considering loss function, it calculates binary cross entropy loss with each class, and summate these losses. After loss is calculated, centroid of the class are updated using and exponential moving average of the feature vectors of data belonging to that class. 3 . 1 . 2 Evaluations and future investigations I think that this method is notable in that it quantifies uncertainty using dis- tance function. OoD detection is one of the major drawback of classifier trained in supervised manner. I had a chance to test few approaches based on sam- pling methods using Dropout layer.(Gal and Ghahramani, 2016 ) But, there is a latency problem, since the model should run single input data multiple times. This method can solve this problem, because it measures uncertainty with single 6
Figure 3 A depiction of the architecture of DUQ. The input is mapped to feature space, where it is assigned to the closest cen- troid. The distance to that centroid is the uncertainty. inference in deterministic way. When quantifying uncertainty, uncertainty is separated out as "epistemic" and "aleatoric" uncertainty. Epistemic uncertainty is from the parameters of the model, and aleatoric uncertainty is uncertainty inherent in the data, for example, 3 looks similar to 8 . DUQ captures these uncertainties as a whole, so that future re- search should investigate how to distinguish uncertainties using RBF. And, the experiment in this paper uses simple dataset such as CIFAR- 10 and SVHN in classification task. I think ablation study with more scalable dataset is required to check which factors are critical in identifying OoD samples. 3 . 2 Paper 2 : FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence, NeurIPS’ 20 (Sohn et al., 2020 ) 3 . 2 . 1 Summary and contributions This paper presents new semi-supervised learning(SSL) method called FixMatch, which is made up of two common methods, consistency regularization and pseudo-labeling. To be specific, as Figure 4 shows, FixMatch follows few steps, firstly, a weakly-augmented image is fed into the model to obtain its predic- tions. And if the model assigns a probability to any class which is above a threshold(dotted line in Figure 4 ), the prediction is converted to a one hot pseudo-label. The pseudo-label simply means that it utilizes its own prediction on unlabeled data as true label. Then, the strongly-augmented version of the same image is prepared, and the model is trained to make its prediction on this image match the pseudo-label via a standard cross entropy loss. The back- 7
ground of this method is from consistency regularization. It utilizes unlabeled data by relying on the assumption that the model should output similar predic- tions when fed perturbed versions of the same image. Considering augmenta-

  • Left Quote Icon

    Student Picture

  • Left Quote Icon

    Student Picture

  • Left Quote Icon

    Student Picture