Visual Reasoning(3): A simple neural network module for relational reasoning

DeepMind的一项工作
https://www.zhihu.com/question/60784169/answer/180518895
https://zhuanlan.zhihu.com/p/28654835

Introduction

前面编的故事都没什么，分析了下symbolic&neural network。主要是提出了RN，解决了relational reasoning
Here, we explore “Relation Networks” (RN) as a general solution to relational reasoning in neural networks.

Methods

首先把image送进conv layers, 然后feature上的每个pixel都对应一个"object"，任意两个object都会构成一对特征：

Question会用LSTM进行embedding，final state of the LSTM is concatenated to each object-pair

object-pair+LSTM state，都经过MLP，再叠加又经过一个MLP。然后后面接一些FC layers最后softmax分类到某个答案词上面

模型配置细节

真的很简单的模型啊，
这一小块，以object-pair+LSTM state为输入，强行学习了之间的relation，进行relation reasoning

Experiments & Conclusion

效果惊人，只比上一篇用到了GT program的效果差一点点，比传统VQA model好太多。不过也是有点针对CLEVR数据集，毕竟这里面的relation算是很简单了，这样reasoning很奏效~

看看别人的评价：