主题：Deep image reconstruction from the brain

报告人：Yukiyasu Kamitani, Ph.D.

Professor，

Neuroinformatics Laboratory，

Department of Information Science and Technology，

Graduate School of Informatics，

Kyoto University

时间：2018年8月16日（周四），13:00-14:30

地点：北京大学王克桢楼1113会议室

摘要：

The internal visual world is thought to be encoded in hierarchical representations in the brain. However, previous attempts to visualize perceptual contents based on machine-learning analysis of fMRI patterns have been limited to reconstructions with low level image bases or to the matching to exemplars. While categorical decoding of imagery contents has been demonstrated, the reconstruction of internally generated images has been challenging. I introduce our recent study showing that that visual cortical activity can be decoded (translated) into the hierarchical features of a pre-trained deep neural network (DNN) for the same input image, providing a way to make use of the information from hierarchical visual features (Horikawa & Kamitani, Nature Communications 2017). Next I present a novel image reconstruction method, in which the pixel values of an image are optimized to make its DNN features similar to those decoded from human brain activity at multiple layers (Shen, Horikawa, Majima, Kamitani, bioRxiv 2017). We found that our method was able to reliably produce reconstructions that resembled the viewed natural images. While our model was solely trained with natural images, it successfully generalized to artificial shapes, indicating that our model was not simply matching to exemplars. The same analysis applied to mental imagery demonstrated rudimentary reconstructions of the subjective content. Our method can effectively combine hierarchical neural representations to reconstruct perceptual and subjective images, providing a new window into the internal contents of the brain.

邀请人： 方方 教授