Quantum entanglement in deep learning architectures

We establish a Tensor Network (TN) based common language between the fields of deep learning and many-body quantum physics, which allows us to offer bidirectional contributions. By showing that many-body wave-functions are structurally equivalent to mappings of ConvNets and RNNs, we construct their TN equivalents, and suggest quantum entanglement measures as natural quantifiers of dependencies in such networks. Accordingly, we propose a novel entanglement based deep learning design scheme. In the other direction, we identify that an inherent re-use of information in state-of-the-art deep learning architectures is a key trait that distinguishes them from standard TNs. Therefore, we employ a TN manifestation of information re-use and construct TNs corresponding to powerful architectures such as deep recurrent and overlapping convolutional networks. This allows us to prove that the entanglement scaling supported by state-of-the-art deep learning architectures matches that of MERA TN in 1D, and that they support volume law entanglement in 2D, polynomially more efficiently than RBMs. We thus provide theoretical motivation to shift trending neural-network based wave-function representations closer to state-of-the-art deep learning architectures.