Face processing supports our ability to recognize friend from foe, form tribes and understand the emotional implications of changes in facial musculature. This skill relies on a distributed network of brain regions, but how these regions interact is poorly understood. Here we integrate anatomical and functional connectivity measurements with behavioural assays to create a global model of the face connectome. We dissect key features, such as the network topology and fibre composition. We propose a neurocognitive model with three core streams; face processing along these streams occurs in a parallel and reciprocal manner. Although long-range fibre paths are important, the face network is dominated by short-range fibres. Finally, we provide evidence that the well-known right lateralization of face processing arises from imbalanced intra- and interhemispheric connections. In summary, the face network relies on dynamic communication across highly structured fibre tracts, enabling coherent face processing that underpins behaviour and cognition.