The axon plasma membrane consists of the membrane skeleton, which comprises ring-like actin filaments connected to each other by spectrin tetramers, and the lipid bilayer, which is tethered to the skeleton via, at least, ankyrin. Currently it is unknown whether this unique axon plasma membrane skeleton (APMS) sets the diffusion rules of lipids and proteins in the axon. To answer this question, we developed a coarse-grain molecular dynamics model for the axon that includes the APMS, the phospholipid bilayer, transmembrane proteins (TMPs), and integral monotopic proteins (IMPs) in both the inner and outer lipid layers. We first showed that actin rings limit the longitudinal diffusion of TMPs and the IMPs of the inner leaflet but not of the IMPs of the outer leaflet. To reconcile the experimental observations, which show restricted diffusion of IMPs of the outer leaflet, with our simulations, we conjectured the existence of actin-anchored proteins that form a fence which restricts the longitudinal diffusion of IMPs of the outer leaflet. We also showed that spectrin filaments could modify transverse diffusion of TMPs and IMPs of the inner leaflet, depending on the strength of the association between lipids and spectrin. For instance, in areas where spectrin binds to the lipid bilayer, spectrin filaments would restrict diffusion of proteins within the skeleton corrals. In contrast, in areas where spectrin and lipids are not associated, spectrin modifies the diffusion of TMPs and IMPs of the inner leaflet from normal to confined-hop diffusion. Overall, we showed that diffusion of axon plasma membrane proteins is deeply anisotropic, as longitudinal diffusion is of different type than transverse diffusion. Finally, we investigated how accumulation of TMPs affects diffusion of TMPs and IMPs of both the inner and outer leaflets by changing the density of TMPs. We showed that the APMS structure acts as a fence that restricts the diffusion of TMPs and IMPs of the inner leaflet within the membrane skeleton corrals. Our findings provide insight into how the axon skeleton acts as diffusion barrier and maintains neuronal polarity.