The standard architecture of neocortex is a network with excitation and inhibition in closely maintained balance. These networks respond fast and with high precision to their inputs and they allow selective amplification of patterned signals. The stability of such networks is known to depend on balancing the strengths of positive and negative feedback. We here show that a second condition is required for stability which depends on the relative strengths and time courses of fast (AMPA) and slow (NMDA) currents in the excitatory projections. This condition also determines the response time of the network. We show that networks which respond quickly to an input are necessarily close to an oscillatory instability which resonates in the delta range. This instability explains the existence of neocortical delta oscillations and the emergence of absence epilepsy. Although cortical delta oscillations are a network-level phenomenon, we show that in non-pathological networks, individual neurons receive sufficient information to keep the network in the fast-response regime without sliding into the instability.