The combination of Transcranial Magnetic Stimulation (TMS) with Electroencephalography (EEG) exposes the brains global response to localized and abrupt stimulations. However, large electric artifacts are induced in the EEG by the TMS, obscuring crucial stages of the brains response. Artifact removal is commonly performed by data processing techniques. However, an experimentally verified physical model for the origin and structure of the TMS-induced discharge artifacts, by which these methods can be justified or evaluated, is still lacking. We re-examine the known contribution of the skin in creating the artifacts, and outline a detailed model for the relaxation of the charge accumulated at the electrode-gel-skin interface due to the TMS pulse. We then experimentally validate implications set forth by the model. We find that the artifacts decay like a power law in time rather than the commonly assumed exponential. In fact, the skin creates a power-law decay of order 1 at each electrode, which is turned into a power law of order 2 by the reference electrode. We suggest an artifact removal method based on the model which can be applied from times after the pulse as short as 2 milliseconds onwards to expose the full EEG from the brain. The method can separate the capacitive discharge artifacts from those resulting from cranial muscle activation, demonstrating that the capacitive effect dominates at short times. Overall, our insight into the physical process allows us to accurately access TMS-evoked EEG responses that directly follow the TMS pulse, possibly opening new opportunities in TMS-EEG research.