As it was established that aging is not associated with massive neuronal loss, as was believed in the mid-20th Century, scientific interest has addressed the influence of aging on particular neuronal subpopulations and their synaptic contacts, which constitute the substrate for neural plasticity. Inhibitory neurons represent the most complex and diverse group of neurons, showing distinct molecular and physiological characteristics and possessing a compelling ability to control the physiology of neural circuits. This review focuses on the aging of GABAergic neurons and synapses. Understanding how aging affects synapses of particular neuronal subpopulations may help explain the heterogeneity of aging-related effects. We reviewed the literature concerning the effects of aging on the numbers of GABAergic neurons and synapses as well as aging-related alterations in their presynaptic and postsynaptic components. Finally, we discussed the influence of those changes on the plasticity of the GABAergic system, highlighting our results concerning aging in mouse somatosensory cortex and linking them to plasticity impairments and brain disorders. We posit that aging-induced impairments of the GABAergic system lead to an inhibitory/excitatory imbalance, thereby decreasing neuron's ability to respond with plastic changes to environmental and cellular challenges, leaving the brain more vulnerable to cognitive decline and damage by synaptopathic diseases.