Diffusion tensor imaging (DTI) has been used extensively to investigate white matter (WM) microstructural changes during healthy adult aging. However, WM fibers are known to shrink throughout the lifespan, leading to larger interstitial spaces with age. This could allow more extracellular free water molecules to bias DTI metrics, which are relied upon to provide WM microstructural information. Using a cohort of 212 participants, we demonstrate that WM microstructural changes in aging are potentially less pronounced than previously reported once the free water compartment is eliminated. After free water elimination, DTI parameters show age-related differences that match histological evidence of myelin degradation and debris accumulation. The fraction of free water is further shown to associate better with age than any of the conventional DTI parameters. Our findings suggest that DTI analyses involving free water are likely to yield novel insight into retrospective re-analysis of data and to answer new questions in ongoing DTI studies of brain aging.