PURPOSE: Characterizing the relation between the applied gradient sequences and the measured diffusion MRI signal is important for estimating the time-dependent diffusivity, which provides important information about the microscopic tissue structure. THEORY AND METHODS: In this article, we extend the classical theory of Stepi\v snik for measuring time-dependent diffusivity under the Gaussian phase approximation. In particular, we derive three novel expressions which represent the diffusion MRI signal in terms of the mean-squared displacement, the instantaneous diffusivity, and the velocity autocorrelation function. We present the explicit signal expressions for the case of single diffusion encoding and oscillating gradient spin-echo sequences. Additionally, we also propose three different models to represent time-varying diffusivity and test them using Monte-Carlo simulations and in vivo human brain data. RESULTS: The time-varying diffusivities are able to distinguish the synthetic structures in the Monte-Carlo simulations. There is also strong statistical evidence about time-varying diffusivity from the in vivo human data set. CONCLUSION: The proposed theory provides new insights into our understanding of the time-varying diffusivity using different gradient sequences. The proposed models for representing time-varying diffusivity can be utilized to study time-varying diffusivity using in vivo human brain diffusion MRI data.