We describe a technique to simultaneously estimate a weighted, positive-definite multi-tensor fiber model and perform tractography. Existing techniques estimate the local fiber orientation at each voxel independently so there is no running knowledge of confidence in the estimated fiber model. We formulate fiber tracking as recursive estimation: at each step of tracing the fiber, the current estimate is guided by the previous. To do this we model the signal as a weighted mixture of Gaussian tensors and perform tractography within a filter framework. Starting from a seed point, each fiber is traced to its termination using an unscented Kalman filter to simultaneously fit the local model and propagate in the most consistent direction. Further, we modify the Kalman filter to enforce model constraints, i.e. positive eigenvalues and convex weights. Despite the presence of noise and uncertainty, this provides a causal estimate of the local structure at each point along the fiber. Synthetic experiments demonstrate that this approach significantly improves the angular resolution at crossings and branchings while consistently estimating the mixture weights. In vivo experiments confirm the ability to trace out fibers in areas known to contain such crossing and branching while providing inherent path regularization.