This paper investigates the impact of spatially correlated unobservable variables on the refinancing, selling, and default decisions of mortgage borrowers. Virtually the entire mortgage literature acknowledges that borrower specific characteristics, such as culture, education, or access to information, play an important role in the mortgage termination decisions. While we do not observe these variables directly, we note that borrowers of similar background tend to cluster together in neighborhoods. We propose a method to take advantage of this information and reconcile the theoretical option-based models of mortgage terminations with the empirical experience of mortgage refinancing, sale and default. Specifically, we combine the three-stage maximum likelihood estimation (3SMLE) approach for competing risks hazard model with random effect proposed by Deng and Quigley (2002) with the space-varying coefficient method (SVC) of Pavlov (2000) to modify the covariance structure according to the spatial distribution of the observations. Beyond a significant improvement of the model performance, this yields a number of insightful implications for mortgage termination behavior. For instance, borrowers of the affluent "West Side" of Los Angeles County both refinance and move at a higher rate than predicted by the standard maximum likelihood estimation method. At the same time, borrowers from some lower-valued neighborhoods tend to stay longer than expected with their mortgages and properties. Such findings have direct implications for mortgage pricing and have the potential to ultimately improve the equity and efficiency of the lending markets.