Developed a novel non-negative matrix factorization (NMF) algorithm capable of operating on sensitive data, while closely approximating the results of the non-private algorithm. Additionally, we consider the effect of outliers by specifically modeling them, such that the presence of outliers has very little effect on our estimated differentially-private basis matrix. This theoretical work is also validated on six real datasets of different applications. This work led to this ACM Transactions on Knowledge Discovery from Data publication.