Proposed a computationally-light and robust neural network for estimating heart rate in remote healthcare systems. The model can be trained on consumer-grade graphics processing units (GPUs), and can be deployed on edge devices for swift inference. The proposed hybrid model is based on convolutional neural network (CNN) and bidirectional long short-term memory (BiLSTM) architectures for estimating heart rate from Electrocardiogram (ECG) and Photoplethysmography (PPG) signals. Considering the sensitive nature of the ECG signals, a formal privacy guarantee, differential privacy, is ensured for the model training. A tight accounting of the overall privacy budget of our training algorithm is performed using the Rényi Differential Privacy technique. The model outperforms state-of-the-art networks on a benchmark dataset for both ECG and PPG signals despite having a much smaller number of trainable parameters and, consequently, much smaller training and inference times. The CNN-BiLSTM architecture can also provide excellent heart rate estimation performance even under strict privacy constraints. A prototype Arduino-based data collection system is also developed that is low-cost, efficient, and useful for providing access to modern healthcare services to people living in remote areas. This work led to this Elsevier Healthcare Analytics publication.