Home surveillance systems available today notify homeowners of any motion detected by the cameras. This behavior leads to mostly false positives and annoying notifications. Worse, these systems suffer from the “one size fits all” approach and do not offer any ability to provide personalized feedback and make the notifications only appear for unknown persons and objects. The personalization can be based on the unique features of a home’s landscape, location and known persons and pets. Our team intends to address this gap. We are proposing to develop the building blocks for a smart home surveillance system. The system will use video streams from off-the-shelf outdoor IP cameras. It will analyze and interpret the content of the streams and employ feedback mechanisms to generate customizable notifications for the homeowner. We plan to implement an end to end pipeline that can detect objects and faces in the data streams in order to recognize unfamiliar faces, identify anomalous situations and notify the homeowner about the anomaly. The feedback loop will enable the system to improve its capabilities over time and generate notifications that are personalized for each homeowner.