Visual inspection of bridges is customarily used to identify and evaluate faults. However, current procedures followed by human inspectors demand long inspection times to examine large and difficult to access bridges. To address these limitations, we investigate a computer vision‐based approach that employs SIFT keypoint matching on collected images of defects against a pre-existing reconstructed 3D point cloud of the bridge. We also investigate methods of reducing computation time with ML-based and conventional CV methods of segmentation to eliminate redundant keypoints. Our project successfully localizes the defect images and achieves a savings in runtime from filtering keypoints.