Disease control programs need to identify breeding sites of mosquitoes which transmit malaria and other diseases to target interventions and identify environmental risk factors. Increasing availability of very high resolution drone data provides new opportunities to find and characterize these vector breeding sites. Within this study, we identified land cover types associated with malaria vector breeding sites in West Africa. Drone images from two malaria endemic regions in Burkina Faso and Côte d’Ivoire were assembled and labeled using open-source tools. We developed and applied a workflow using region of interest-based and deep learning methods to classify these habitat types from very high resolution natural color imagery. Analysis methods achieved a dice coefficient ranging between 0.68 and 0.88 for different vector habitat types; however, this classifier consistently identified the presence of specific habitat types of interest. This establishes a framework for developing deep learning approaches to identify vector breeding sites and highlights the need to evaluate how results will be used by control programs.