Evidence-based frameworks have helped to transform decisions in medicine, education, agriculture, and international development. Conservation has lagged other fields in embracing the data-driven revolution, largely because of the difficulty and expense of collecting ecological data over large areas and long time-scales. The decline of forest cover, species loss, and impacts of climate change make it imperative that we develop better tools to measure ecological change and conservation outcomes. Passive acoustic sensors that expand survey effort and machine learning techniques that automate data-processing, are one approach for collecting robust and cost-effective ecological metrics at the required scale. We present data from three case studies where we have developed and applied Deep Learning models to analyze passive acoustic survey data to 1) detect rare and elusive species, 2) estimate population trends from call rates, and 3) quantify wildlife impacts in the built environment and test potential mitigation measures. Our approach to acoustic analysis has contributed rigorous metrics to inform decisions for more than 100 conservation monitoring projects around the globe over the last six years. We discuss our efforts to develop transparent and efficient acoustic analysis workflows, our experience on the relative strengths and limitations of acoustic approaches, and the potential for combining soundscape indices with detection/classification approaches focused on target-species.