The trend toward shorter delivery lead times reduces operational efficiency and increases transportation costs for Internet retailers. However, mobile technology creates new opportunities to organize the last mile. In this paper, we study the concept of crowdsourced delivery that aims to use excess capacity on journeys that already take place. We consider a service platform that automatically creates matches between parcel delivery tasks and ad hoc drivers. The platform also operates a fleet of dedicated vehicles to serve the tasks that cannot be served by the ad hoc drivers. The matching of tasks, drivers, and dedicated vehicles in real time gives rise to a new variant of the dynamic pickup and delivery problem. We propose a rolling horizon framework and develop an exact solution approach to solve the matching problem each time new information becomes available. To investigate the potential benefit of crowdsourced delivery, we conduct a wide range of computational experiments. The experiments provide insights into the viability of crowdsourced delivery under various assumptions about the behavior of the ad hoc drivers. The results suggest that the use of ad hoc drivers has the potential to make the last mile more cost-efficient and can provide system-wide vehicle-mile savings up to 37% compared to a traditional delivery system with dedicated vehicles.