One of the limitations of ride-sharing is that matched drivers and riders need to have similar itineraries and desired arrival times for ride-sharing to be competitive against other transport modes. By allowing a single transfer at a designated transfer hub, their itineraries need to be only partially similar, and therefore more matching options are created. In this paper, we develop an optimal matching approach that matches riders to drivers, taking into account multi-modal routing options to model competition and collaboration between multiple modes of transport. We allow for transfers between modes and between multiple drivers. We model this as a path-based integer programming problem and we develop a simulated annealing algorithm to efficiently solve realistic large-scale instances of the problem. Our analysis indicates that a single transfer hub can reduce significantly the average generalized cost of riders and the total vehicle hours traveled by creating efficient matches. As opposed to previous studies, our work shows that ride-sharing not only attracts former public transport users but also former private car users. By allowing for intermodal transfers and by choosing the cost parameters such that transfers are favorable, itineraries where commuters use their car first, before sharing a ride on the second part of their journey, becomes an appealing alternative. Multi-modal ride-matching with transfers has the potential to increase ride-sharing, reduce the number of vehicle hours traveled in private cars, and reduce the number of cars that are present in urban areas during peak hours of congestion.