The sluggish adoption of building information modeling (BIM) is attributable to various technical, managerial, personnel, procedural, and institutional issues encountered by an organization in which such adoption takes place. However, these issues are under researched from a holistic perspective. Based on a proposed human-organization-technology (HOT) fit model, this paper aims to study the impacting factors of HOT fit in BIM adoption within construction project organizations (CPOs). The HOT fit indexes of 14 BIM case projects were operationalized using social network analysis (SNA) method and how different factors impact the HOT fit and its three subdimensions [i.e., human-technology (HT) fit, organization-technology (OT) fit, and human-organization (HO) fit] was investigated using a comparative case study. It was found that the project size has significantly negative relations with HOT fit, HT fit, and OT fit; while hierarchy steepness has positive correlations with HT fit, OT fit, and HO fit. OT fit was also found to have a weakly negative relationship with BIM level of details (LODs). A joint factor analysis further disclosed that the flatter the hierarchy, the larger the project size, and the higher the BIM LOD, the more difficult to achieve a high HOT fit, HT fit, or OT fit. Thus, CPOs should use steeper hierarchical structure and take a progressive BIM adoption strategy by adopting from smaller projects and/or lower LODs. This research empirically examined how project organizational and technological factors can impact BIM adoption. The HOT fit model can help CPOs evaluate their general HOT fit status, redesign optimal HOT configuration, diagnose the problems when the HOT fit is not ideal, and make strategic directions to better harvest the benefits of BIM. Limitations and future research directions are also identified.