We propose a computational procedure to assess size effects in nonfunctionalized single-walled carbon nanotube (CNT)-polymer composites. The procedure upscales results obtained with atomistic simulations on a composite unit cell with one CNT to an equivalent continuum composite model with a large number of CNTs. Molecular dynamics simulations demonstrate the formation of an ordered layer of polymer matrix surrounding the nanotube. This layer, known as the interphase, plays a central role in the overall mechanical response of the composite. Due to poor load transfer from the matrix to the CNT, the reinforcement effect attributed to the CNT is negligible; hence the interphase is regarded as the only reinforcement phase in the composite. Consequently, the mechanical properties of the interface and the CNT are not derived since their contribution to the elastic response of the composite is negligible. To derive the elastic properties of the interphase, we employ an intermediate continuum micromechanical model consisting of only the polymer matrix and a three-dimensional fiber representing the interphase. The Young's modulus and Poisson's ratio of the equivalent fiber, and therefore of the interphase, are identified through an optimization procedure based on the comparison between results from atomistic simulations and those obtained from an isogeometric analysis of the intermediate micromechanical model. Finally, the embedded reinforcement method is employed to determine the macroscopic elastic properties of a representative volume element of a composite with various fiber volume fractions and distributions. We then investigate the role of the CNT diameter on the elastic response of a CNT-polymer composite; our simulations predict a size effect on the composite elastic properties, clearly related to the interphase volume fraction.