We propose a novel adaptive, adjoint-based, iterative multiscale finite volume (i-MSFV) method. The method aims to reduce the computational cost of the smoothing stage of the original i-MSFV method by selectively choosing fine-scale sub-domains (or sub-set of primary variables) to solve for. The selection of fine-scale primary variables is obtained from a goal-oriented adjoint model. An adjoint-based indicator is utilized as a criterion to select the primary variables having the largest errors. The Lagrange multipliers from the adjoint model can be interpreted as sensitivities of the objective function value with respect to deviations from the constraints. In case of adjoining the porous media flow equations with Lagrange multipliers, this implies that the multipliers are the sensitivities of the objective function with respect to the residuals of the flow equations, i.e., to the residual error that remains after approximately solving linear equations with the aid of an iterative solver. This allow us to recognize at which locations the solution contains more errors. More specifically, we propose a modification to the i-MSFV method to adaptively reduce the size of the fine-scale system that must be smoothed. The aim is to make the fine-scale smoothing stage less computationally demanding. To that end, we introduce a goal-oriented, adjoint-based fine-scale system reduction criterion. We demonstrate the performance of our method via single-phase, incompressible flow simulation models with challenging geological settings and using a history-matching like misfit objective function as the goal. The performance of the newly introduced method is compared to the original i-MSFV method. We investigate the adaptivity versus accuracy of the method and demonstrate how the solution accuracy varies by varying the number of unknowns selected to be smoothed. It is shown that the method can provide accurate solutions at reduced computational cost. The proof-of-concept applications indicate that the method deserves further investigations.