A generalization of the S-parameter retrieval method for finite three-dimensional inhomogeneous objects under arbitrary illumination and observation conditions is presented. The effective permittivity of such objects may be rigorously defined as a solution of a nonlinear inverse scattering problem. We confirm analytically and observe numerically effects that were previously reported in the one-dimensional strongly inhomogeneous slabs: the nonuniqueness of the effective permittivity and its dependence on the illumination and observation conditions, and the geometry of the object. Moreover, we show that, although the S-parameter retrieval of the effective permittivity is scale free at the level of problem statement, the exact solution of this problem either does not exist or is not unique. Using the results from the spectral analysis we describe the set of values of the effective permittivity for which the scattering problem is ill-posed. Unfortunately, real nonpositive values, important for negative refraction and invisibility, belong to this set. We illustrate our conclusions using a numerical reduced-order inverse scattering algorithm specifically designed for the effective permittivity problem.