We identify experimentally the magnetostatic modes active for Brillouin light scattering in the optical whispering gallery modes of a yttrium iron garnet sphere. Each mode is identified by magnetic-field dispersion of ferromagnetic-resonance spectroscopy and coupling strength to the known field distribution of the microwave drive antenna. Our optical measurements confirm recent predictions that higher-order magnetostatic modes can also generate optical scattering, according to the selection rules derived from the axial symmetry. From this we summarize the selection rules for Brillouin light scattering. We give experimental evidence that the optomagnonic coupling to nonuniform magnons can be higher than that of the uniform Kittel mode.