The nonlinearity is essential for creation of non-classical states of the cavity or mechanical resonator such as squeezed or cat states. A microwave cavity can be made nonlinear by, for instance, adding Josephson junctions. The mechanical resonator is inherently nonlinear. The radiation pressure interaction between cavity and mechanical resonator is also inherently nonlinear but typically under strong drive of the cavity interaction can be linearized. However, if the optomechanical system is in the strong coupling regime nonlinear quantum effects become observable. These three cases provide the motivation for our studies. We start with backaction analysis of the classical regime of a dc SQUID (superconducting quantum interference device) with an embedded mechanical resonator. Then, we perform quantum analysis to understand how the asymmetry of two Josephson junctions influences the coupling strength of optomechanical interactions such as radiation pressure, cross-Kerr and single-photon beam splitter. Using variational method and self-consistent harmonic approximation we estimate the influence of the radiation pressure coupling and Kerr nonlinearity of the cavity on the effective frequency and dissipation of the cavity. The second part of this thesis focuses on the nonlinearity of the mechanical resonator coupled to the optical/microwave cavity. We study in details the nonlinear optomechanical response and draw the response map to summarize all results of the overcoupled and undercoupled cavity for the red and blue sidebands additionally to the weak and strong drive powers. In the end, w e confirm the developed theory by using it to numerically fit the experimental results. @en

We investigate a superconducting interference device (SQUID) with two asymmetric Josephson junctions coupled to a mechanical resonator embedded in the loop of the SQUID. We quantize this system in the case when the frequency of the mechanical resonator is much lower than the cavity frequency of the SQUID and in the case when they are comparable. In the first case, the radiation pressure and the cross-Kerr type interactions arise and are modified by the asymmetry. The cross-Kerr type coupling is the leading term at the extremum points where the radiation pressure is zero. In the second case, the main interaction is the single-photon beam splitter, which exists only at a finite asymmetry. Another interaction in this regime is of cross-Kerr type, which exists at all asymmetries, but is generally much weaker than the beam splitter interaction. Increasing magnetic field can substantially enhance the optomechanical couplings strength with a potential for the radiation pressure coupling to reach the single-photon strong coupling regime, even the ultrastrong coupling regime, in which the single-photon coupling rate exceeds the mechanical frequency.

@en

We experimentally investigate the nonlinear response of a multilayer graphene resonator using a superconducting microwave cavity to detect its motion. The radiation pressure force is used to drive the mechanical resonator in an optomechanically induced transparency configuration. By varying the amplitudes of drive and probe tones, the mechanical resonator can be brought into a nonlinear limit. Using the calibration of the optomechanical coupling, we quantify the mechanical Duffing nonlinearity. By increasing the drive force, we observe a decrease in the mechanical dissipation rate at large amplitudes, suggesting a negative nonlinear damping mechanism in the graphene resonator. Increasing the optomechanical backaction further, we observe instabilities in the mechanical response.

@en