Nonlinear interaction effects in a three-mode cavity optomechanical system

We investigate the resonant enhancement of nonlinear interactions in a three-mode cavity optomechanical system with two mechanical oscillators. Using the Keldysh Green's function technique, we find that nonlinear effects on the cavity density of states can be greatly enhanced by the resonant interaction of two phononic polaritons, due to their small effective dissipation. In the large detuning limit and taking into account an upper bound on the achievable dressed coupling, the optimal point for probing the nonlinear effect is obtained, showing that such a three-mode system can exhibit significant nonlinear features also for relatively small values of g/kappa.