For the study and application of graphene membranes, it is essential to have means to control their resonance frequency and temperature. Here, we present an on-chip heater platform for local tuning of in-plane tension in graphene mechanical resonators. By Joule heating of a metallic suspension ring we show thermomechanical resonance frequency tuning in a few-layer (FL) graphene nanodrum, which is accompanied by an increase in its quality factor, which we attribute to the increase of the in-plane tension. The in situ control of temperature, in-plane tension, resonance frequency, and quality factor of suspended two-dimensional (2D) nanodrums makes this device a unique platform for investigating the origin of dissipation in these ultrathin structures and can be of fundamental importance for studying the thermal properties of 2D materials. Moreover, by simultaneously controlling the heater and the backgate voltage, we can independently control the resonance frequency and quality factor, which is of great importance for applications in sensors and resonant mechanical filters.