Traditional centralized power plants have limited ability to adapt to the varying power demands caused due to the increasing deployment of renewable energy sources. For power grids, willing to increase the use of renewable energy and thereby decrease the energy bills, demand side energy management could act as an effective solution. Demand side energy management of the power grid refers to the process of regulating the power demands of the devices it serves. A large fraction of this power demand on the grid lines is caused due to Thermostatically Controlled Loads such as residential refrigerators, electric water heaters, air conditioners, industrial heaters, ovens, etc. Traditionally, the energy management of these devices is achieved using model predictive control and linear quadratic regulation. To better handle the system heterogeneity and computation costs, model-free adaptive control algorithms are explored.