Wearable devices have paved way for several context-aware applications in the field of health-care and sports to improve the well-being of users and their performance for human augmentation. During rehabilitation patients need accurate feedback that can empower and improve the speed of recovery. On the other hand competitive athletes need a reliable, flexible and real-time feedback on their performance and technique. In this thesis, we present a framework capable of performing Micro-Activities Recognition (MAR) by decomposing complex activities. These models employ data only from the wearable devices. We present two real-world applications, viz., (i) the analysis of the Lunge exercise performed during knee rehabilitation and (ii) the study of the Stroke activity in long-track speed skating. Hitherto, most of the models used additional data such as camera and 3D tracking for identifying activities. The models proposed in this thesis aims to go one step forward to understand fine-grained activity (micro-activity) information. In knee rehabilitation, we proposed models to identify the exercise performed by the patient and its micro-activities. Providing feedback in these systems is non-trivial due to the overlapping labels. The feedback provided using the models proposed is based on the labels that are highly similar. In speed skating, we aim to identify the micro-activities of the stroke to determine its frequency, and other characteristics of the speed skaters. The model identified the top signal/IMU that can classify a stroke and its micro-activities accurately. The top signal identified the correct number of strokes across all laps. The model was also able to classify a stroke performed in straight and curve sections. Furthermore, the average length and offset values of a stroke for a complete lap is 5.4% and 135 ms respectively. In this thesis, we derive fine-grained activity information using data from IMUs. The models identified the top signal that maximizes the micro-activity recognition among all the signals. This information can be used to determine the optimal placement of IMUs and also to reduce the data collection/processing. The fine-grained information obtained using MAR can provide meaningful feedback for human augmentation systems.