To meet the challenges of ubiquitous computing, ambient technologies and an increasingly older population, researchers have been trying to break away from traditional modes of interaction. The studies over the past 30 years suggest that Gesture Controlled User interfaces (GCUI) now offer realistic and affordable opportunities, which may be appropriate for the elderly and disabled. Indeed, the advent of computer games technologies demonstrated the fulfilment of this trend. To verify affordability, usability, inclusive design and effectiveness of a GCUI, a prototype application has been to carry out everyday activities such as making phone calls, controlling a television and performing mathematical calculations described. Simple hand gestures have been used to perform a diverse range of tasks via a television screen. The best configuration of development and use of a GCUI has been identified which is then proposed as a baseline GCUI framework. The framework can inform application development and future enhancement in this domain. The evaluation study shows GCUI can provide independence for a wide range of users including elderly and disabled. This is a PhD research based book.