These days, there is an increasing trend of opting for federal government structures to address perplexing political issues of different groups in the countries around the world. As federations are established based on their own political context, every federal system on this planet has its own unique features. But, all revolve around striking the balance between unity and diversity. Protection of rights of minority groups is also among the most important concerns of federations. Lack of accommodation of diversity through shared federal institutions and inadequate representation of groups in a federal system may disturb the whole union. This is more realistic in multicultural federal systems like India, Nigeria and Ethiopia. This author will address the following two basic issues: How diversity could be sufficiently accommodated? And, what mechanisms should be used to hold the units together efficiently? Therefore, this work aims at identifying problems and demonstrating feasible solutions regarding protection of diverse groups and preservation of the integrity of a union in federal systems. Thus, this work is an important input for those who want to explore on federal systems.