Field Programmable Gate Arrays (FPGAs) have scaled continuously in speed, capacity and energy efficiency, allowing the integration of ever-larger systems into a single FPGA chip. This brings challenges to the productivity of developers in leveraging the sea of FPGA resources. Higher level of design abstractions and programming models are needed to improve the design productivity, which in turn ask for memory architectural supports on FPGAs. Given the wide range of FPGA applications, it is impractical to address the diverse needs with a single generic architecture. Instead, a library consisting of common building blocks is more promising. This work investigates three key problems including scheduling, switching and caching, that are critical to memory system performance. For each of them, a driving application is studied to understand the true design requirements, and the physical characteristics of FPGAs are exploited to achieve efficient solutions. The efforts lead to performance improvements, resource savings and feasibilities of new approaches for well-known problems.