Cache coherence and memory consistency are of the most decisive and challenging issues in the design of shared-memory multi-core systems that influence both the correctness and performance of parallel programs. In this thesis, we identify and analyze the problem of designing a coherent/consistent memory subsystem in general and then focus on FPGA-based multi-core embedded systems containing general purpose CPUs and dedicated hardware accelerators. We narrow down the range of the problem by targeting only the stream-based applications and developing dedicated application-specific solutions. A flexible Windowed-FIFO communication pattern is proposed to be used by the parallel programs being run on the multi-core system. The software APIs for the FPGA platform are implemented and tested, a customized streaming cache memory is designed, implemented and tested based on the proposed communication pattern and in the end, example embedded systems are developed and tested on the FPGA platform to prove the correct functionality of the APIs, the cache memory and the coherent data communication between the cores. All the tests are done on a Xilinx Spartan3dsp development board and all the hardware and software aspects of the FPGA platform are studied and their influence on the memory system is analyzed. The simulations and analyses show that the developed solution has less complexity and more scalability and portability comparing to existing solutions while it provides a flexible range of functionality that different streaming parallel applications can benefit from.