Software and Hardware Methods for Memory Performance Improvement

With the rapid development of VLSI technology, the speed of processors has increased dramatically in the past decade. Nevertheless, the speed of memories has been increased at a much slower pace. As long as the DRAM technology is the most cost-effective choice of high density for the main memory, we will continue to see an increasing gap in speed between the processor and the memory. Regarding computer applications, the workloads are becoming more and more data-intensive and memory demanding. Therefore, the memory access latency of data-intensive applications on a computer system is a major performance bottleneck.

We are conducting several research projects to improve memory performance. Our objectives are to understand the memory access patterns of different application types, and to exploit data locality at different levels of the memory hierarchy. We are developing, evaluating, and implementing software/hardware methods from different system aspects: application programming, runtime software support, and memory architecture designs.

Technology Transers and Impact based on Our Research

• The permutation technique we presented/published in Micro-33, has been adopted in the Sun UltraSPARC IIIi processor.
  • A news report about this technology transfer.
  • An acknowledgement letter from the Sun MicroSystems Inc..
  • The permutation technique has also been used in the Sun's first dual-core processor Gemini, and is reported in this IEEE Micro paper.

• The cache optimization methods published in the ACM Journal of Experimental Algorithmics have become a open source released by the ACM Digital Library for a public usage.
  • Sorting source code with cache optimization.

Processor Cache Designs

We have proposed and evaluated different cache designs to increase cache hit rates, and to save power consumption.

Representative Publications:

• X. Ding, D. Nikolopoulos, S. Jiang, and X. Zhang, ``MESA: reducing cache conflicts by increasing static and run-time methods", Proceedings of International Symposium on Performance Analysis of Systems and Software (ISPASS-2006), Austin, Texas, March 19-21, 2006.

• Z. Zhang, Z. Zhu, and X. Zhang, ``Design and optimization of large size and low overhead off-chip caches", IEEE Transactions on Computers, Vol. 53, No. 6, 2004, pp. 843-855.

• Z. Zhu, and X. Zhang, `Look-ahead architecture adaptation to reduce processor power consumption", IEEE Micro, Vol. 25, No. 4, 2005, pp. 10-19.

• Z. Zhu, and X. Zhang, ``Access-mode predictions for low-power cache design", IEEE Micro, Vol. 22, No. 2, 2002, pp. 58-71.

• C. Zhang, X. Zhang, and Y. Yan, ``Two fast and high-associativity cache schemes", IEEE Micro, Vol. 17, No. 5, 1997, pp. 40-49.

Memory Architecture Designs

We have proposed and evaluated different architecture designs to reduce cache and memory access latencies.

Representative Publications:

• Z. Zhu, Z. Zhang, and X. Zhang, ``Fine-grain priority scheduling on multi-channel memory systems", Proceedings of the 8th International Symposium on High Performance Computer Architecture, (HPCA-8), Cambridge, Massachusetts, February 2-6, 2002, pp. 106-117.

• Z. Zhang, Z. Zhu, and X. Zhang, ``Cached DRAM for ILP processor access latency reduction", IEEE Micro, Vol. 21, No. 4, 2001, pp. 22-32.

• Z. Zhang, , Z. Zhu, and X. Zhang, ``A permutation-based page interleaving scheme to reduce row-buffer conflicts and exploit data locality", Proceedings of the 33rd Annual International Symposium on Microarchitecture, (Micro-33), Monterey, California, December 10-13, 2000, pp. 32-41.

Runtime Software Support to Exploit Cache Locality

Exploiting cache locality of parallel programs at runtime is a complementary approach to a compiler optimization. This is particularly important for those applications with dynamic memory access patterns. We have developed and implemented a memory-layout oriented method to exploit cache locality of parallel loops at runtime on Symmetric Multi-Processor (SMP) systems.

Representative Publications:

• Y. Yan, X. Zhang, and Z. Zhang, ``Cacheminer: a runtime approach to exploit cache locality on SMP", IEEE Transactions on Parallel and Distributed Systems, Vol. 11, No. 4, 2000, pp. 357-374.

• Y. Yan, X. Zhang, and Z. Zhang, ``A memory-layout oriented run-time technique for locality optimization", Proceedings of 1998 International Conference of Parallel Processing, (ICPP'98), Minneapolis, Minnesota, August, 1998, pp. 189-196.

Cache-effective Algorithm Designs and Implementations

Some commonly-used algorithms, such as sorting, matrix multiplication, matrix transpose, and bit-reversals, are highly sensitive to memory performance. Since these algorithms are often repeatedly used as fundamental subroutines for scientific and commercial applications, memory-effective methods and their architecture-dependent implementations are useful program libraries for high performance programming at the user level.

Representative Publications:

• Z. Zhang and X. Zhang, ``Fast bit-reversals on uniprocessors and multiprocessors", SIAM Journal on Scientific Computing, Vol. 22, No. 6, 2001, pp. 2113-2134.

• L. Xiao, X. Zhang, and S. A. Kubricht, ``Improving memory performance of sorting algorithms", ACM Journal on Experimental Algorithmics, Vol. 5, 2000, pp 1-23.

• Z. Zhang and X. Zhang, ``Cache-optimal methods for bit-reversals", Proceedings of Supercomputing'99, (SC'99), Portland, Oregon, November 1999.