Machine learning group is focus on the acceleration of machine learning algorithms. Recently, rapid growth of modern applications based on machine learning algorithms has further improved research and implementation. As the problem/model gets more complicated and the data explodes, the need for high-performance machine learning algorithms is more and more urgent.
Accelerating Algorithms for Probabilistic Graphical Model
Probabilistic Graphical Model (PGM) is a widely used mathematical tool based on both probability theory and graph theory. The well known models, such as Bayes Network, Markov Random Field (MRF) and Hidden Markov Model (HMM), have been playing important roles in many research field including computer vision, data mining and machine learning.
Algorithms for learning and inference of large scale PGMs requires large amount of computational resources. Therefore, seeking high efficiency solutions for PGM learning and inference algorithms are one of the important branch in PGM researches.
Our work takes good advantage of the heterogeneous platforms, such as GPUs MICs and FPGAs, to accelerate PGM algorithms. One of our researches focusing on FPGA based MRF inference algorithm has achieved good results and published on FPL 2014. We propose a parallel algorithm based on the existing MRF inference algorithms and design a fully-pipelined kernel based on FPGA, which is shown in Fig 1. We build a hybrid CPU/FPGA system (Fig 2), and use this kernel to accelerate stereo matching, one of the computer vision problems. Experimental results show that our design provides significant improvement on performance with less resource consumption compared with previous designs.
Besides, we are still working on accelerating learning algorithms for PGMs, which are more complicated and time consuming.
Fig 1. FPGA design for TRW algorithms
Fig 2. Hybrid system for Stereo Matching
Parameter Optimization Based on EnKF
As the rapid development of information technology, the collection and storage of the data has become larger and larger, and been involved in almost every filed in the science research and business. How to effectively deal with the big data and dig out the valuable information becomes a urgent issue. Machine learning, an interdiscipline referring to probability theory, statistics and optimization, is widely used into application as an effective method. The parameters in the machine learning algorithm will badly affect the performance of the predicting results, and it is necessary to optimize the parameters.
Parameter optimization usually comprises the general non-optimized methods, the numerical methods and nonnumerical methods. The general non-optimized methods are easy to implement and highly parallel, but with bad scalability and low efficiency; numerical methods have the explicit search direction and can converge fast, but are easy to get stuck in the local optima; nonnumerical methods usually reach the approximate global optima but with a low efficiency and need plenty of parameter sample. Is there a trade-off to achieve a higher performance? Is there some way to combine these method and form a method with high efficiency and high accuracy? Some attempts even including borrowing some thoughts from other fields, such as geophysics and weather forecast, are encouraged. The work is still undergoing, and some details are not shown here, but will be introduced after some publications.
Wenlai Zhao, 5th year PhD. student, Tsinghua University
Weijie Zheng, 3nd year PhD. student, Tsinghua University
Jiahe Liu, 2st year Master student, Tsinghua University
Zihong Lv, Master, Tsinghua University
Yang You, Master, Tsinghua University
Yushu Chen, Ph.D., Tsinghua University