Transcript Mars: A MapReduce Framework on Graphics Processors

Weekly Report
Ph.D. Student: Leo Lee
date:
Oct. 9, 2009
Outline
• Courses
• Research
• Work plan
Outline
• Courses
• Research
• Work plan
Courses
• Data mining
– Homework;
– Hidden Markov Model
– Read the most classical tutorial;
• Forward-backward procedure;
• Viterbi algorithm;
Courses
• Network security
– Check the homework;
– Modify the tutorial for next week;
• Learn C# ;
• Dev. an easy chat application.
Outline
• Courses
• Research
• Work plan
Research
Mars: A MapReduce Framework
on Graphics Processors
• Introduction
– For search engines and other web server
applications, high performance is essential.
– The MapReduce framework is a successful paradigm
to support such data processing applications, which
reduces the complexity of parallel programming.
– Encouraged by the success of the CPU-based
MapReduce frameworks, we develop Mars, a
MapReduce framework on graphics processors, or
GPUs.
Mars: A MapReduce Framework
on Graphics Processors
• Introduction
– Since GPUs are traditionally designed as special-purpose coprocessors for gaming applications, their languages lack support
for some basic programming constructs.
• variable-length data types;
• more complex functions such as recursion.
– GPU architectural details are highly vendor-specific and
programmers have limited access to these details.
– All these factors make the GPU programming a difficult task in
general and more so for complex tasks such as web data
analysis. Therefore, we propose to develop a MapReduce
framework on the GPU so that programmers can easily harness
the GPU computation power for their data processing tasks.
Mars: A MapReduce Framework
on Graphics Processors
• Introduction
– First, the synchronization overhead must be low so
that the system can scale to hundreds of processors.
– Second, due to the lack of dynamic thread scheduling
on current GPUs, it is essential to allocate work
evenly across threads on the GPU to exploit its
massive thread parallelism.
– Third, the core tasks of MapReduce programs,
including string processing, file manipulation and
concurrent reads and writes, are unconventional to
GPUs and must be handled efficiently.
Mars: A MapReduce Framework
on Graphics Processors
• Preliminaries and overview
– GPUs
– GPGPU
– MapReduce
Mars: A MapReduce Framework
on Graphics Processors
• Design and implementation
– Ease of programming. Ease of programming
encourages developers to use the GPU for
their tasks.
– Performance. The overall performance of our
GPU-based MapReduce should be
comparable to or better than that of the stateof-the-art CPU counterparts.
Mars: A MapReduce Framework
on Graphics Processors
• Design and implementation-APIs
– User-implemented
Mars: A MapReduce Framework
on Graphics Processors
• Design and implementation-APIs
– System-provided
Mars: A MapReduce Framework
on Graphics Processors
• System Workflow and Configuration
Mars: A MapReduce Framework
on Graphics Processors
• Optimization Techniques
– Coalesced accesses
– Accesses using built-in vector types: char4 and int4?
Mars: A MapReduce Framework
on Graphics Processors
• Experimental evaluation
Mars: A MapReduce Framework
on Graphics Processors
• Experimental evaluation
Mars: A MapReduce Framework
on Graphics Processors
• Experimental evaluation
Mars: A MapReduce Framework
on Graphics Processors
• Experimental evaluation
Mars: A MapReduce Framework
on Graphics Processors
• Experimental evaluation
Mars: A MapReduce Framework
on Graphics Processors
• Experimental evaluation
Mars: A MapReduce Framework
on Graphics Processors
• Experimental evaluation
Mars: A MapReduce Framework
on Graphics Processors
• Experimental evaluation
Mars: A MapReduce Framework
on Graphics Processors
Outline
• Courses
• Research
• Work plan
Work Plan
• Go on paper reading
• Learn more CUDA applications
• Work hard on data mining, try to
implement some classical algorithm
• Learn C#
• Thanks for your listening