5194

<HEAD>
<TITLE>5194</TITLE>
</HEAD>

<P>

<center>

<h2> Network-Based Computing for HPC, Big Data, Deep Learning, and Cloud<br>
<p>

<h3> Instructor: Prof. Dhabaleswar K. (DK) Panda<br>
<!--
      Spring 2018 <br>
-->
</h3>
</h2>
</center>

<!--
<h4> Course Number: 5194.01 and 5194.02</h4> 

<p>
<h4> Class Number: 35363 and 35364 </h4> 
-->

<p>

<h4> Credits: 3 </h4> 

<!--
<p>
<h4> Course Time: TR 11:10 am - 12:30 pm </h4> 

<p>

<h4> Classroom: BO 313 </h4> 
-->

<p>

<h4> Course Description: </h4> 

The objective of this course is to understand the principles and the
practice of the emerging network-based computing paradigm.  We will
study advances in networking, computing, accelerator, and storage
technologies as well as the computational/networking demands of
current and emerging applications. Brief overview of the emerging
network-based computing applications will be covered. System
architectures for different kinds of network-based computing systems
(HPC, Big Data, Deep Learning, and Cloud) will be analyzed.  The role
of computing technologies (multi-core/many-core), networking
technologies (InfiniBand, High-Speed Ethernet, Omni-Path,
proprietary), accelerators (GPGPUs and FPGAs), storage technologies
(SSDs, NVMe-based SSDs, Burst Buffers), programming models and
runtimes (MPI, PGAS, Hybrid MPI+PGAS, Hadoop MapReduce, Spark, Caffe,
CNTK, and TensorFlow), storage/file systems (parallel file systems,
HDFS, and Ceph), fault-tolerance, virtualization and
power-/energy-aware issues in designing next-generation exascale
systems will be focussed.  Limitations of current solutions, impact of
next generation technologies on designing future exascale computing
systems and applications and future research challenges will be
discussed.

<p>

<h4> Topics to be Covered (Tentative) </h4> 

<ul>
<li> Overview of Network-Based Computing 
<li> Trends in Designing Systems for HPC, Big Data, Deep Learning, and Cloud 
<li> Computing Technologies and Trends 
     <ul>
     <li> Multi-Core (x86, ARM, OpenPower), GPGPUs, and FPGAs 
     </ul>
<li> Networking Technologies and Trends 
     <ul>
     <li> InfiniBand, Ethernet, Omni-Path, and Proprietary 
     </ul>
<li> Storage Technologies and Trends 
     <ul>
     <li> SSDs, NVMe-based SSDs, and Burst Buffers 
     </ul>
<li> User-Level Communication Protocols and Benefits 
<li> Programming Models and Environments 
     <ul>
     <li> MPI, Shared Memory and Distributed Shared Memory, 
     <li> Partitioned Global Address Space (PGAS) - UPC, OpenSHMEM, and CAF  
     <li> Hadoop MapReduce, HBase, and Spark
     <li> Caffe, CNTK, and Tensorflow 
     </ul>
<li> High-Performance and Scalable Implementation of Programming Models 
     and Environments
     <ul>
     <li> RDMA-based networking 
     <li> Multi-core
     <li> GPGPUs and FPGAs 
     </ul>
<li> Collective Communication Optimizations 
     <ul>
     <li> Shared-Memory-aware, Offload-based, Topology-aware and Power-aware 
     </ul>
<li> Scalable I/O and File Systems 
<li> Designing High-Performance and Scalable Data Centers 
     <ul>
     <li> Multi-tier Data Centers
     <li> Memcached Designs (Memory-based and Hybrid Memory+SSD-based) 
     </ul>
<li> Desing High-Performance and Scalable Big Data Systems 
     <ul>
     <li> HDFS, MapReduce, RPC, HBase, and Spark
     <li> RDMA-based Networking and Multi-core
     </ul>
<li> Desing High-Performance and Scalable Deep Learning Systems 
     <ul>
     <li> Caffe, CNTK, and Tensorflow 
     <li> RDMA-based Networking and Multi-core
     </ul>
<li> Designing High-Peformance Cloud Environment
     <ul>
     <li> Virtualization including SR-IOV 
     <li> VMM-Bypass
     <li> WAN Technology and RDMA
     </ul>
</ul>

<p>

The objective of this course is to understand the principles and
practice of the emerging network-based computing paradigm.  We will
study advances in networking and computing technologies as well as the
computational/networking demands of current and emerging
applications. Brief overview of the emerging network-based
computing applications will be covered. System architectures
for different kinds of
network-based computing systems will be analyzed. Computational,
communication, networking, I/O, and QoS requirements of emerging
network-based computing applications together with emerging 
trends (such as GPUs/accelerators, Partitioned Global Address 
Systems (PGAS), Hadoop/MapReduce/HBase/Spark, Deep Learning, 
and virtualization) will be analyzed.
Challenges and research issues in designing
network-based computing systems and applications will be discussed in
detail.  
Limitations of current solutions, impact of next generation
networking technologies
on designing future network-based computing    
systems and applications, and future research challenges will be
discussed.     

<!--
<h4> Text: </h4>  
Selected papers from the literature including papers focusing on 
past and on-going 
research activities in the group. 
Please refer to the attached
<a href="reading.html">reading list </a> and 
<a href="presentation_schedule">presentation schedule.        
</a>

<h4> Grading Plan: </h4>  
There are five components: 
<pre> <strong> 
- Attendance, participation in class discussion,   (10%)
  and evaluation of class presentations
- Class presentation (two, 12.5% each)             (25%)                      
- Final exam                                       (30%)
  (comprehensive, open book) 
- Class project                                    (35%)
</strong>
</pre>    

<h4> Class Presentation: </h4>  

Each student will make two class presentations.
Please refer to the <a href="reading.html">reading list </a> 
and <a href="presentation_schedule">presentation schedule </a>
and select 
two dates (with the corresponding set of papers) for presentation. 
If you are working on a given project, it will be preferable  
to choose a different set of papers for the presentations. 
Similarly, you can select papers from two different topics. This will 
allow you to learn different things in-depth.  

<p>

The presentation schedule will be filled-up based on the order
in which I receive the preferences. To avoid conflicts with others, 
you can give me preferences
for 3-4 dates. 

<p>

The class presentation will be evaluated based on preparation
(knowledge assimilation), presentation style (organization,
smoothness, and clarity), finishing the presentation in time, and
answering questions during the discussion.

<p>

We will be using an   
<a href="evaluation_form ">evaluation form </a> to evaluate every 
student presentation. Each student needs to fill out this form for 
every student speaker 
and return it (hardcopy only) to me by the following class. I will 
give them  
back to the student after removing the evaluator's information. 
I will use this information to evaluate the speaker's presentation 
as well as the evaluators' skill to evaluate a presentation.  

<p> 

You must consult me when preparing your slides and finalizing them.
This is to ensure that the presentations are compact and provide a
smooth flow. Please discuss about your presentation plans with me
two weeks before the presentation date and the slides with me 
one week before
the presentation date.               

<h4> Class Project: </h4>  

Due to the research-oriented nature of this course, the project will
play an important part in the learning experience and in the grading
process. Projects will be evaluated based on their 
technical quality, originality, depth of analysis, and completeness.

<p>

Projects will be mostly done in an individual manner.  Maximum two
people may be in a group if the scope of the project is big and there
is sufficient understanding between the members that both will
contribute equally to the success of the project (members will get the
same grade in project).  The project will be
research-oriented. Depending on the nature of the topic, it may
consist of one or more of the following components: theory, design,
analysis, simulation, or experimental results.            

<p>

You are free to consult with me while defining the project and working 
on it during 
the quarter. I will provide a list of possible projects during the 
second week of classes. 
Those who have been already involved in my 6429 in earlier semesters 
can carry out their earlier projects.  The new students can meet 
with me to discuss their research interests and we will define 
project topics.    

<p>

I will be meeting with each of you frequently during the semster 
to discuss the progress
of your project. 

<h4> Project Schedule: </h4>  
<pre> <strong> 
End of 3rd/4th week     - discussion with the instructor to select/focus topic
End of 4th/5th week     - proposal (around two pages) due
End of 5th week         - feedback from instructor and finalizing the topic
Project Report due      - final exam week (on or before 05/01/18)          
</strong>
</pre>  

<p>
-->
<hr>

<ADDRESS>
Last Updated: August 5, 2018 
</ADDRESS>