model parallelism

CellStrat > Research/Blog > model parallelism
10
May
GPU Architectures and GPGPU Computing

#CellStratAILab #disrupt4.0 #WeCreateAISuperstars #AlwaysUpskilling Last Friday, our AI Lab Researcher Darshan C G presented a wonderful overview of “GPU Architectures and General-Purpose GPU Computing“. Parallelism :- Performance Improvements are often based on parallelism techniques, which are found everywhere :- Pipelining, Instruction-Level Parallelism. Vector Processing. Array processors/MPP. Multiprocessor Systems. Multicomputers/cluster computing. Multicores. Graphics Processing Units (GPUs) […]

Posted in: AI Hardware,
Tags: Compute Unified Device Architecture, Fermi GTX, Fermi SM, GPU, GPU Architectures, Jetson, model parallelism, Multi-threading, Nvidia, Nvidia Volta, OpenCL, Pascal, SIMT, Tesla GPU,
30
Apr
Compiler Level Optimizations for Accelerated Deep Learning

#CellStratAILab #disrupt4.0 #WeCreateAISuperstars #AlwaysUpskilling Last Saturday (25th Apr ’20), our AI Lab Researcher Darshan G. presented a fabulous hands-on workshop on Tensor Fusion, Accelerated Linear Algebra (XLA) and many other techniques to speed up Deep Learning computations. The following discussion covers many of these techniques. TensorFlow Graph Concepts :- TensorFlow (v1.x) programs generate a DataFlow […]

Posted in: AI Hardware, Deep Learning,
Tags: accelerated linear algebra, Compiler Optimization, computation graph, data parallelism, DNN Pruning, GEMM, General Matrix Multiplication, grappler, High Level Optimizer, HLO, HLO IR, Intermediate Representation, kernel fusion, kernel optimization, Memory Access Coalescing, MLIR, model parallelism, Multi-Level Intermediate Representation, Op Fusion, Remapper, Sparse Matrix Vector Multiplication, SpMV, Tensor Fusion, tensorflow, XLA,