Section outline

  • 09:00-16:00 Data Parallel Neural Networks

    Location: Paderborn Center for Parallel Computing (PC2), Room E5.333

    Lecturers: Dr. Charlotte Debus (KIT), Dr. Marie Weiel (KIT), David Li (KIT)

    Materials https://github.com/mcw92/nhr-summerschool

    Agenda

    • 09:00 - 10:15 - Introduction to Neural Networks
      • Backpropagation and Stochastic Gradient Descent
      • Layer Architectures
      • Training a Neural Network
    • 10:30 - 12:00 - Hands-on Session
      •  Neural Networks with PyTorch
    • 12:00 - 13:00 -  Lunch break
    • 13:00 - 14:15 - Data-parallel Neural Networks
      • Parallelisation Strategies for Neural Networks
      • Distributed SGD
      • IID and Large Minibatch Effects
    • 14:30 - 16:00 - Hands-on Session
      • PyTorch Distributed Dataparallel
    • 16.00 Walk into the City to the LWL Museum


    Prerequisites / Preparation: 

    You should be familiar with basic Python and Jupyter. A basic understanding of machine-learning techniques and gradient-based optimisation is beneficial. Furthermore you should have basic knowledge on distributed computing and communication via MPI.


    From 17:00 Archäologische Spurensuche

    Start and End: LWL Museum Kaiserpfalz, Paderborn (View on map)

    Language: English

    After that dinner together at Kitzgams, Paderborn (View on map)