Welcome to DAT255: Deep learning engineering

Deep learning is a sub-field of machine learning and is what both launched and propels the recent surge of interest in artificial intelligence. The course will focus on deep learning and its applications in computer vision, natural language processing, and recommendation systems. The methods, tools, and techniques covered by the course are widely applicable, and the course aims to be instructive to anyone wanting to apply deep learning to any task.

In addition to getting a solid understanding of deep learning, the course will provide you with hands-on experience with the design and deployment of deep learning solutions for practical, real-life problems using state-of-the-art techniques and software frameworks from machine learning, machine learning engineering, and deep learning. You will experience first-hand how deep learning engineering relates to the broader discipline of software engineering.

Go to Welcome to DAT255! to get started.

Practical information:

Course description: https://www.hvl.no/en/studies-at-hvl/study-programmes/courses/38/dat255

Course responsible: Alexander S. Lundervold

Discord server: https://discord.gg/ztnUnx9MKE

Code repository: https://github.com/alu042/DAT255-2022

Textbook

A major part of the course will be based on material from the textbook by Jeremy Howard and Sylvain Gugger, chapters 1-12. It isn't necessary to buy the textbook as all the material is freely available online. However, you may find it helpful to have the content available in book format:

J. Howard and S. Gugger, Deep Learning for Coders with fastai and PyTorch, 2020.

You can buy it from Akademika or most other online bookstores. The entire book is also freely available as Jupyter Notebooks in the book repo: https://github.com/fastai/fastbook.

Module 9: From prototype to production will not follow the fastai textbook. See the module description for a list of resources.

Additional resources are mentioned in the various course modules.

Semester plan

The DAT255 weekly schedule

The semester plan will be updated throughout the semester:

Week	Topic	Details and comments
Week 2 Jan 10.-16.	About the course and getting started Module 1: Getting started with deep learning	No lectures this week
Week 3 Jan 17.-23.	Module 1: Getting started with deep learning	Tuesday (Zoom): Getting started on DAT255 and deep learning. About the content, structure, philosophy, and objectives of the course. Slides; Recording Wednesday (Zoom): Slides; Recording. About the course projects and mini-projects in DAT255. Demo of a mini-project presentation. Select the next mini-project group.
Week 4 Jan 24.-30.	Module 2: From model to production, Part 1	Tuesday (Zoom & Discord): Mini-project guidance + Q&A Wednesday (Zoom): Slides Mini-project presentation and mini-project selection. More about the course projects.
Week 5 Jan 31.-Feb 6.	Module 2: From model to production, Part 1 Module 3: Deep learning under the hood	No lectures on Tuesday and Wednesday this week. Continue working on Module 2 and Module 3. During this week, you must decide on a course project and make a tentative project plan.
Week 6 Feb 7.-13.	Module 3: Deep learning under the hood	Tuesday (Zoom & Discord): Mini-project guidance + Q&A Wednesday (Zoom): Slides Mini-project presentation and mini-project selection. Project meetings. Deadline DAT255: Project meetings
Week 7 Feb 15.-20.	Module 3: Deep learning under the hood	Tuesday (Zoom & Discord): Mini-project guidance + Q&A Wednesday (physical @ HVL): Mini-project presentation and mini-project selection. Slides. Lecture: Digging deeper into the building blocks of neural networks (using PyTorch). Slides; Source code.
Week 8 Feb 21.-27.	Module 3: Deep learning under the hood Module 4: Data ethics Module 5: Image classification	Tuesday (Zoom & Discord): Mini-project guidance + Q&A Wednesday (@ HVL): Mini-project presentation and mini-project selection. Lecture: Continuing our exploration of the building blocks of neural networks. A quick linear algebra refresher. Tensors, tensor operations, and deep learning from a geometric point of view. Slides ; Source code.
Week 9 Feb 28.-Mar 6.	Module 5: Image classification Module 6: Collaborative filtering	Tuesday (Zoom & Discord): Mini-project guidance + Q&A Wednesday (@ HVL): Mini-project presentation and mini-project selection. Guest lecture by Sathiesh: he will talk about his fastai extension `fastMONAI` and applications of deep learning in medical image processing. Slides.
Week 10 Mar 7.-13.	Module 6: Collaborative filtering	Tuesday (Zoom & Discord): Mini-project guidance + Q&A Wednesday (Zoom): Mini-project selection. Lecture: Data representations and extracting representations from deep neural networks. Slides; Source code
Week 11 Mar 14.-20.	Module 7: Tabular data Module 8: Natural language processing Module 9: From prototype to production	Tuesday (Zoom & Discord): Mini-project guidance + Q&A Wednesday (@ HVL): Mini-project presentation and mini-project selection. Lecture: ML engineering and a quick look at other useful deep learning libraries. Slides. Lecture: A crash course in TensorFlow and Keras, Part 1. Source code.
Week 12 Mar 21.-27.	Module 8: Natural language processing Module 9: From prototype to production	Tuesday (Zoom & Discord): Mini-project guidance + Q&A Wednesday (@ HVL): Mini-project presentation and mini-project selection. Lecture: A crash course in TensorFlow and Keras, Part 2. Slides; Source code.
Week 13 Mar 28.- Apr 3.	No lecture. MSc seminar in Voss
Week 14 Apr 4.-10.	Module 9: From prototype to production	Tuesday (@ HVL): Lecture: ML pipelines using TensorFlow Extended. Slides; Source code; Recording Wednesday (@ HVL) at 12.15 – 14.00 Mini-project presentation Lecture: ML pipelines using TensorFlow Extended. Slides; Source code; Recording Some details about the exam. Slides; Recording
Week 15 Apr. 11.-17.	Easter break
Week 16 Apr. 18.-24.	Module 9: From prototype to production	Tuesday: No lecture Wednesday (@ HVL): Lecture: ML pipelines using TensorFlow Extended. Preprocessing and modeling. Slides; Source code;
Week 17 Apr. 25.- May 1.	Module 9: From prototype to production	Deadline DAT255 course project Tuesday (@ HVL): Lecture: ML pipelines using TensorFlow Extended. Model analysis and model deployment. Summary. Slides; Source code.
Week 18 May 2.-8.
Week 19 May 9.-15.
Week 20 May 16.-22.	Oral exam Friday, May 20th	Some example questions can be found here