Foundations of Data Science

Section outline

• Select section General

  Collapse Expand

  General

  Collapse all Expand all

  Summary

  We discuss a set of topics that are important for the understanding of modern data science but that are typically not taught in an introductory ML course. In particular we discuss fundamental ideas and techniques that come from probability, information theory as well as signal processing.

  Content

  This class presents basic concepts of Information Theory and Signal Processing and their relevance to emerging problems in Data Science and Machine Learning.

  A tentative list of topics covered is:

  1. Information Measures
  2. Multi-arm Bandits
  3. Detection and Estimation
  4. Distribution Estimation, Property Testing, and Property Estimation
  5. Exponential Families
  6. Signal Representations
  7. Compression and Dimensionality Reduction
  8. Information Measures and Generalization Error

  
  

  Materials

  • Lecture Notes will be downloadable from here very soon!

   

  Additional Material:

  • T. M. Cover and J. A. Thomas, Elements of Information Theory (Click to get access to the full PDF via the EPFL library). New York: Wiley. Second Edition, 2006.
  • T. Lattimore and C. Szepesvari, Bandit Algorithms

  Schedule

  • Tuesdays:
    
    • 11:15-13:00, BC 01 (Lecture)
    • 13:15-15:00, BC 01 (Exercises)
  • Thursdays:
    • 8:15-10:00, BC 01 (Lecture) 

   

  ED Discussion Forum

  • We will use the ED Discussion Forum for this class. Everyone is strongly encouraged to make the most of this!
    • Ask questions!
    • Answer questions!
  • The class staff will check the forum on Monday afternoon and on Thursday afternoon.

   

  SWITCHtube Channel

  We will not make new video recordings this year. You can access the videos from a couple of years ago. The content is largely the same, but the order of the topics is slightly different.

  Grading

  • If you do not hand in your final exam your overall grade will be NA.
  • Otherwise, your grade will be determined based on the following weighted average: 10% for the Homework, 30% for the Midterm Exam, 60% for the Final Exam.
  • The Midterm Exam will take place on Thursday, November 12, 2026, 8:15-10:00

  • The Final Exam is an Oral Exam and will take place on at some point between January 11, 2027 and January 30, 2027.

  • Select activity Discussion Forum

    

    Discussion Forum External tool
  • Select activity Announcements

    

    Announcements Forum
  • Select activity Past Exams

    

    Past Exams Folder
• Select section Week 1 (Basics of Probability; Information Measures)

  Collapse Expand

  Week 1 (Basics of Probability; Information Measures)

  Current week
• Select section Week 2 (Information Measures)

  Collapse Expand

  Week 2 (Information Measures)
• Select section Week 3 (Information Measures; Bandits)

  Collapse Expand

  Week 3 (Information Measures; Bandits)
• Select section Week 4 (Bandits)

  Collapse Expand

  Week 4 (Bandits)
• Select section Week 5 (Bandits; Detection & Estimation)

  Collapse Expand

  Week 5 (Bandits; Detection & Estimation)
• Select section Week 6 (Detection & Estimation)

  Collapse Expand

  Week 6 (Detection & Estimation)
• Select section Fall Break (October 19-25)

  Collapse Expand

  Fall Break (October 19-25)

  No class, No exercise session
• Select section Week 7 (Distribution Estimation)

  Collapse Expand

  Week 7 (Distribution Estimation)
• Select section Week 8 (Distribution Estimation; Property Testing)

  Collapse Expand

  Week 8 (Distribution Estimation; Property Testing)
• Select section Week 9 (Exponential Families)

  Collapse Expand

  Week 9 (Exponential Families)

  Nov 12, 8:15-10:00: Midterm Exam
• Select section Week 10 (Exponential Families)

  Collapse Expand

  Week 10 (Exponential Families)

  Thursday, November 19, 8:15-10:00 : Class exceptionally in a different room, watch out for announcements.
• Select section Week 11 (Signal Representations)

  Collapse Expand

  Week 11 (Signal Representations)
• Select section Week 12 (Signal Representations; Compression)

  Collapse Expand

  Week 12 (Signal Representations; Compression)
• Select section Week 13 (Compression)

  Collapse Expand

  Week 13 (Compression)
• Select section Week 14 (Information Measures and Generalization)

  Collapse Expand

  Week 14 (Information Measures and Generalization)