Information Theory

This course, Information Theory (Course Code: K35), is a compulsory track course within the Computer Engineering and Telecommunications (CET) track for Bachelor's students. It is typically taken in the 6th semester and contributes 6 ECTS credits. The module delves into the fundamental principles of information theory, covering its historical context, core mathematical concepts, and practical applications. It explores how information is quantified, transmitted, and compressed, laying a theoretical groundwork for advanced studies in computer science and engineering.

Curriculum

The course covers a broad range of topics in information theory, from foundational concepts to advanced applications. It is structured into 15 sections, providing a comprehensive overview of the field.

1. Year 1
2. Year 2
3. Year 3
4. Year 4
5. Further Study
   • C. E. Shannon: His life, work, and influence on modern communications.
   • Information measures and basic properties: Entropy, mutual information, Kullback-Leibler divergence, convexity.
   • Typicality and the asymptotic equipartition property.
   • Stationary (ergodic) sources and entropy rate.
   • Lossless source compression, prefix codes, fundamental compression limits based on entropy rate, Shannon and Huffman codes.
   • Channel capacity examples (binary symmetric channel, erasure channel) and properties, statement and proof of the channel coding theorem for discrete memoryless channels, achievability, joint typicality.
   • Fano's inequality and the converse theorem, feedback capacity.
   • Continuous-time sources and channels, differential entropy, mutual information and properties, entropy of a Gaussian random vector.
   • Additive Gaussian channel and its capacity, typicality, coding theorem, bandlimited channel capacity.
   • Parallel Gaussian channels, the colored noise channel, power allocation for rate maximization, water-filling method.
   • Source coding with stream codes, arithmetic coding, and Lempel-Ziv coding.
   • Introduction to rate-distortion theory and lossy compression.
   • Linear codes, description and encoding, Hamming codes.
   • Introduction to statistical signal processing and the Wald test.
   • Fundamental information-theoretic inequalities and applications to communication networks.

Frequently asked questions

What are the entry requirements for this programme?

This is a compulsory track course within the Bachelor's in Computer Engineering and Telecommunications. Expected prior knowledge includes Probability and Statistics, which is available in Greek.

What is the duration and credit value of this course?

This course is typically taken in the 6th semester and contributes 6 ECTS credits.

What topics are covered in the Information Theory course?

The course covers fundamental principles of information theory including information quantification, transmission, and compression. It delves into topics like entropy, channel capacity, source coding, and rate-distortion theory.

What are the potential career outcomes after completing this track?

This course provides foundational knowledge relevant to specializations within Computer Engineering and Telecommunications, including Data and Knowledge Management, Software, Hardware and Architecture, Communications and Networking, and Signal and Information Processing.

Is there information available on tuition fees and living costs?

Tuition fees and living cost information are not provided for this specific course or program.

How are course materials accessed?

Course materials and lectures are accessible via the university's e-Class platform, along with provided reading lists and supplementary materials.