COL868/8395 Special Topics in Data Management

🕰 Monday and Thursday 3:30 PM -- 5:00 PM
📍 Block III 342

Notice

Overview

Data management systems are being reshaped by cloud architectures, vector search, and large language models. The course studies this transition from disk-based relational systems to modern AI-native data systems, while emphasizing that many classical database ideas including but not limited to indexing, caching, selectivity estimation, predicate pushdown, and cost-based optimization remain central.

The course is organized as a seminar rather than a broad survey. In the first phase, the we will look at key systems ideas along with representative research papers. In the second phase, students will present, critique, and defend recent papers, primarily from recent SIGMOD and VLDB. A programming project will run alongside the lectures.

The course primarily targets graduate students and advanced undergraduate students who are interested in data and AI systems research and modern data infrastructure.

Syllabus

1. Logistics & Course introduction: from disk-based to AI-native data systems
2. Columnar storage and modern data formats
3. Vectorized versus compiled query execution
4. Cloud-native databases and compute-storage disaggregation
5. Lakehouse architectures and open table formats
6. Approximate nearest-neighbour search I: IVF, quantization, and system trade-offs
7. Approximate nearest-neighbour search II: HNSW, DiskANN, and disk-resident indexes
8. Filtered and hybrid search; vectors inside relational database systems
9. Natural-language-to-SQL and semantic query operators
10. LLM serving systems: KV cache management, batching, and SLOs
11. Learned components in data systems and the reproducibility critique
12. Benchmarking and adversarial reading of experimental evaluations

The syllabus is tentative and may change based on the pace of the course.

Course Components

Phase 1: Instructor-led Lectures

• The first part of the semester will cover the evolution of modern database architectures and the systems challenges introduced by vector search and AI workloads.
• Lectures will connect classical database concepts to current systems problems, such as HNSW indexing, hybrid vector search, semantic operators, and LLM serving.

Phase 2: Paper Presentation and Discussion

Each Phase-2 lecture will include some or all of the following roles:

For each Phase-2 lecture, the expected workflow is:

Programming Project

• Students will work in teams of two on one programming project related to modern data systems.
• Evaluation will be based on a working system, demonstration, and oral examination - not only a written report.
• Students should be prepared to explain and defend their design decisions and implementation.

Class Participation

• Participation includes attendance, paper reviews, discussant turns, and engagement during in-class questions and discussions.
• Students may be cold-called to discuss a paper, system design choice, figure, table, or experimental result.

Prerequisites

• COL362, COL632, COL7362, or an equivalent database systems course
• Familiarity with the internals of a relational database management system
• Comfort with reading systems research papers and implementing a non-trivial programming project

Policy on AI Tools

• Understanding papers: Students may use AI tools to help understand papers and background material.
• Project code: AI tools may be used, but students must defend the design and code orally.
• Paper reviews: Reviews must not be AI-generated. Every review must include at least one substantive claim anchored to a specific figure, table, or equation, and must go beyond what is stated in the abstract.
• Exam and live Q&A: No AI tools are permitted.

Course Policies

• Attendance: Attendance will be recorded in every class. Each class carries 0.25 marks, up to a maximum of 5 marks.
• Academic integrity: Cheating in a proctored examination will result in an F grade. Other forms of academic dishonesty may result in strict disciplinary action and referral to the CSE Department’s disciplinary committee.
• Missed evaluations: A re-mid-term examination will be considered only with an IIT Delhi medical certificate.
• Audit policy: B- or higher with non-zero marks in all components.

Recommended Texts/Literature

• Class notes
• Recent research papers from SIGMOD, VLDB, and related systems venues, made available during the course

Grading Scheme

Schedule