AI Optimization of Database-y Workloads

BSc Thesis / MSc Thesis.

The goal of this project is to use AI coding tools such as ChatGPT Codex and Claude Code together with optimization tools such as OpenEvolve to find innovative solutions for various database-y workloads, including:

• SIEM workloads, such as finding IP addresses in large security data sets
• String search workloads, such as finding substrings or evaluating regular expressions in large bodies of text
• Data analysis workloads, such as executing SQL queries over application logs

The core idea is to find better statistics to summarize data partitions. Currently, min/max indexes are the de-facto standard for summarizing columns within partitions. Min/max indexes help skip data partitions when compiling and running queries, thus speeding up query execution over large data sets. The hypothesis in this project is that AI can find targeted statistics for specific data sets and workloads that are less generic but more efficient than min/max indexes. The goal is to experimentally validate or debunk this hypothesis and to show, for a given data set and workload, whether customized, AI-based statistics make sense.

For background on min/max indexes and partition skipping, see Lecture 9 of the Spring 2026 Introduction to Database Systems course (recording).

Multiple groups, different workloads. I aim to have multiple groups working on this project at the same time, with each group targeting a different workload from the list above. I expect around 3–4 groups of 1–3 students each. Every group picks one workload and carries out the full project for that workload. We will hold individual supervision meetings per group, as well as larger meetings among all groups where each group presents its progress. The exact schedule and the assignment of workloads to groups will be discussed in the first week of the semester.

Work includes:

• Understanding your workload domain, and finding and replaying representative workloads
• Understanding AI coding and optimization toolkits, and replaying them locally and on the HPC cluster
• Building partitioned data sets for your chosen workload, where partitions are summarized by statistics
• Using AI coding and optimization tools to generate statistics against these data sets
• Measuring effects and understanding the trade-offs
• Giving presentations throughout the semester on your progress, and writing up a final thesis based on your findings, learnings, and results

This project is open to bachelor and master students with a background in databases, programming, and experimentation. The build stack can be Java, Python, or C++ (in that order of preference). While much of the code will be produced by AI tools such as Codex, Claude, and OpenEvolve, you should be able to understand and reason about the code these tools generate. It would be best if you have taken the Introduction to Database Systems course given by Omar Shahbaz Khan and me before.