A Metadata Management System for Apache Wayang

BSc / MSc Thesis.

This project focuses on designing and implementing a metadata storage system for Apache Wayang. Currently, Wayang does not provide a dedicated component for managing metadata beyond what is handled implicitly by individual subsystems. The goal of this project is to add such a component and enable Wayang to store, retrieve, and manage its own metadata in a structured and extensible way.

The proposed metadata store should support different types of metadata, such as table metadata, schema metadata, function metadata, or access-control–related information. Rather than relying on a fixed set of predefined object types, the system should allow metadata types to be defined dynamically and then store metadata objects that conform to these types. This makes the approach flexible and adaptable to different use cases and future extensions.

A metadata component of this kind would allow Wayang to maintain internal state that can be used by other Wayang components. Examples include exposing metadata through the Wayang UI or incorporating metadata into optimization and decision-making in the Wayang optimizer. It also opens up opportunities for better integration with external systems that Wayang interacts with.

The project combines system building with research questions. Topics include analyzing the feasibility of such a metadata store in the context of Apache Wayang, evaluating its performance, and studying its compatibility and interoperability with other systems in the Wayang ecosystem.