MetaETL: A Metadata-Intelligent Java Framework for Dynamic and Scalable ETL Workflows

Abstract

MetaETL emerges as a metadata-reflective ETL framework purpose-built to address the architectural stagnation of early enterprise data integration systems. Engineered in Java and governed by IEEE 11179 compliant XML specifications, it decouples execution logic from transformation semantics, enabling declarative, schema agnostic orchestration. Leveraging runtime reflection, the engine dynamically binds transformation rules, adapts to evolving schemas, and supports modular extensibility via plugin interfaces establishing a foundation for adaptive and low-code integration workflows. Benchmarked against Apache NiFi and Talend Open Studio using heterogeneous datasets from healthcare and education sectors, MetaETL demonstrated superior computational performance, achieving 32,000 rows/sec throughput, 1.2 ms latency, and 100% dynamic schema adaptability. A lightweight memory footprint of just 128 MB underscores its optimization for constrained, non-virtualized environments. The architecture aligns with IEEE 1471 modeling principles, embracing semantic modularity, runtime flexibility, and platform independence. By transforming static ETL processes into executable metadata pipelines, MetaETL anticipates core attributes of modern data engineering runtime introspection, self-adaptive control flow, and metadata-centric design. It bridges legacy ETL systems with the emerging landscape of autonomous, cloud-compatible data pipelines, offering a resilient, intelligent, and future-facing alternative for dynamic enterprise ecosystems.