All the issues described in this post lead me to create Kestra [0] . Airflow was a true revolution when it was open-source and we need thanks its innovation. But I totally agree that a large static dag is not appropriate in the actual data world with data mesh and domain responsibility.

[0] https://github.com/kestra-io/kestra