I think consolidation in the space has been coming for quite some time now and this merger only confirms what us, along with many others, have been saying: the data tooling is in a miserable state and we had to glue together a bunch of different tools that don't work with each other.
At this point, I think it is quite obvious that Fivetran is going for Snowflake/Databricks's market share. They own the ingestion for many companies already, and they will offer a managed data lake product in order to compete with the data giants. By owning the means of bringing the data in (Fivetran) as well as the transformation layer (dbt/sqlmesh) they will aim to get ahead of Snowflakes of the world.
I think it'll be a win for the data community if they maintain and continue investing into the existing tooling, as they are running in quite a few places already, especially dbt core running in a self-managed way. I certainly hope they won't try to squeeze revenue for the sake of it from their combined users.
It's an interesting time to be in the space, and it feels great to be one of the few independent players in the market.
How will Fivetran and dbt who are detested for being overpriced and underfeatured in the segment they are supposed to be good at (ETL/ELT) be taking on being a datalake? That's orders of magnitude more complex in engineering and operating and they have no experience. This is really a play to consolidate, get rid of duplicate functions and provide a better experience to customers.
This seems to undermine the engineering muscle these companies have. Fivetran is well-capable of building a query engine, and with this merger, they also get access to SDF's query engine. They have the engineering capabilities, as well as the capital to attract the talent where needed.
I would not underestimate any of these players in the space.
With Snowflake's new OpenFlow offering based on Apache Nifi, Snowflake will be able to become Fivetran faster than Fivetran can become Snowflake/Databricks, though....
Fivetran has 2 pieces: data movement and integration/access (via linking to APIs, etc.) Sure, there's Airbyte and others, but no one has as complete a catalog of integration plugs as Fivetran.
Yes, OpenFlow may be able to replace the data movement part (though prefect, airflow, yadda yadda all tried to be the one ring), but it's a pretty small bunch who support all connections.
It's not the only part of the business, but it's an important one. Just like Plaid's approach became the standard to code against in accessing financial services, Fivetran has become the default in getting data out of tools you already pay for directly into a controlled space.
If they don't muck that up, they've got an embed in every large integration. No one is looking to OpenFlow for that.
Still won't become Snowflake or Databricks (and it's silly to try, imho), but they do have a good moat for a small castle.
OpenFlow feels like an attempt at keeping customers within Snowflake boundaries, and while it might work for some, I see no way Snowflake being able to keep up with the data integration needs unless they allow another way of extending their capabilities other than pre-built integrations.
On the other hand, I do agree with you that it is quite a big challenge for Fivetran to try and become Snowflake.
I think consolidation in the space has been coming for quite some time now and this merger only confirms what us, along with many others, have been saying: the data tooling is in a miserable state and we had to glue together a bunch of different tools that don't work with each other.
At this point, I think it is quite obvious that Fivetran is going for Snowflake/Databricks's market share. They own the ingestion for many companies already, and they will offer a managed data lake product in order to compete with the data giants. By owning the means of bringing the data in (Fivetran) as well as the transformation layer (dbt/sqlmesh) they will aim to get ahead of Snowflakes of the world.
I think it'll be a win for the data community if they maintain and continue investing into the existing tooling, as they are running in quite a few places already, especially dbt core running in a self-managed way. I certainly hope they won't try to squeeze revenue for the sake of it from their combined users.
It's an interesting time to be in the space, and it feels great to be one of the few independent players in the market.