Meltano goes beyond the ELT to become an “open source data mining system”

Meltano, the data integration company spun off from GitLab last year, today unveiled version 2.0 of its platform. And to fuel its transition to what it calls an “open-source data mining system,” the company also announced $8.2 million in new seed funding.

Debuting internally at GitLab in 2018, Meltano has evolved into what is known as an “extract, load, transform” (ELT) platform that helps companies “extract” raw data from disparate silos (e.g. , several SaaS applications); “load” such data into a warehouse or file storage system; and “transform” the data into a standardized format that is easier to analyze.

Meltano occupies a broader data integration space that includes countless incumbents, from proprietary players like Fivetran to open source newbies like Airbyte. Meltano, on the other hand, resides squarely in the open source realm and promises data engineers flexibility and extensibility – it can be hosted wherever they see fit and accessed through their own orchestration tools or through their own web interface. Meltano.

Tied with tape

With Meltano 2.0, which is officially available today, Meltano says it’s ending the era of “saved data platforms,” ​​providing a comprehensive product teams can use to build and manage their data stack. “ideal”.

This means that Meltano goes beyond the ELT, bringing together tools from across the data stack – from integration to business intelligence (BI) and analytics.

“This allows not only a team’s data pipelines to be managed as a software project, but also their entire end-to-end data platform, from EL (extract and load) to BI” , Douwe Maan, CEO of Meltano, told VentureBeat.

While the Meltano platform is an open-source product in its own right – released under a permissive MIT license – it actually takes a modular approach, drawing on a host of third-party open-source tools such as dbt, a command line for data transformation; Apache Airflow for orchestration; and Singer, which is an “open source standard” for writing data integration scripts. And earlier this year, Meltano introduced support for the Great Expectations open data quality standard.

The company had always planned to support Apache Superset for data visualization, and with version 2.0 that plan came to fruition, with users now able to install, configure, integrate, test, and deploy Superset directly through Meltano. Additionally, Meltano also supports alternative data visualization and BI tools such as Lightdash and Evidence.dev.

“It makes Meltano the MVP [minimal viable product] of our vision for the dataops operating system, and the first major step in solving the complexity and fragility of today’s modern data stack, by providing much-needed infrastructure to hold it all together, and the best software development practices to enable rapid iteration with confidence,” Maan continued.

Elsewhere, Meltano has now expanded the reach of its plugin hub, allowing users to find not just supported EL connectors, but all the open source tools to power their data stack.

“We intend for this to become a library of all open source data tools in the ecosystem, with Meltano being the easiest way to install and use them together,” Maan said.

Finally, Meltano removed a bunch of old features, such as its legacy BI/Visualization feature that it designed in-house, which now makes way for the aforementioned open-source third-party incarnations.

From GitLab to GitHub

The release of version 2.0 also follows shortly after Meltano migrated from its original location on GitLab to GitHub, which it said was to support its continued growth in the open source fraternity. Simply put, GitHub has a broader reach in terms of community-driven data tools projects.

“GitLab is a great tool for collaboration within companies, especially those that are entirely remote, and it’s obviously been close to our hearts since we were born into it,” Maan said. “But as our growth accelerates and the community becomes more diverse, the move to GitHub makes perfect sense.”

When officially released from GitLab last year, Meltano revealed a $4.2 million funding round led by Alphabet’s VC arm GV, along with a slew of angel investments from the founder. of WordPress Matt Mullenweg and first Google backer Ram Shriram. Meltano has now extended that round to $12.4 million in total, with Venrock leading the additional raise and GV, Uncorrelated Ventures and Data Tech Fund all participating.


Source link

Steven L. Nielsen