Matillion ETL for Delta Lake on Databricks GA

April 1, 2021

Matillion ETL for Delta Lake on Databricks enables data professionals across the business to aggregate and share data in a single environment for improved cross-functional decision-making. Through the powerful, scalable combination of Matillion ETL and Databricks, enterprises access insights from large, dynamic datasets on a unified platform, empowering data teams to take full advantage of modern lakehouse architecture.

According to a recent IDG survey, more than half of enterprises (56%) plan to use data lakes in the future. Databricks is a pioneer of the modern lakehouse architecture, combining data warehousing performance with data lake economics. Matillion ETL for Delta Lake on Databricks brings no-code/low-code data integration to a lakehouse architecture. Users across the business can take ownership of their data, leveraging best-in-class data transformation from Matillion ETL to enable on-demand machine learning, faster reporting, and BI improvements powered by Delta Lake.

With cloud-native Matillion ETL for Delta Lake on Databricks, enterprise data teams create instantly shareable datasets between previously disparate analytics and data science environments.

This allows data professionals to:

Create quick and repeatable data ingestion: Automate data ingestion workflows into Delta Lake without hand-coding pipelines, using pre-built data sources components including cloud and on-prem databases, files, NoSQL, and SaaS applications. Use Matillion's Create Your Own Connector to easily build custom connectors to any REST API data source. Users can also schedule automated data loads to update Databricks tables at specific intervals.

Support complex data transformations: With a single tool, data teams can quickly access all the data from complex data types and semi-structured data within minutes. Matillion ETL allows data teams to take advantage of all existing Delta Lake features, including ACID transactions, Schema enforcement, and Time Travel to prepare their data for insights.

Combine business logic and underlying transformations: With Matillion ETL's graphical user interface, data professionals can easily design and reuse transformations for analytics and data science purposes, while leveraging the compute power and scalability of Delta Lake.

"Enterprises want to complete using data, but struggle to make data useful quickly for projects such as AI/ML and analytics. Matillion ETL for Delta Lake on Databricks makes it easier for enterprises to make their data useful, taking the pain out of manual data ingestion and transformation tasks to enable faster time to insights," said Matthew Scullion, CEO of Matillion. "We are excited to work alongside Databricks' popular lakehouse architecture to improve time to value for enterprises."

"The lakehouse architecture allows traditional analytics, data science, and machine learning to co-exist on the same platform, opening up even more possibilities for enterprise data initiatives," said Pankaj Dugar, VP of Product Partnerships at Databricks. "Matillion's data integration platform enables data users to get to insights faster, transforming data and orchestrating workflows in a single environment. We are proud to partner with their leading technology and empower enterprises to expand data access for all employees."