Version: Next

DataHub Releases

Summary

Released on 2024-02-29 by @RyanHolstien.

NEW - Asset Documentation Forms & UI-Editable Properties: Define specific documentation requirements via a Form, and empower your asset owners to capture their valuable knowledge via UI-Editable Properties. Watch the demo here!
NEW - DataHub Incidents: Create, communicate, and data quality and observability incidents when they inevitably arise. Watch the demo here! UI Improvements: Editing secrets, handling forms, and rendering token pages and lineage diagrams have been improved for a smoother user interface experience.
UI Improvements: Editing secrets, handling forms, and rendering token pages and lineage diagrams have been improved for a smoother user interface experience.

Security Upgrades: Core dependencies like shiro-core and FastAPI have been upgraded to fix vulnerabilities, ensuring a safer development environment.
GraphQL/OpenAPI Enhancements: New GraphQL endpoints and better OpenAPI documentation provide more powerful tools for API interaction, making developers' jobs easier.
Performance Tuning: Backend improvements for search operations and ingestion processes make the platform faster and more reliable.

Platform Integrations: Enhanced support for dbt, Metabase, BigQuery, AWS Glue, Oracle, and Redshift allows for more comprehensive metadata capture, making integration with these platforms smoother.
Ingestion Framework: The reliability of ingestion has been improved, with new capabilities like support for tags from Tableau datasources and compatibility with Airflow 2.5.0, facilitating a broader range of data synchronization tasks.
Connector Improvements: Ingestion connectors for external data tools have been streamlined, ensuring easier integration and data synchronization.

Enhanced internal testing frameworks with Cypress and pytest-random-order for ingestion tests.
Simplified developer workflows with configurable Docker Compose project names in CLI.
Addressed various ingestion-related bugs for platforms like Feast and Snowflake.
Enhanced the UI codebase with TypeScript compilation linting and updated styles.
Streamlined CI processes for pull requests and linting conditions.
Version Upgrades: Upgraded pytest-docker, Pegasus, and SQLglot, among others, to improve stability and performance. Security vulnerabilities addressed by upgrading FastAPI, gitdb, and follow-redirects.

Updates to MySQL version for quickstarts and migration to Neo4j 5.x may impact existing setups.
JDK17 build requirement and Docker Compose > 2.20 needed for building DataHub.
Python 3.8+ requirement for the acryl-datahub CLI.
Changes in Unity Catalog ingestion source configs and Redshift lineage generation.
Deprecation of Spark 2.x and associated JDK8 build requirements.

For full details on breaking changes, please visit DataHub's update guide.

A huge thank you to all our contributors for making this release possible. Your hard work and dedication are greatly appreciated.