Class OpenLineageDatabaseLogStoreReader

java.lang.Object
org.odpi.openmetadata.adapters.connectors.lineageinsight.openlineage.OpenLineageDatabaseLogStoreReader

public class OpenLineageDatabaseLogStoreReader extends Object
OpenLineageDatabaseLogStoreReader builds the OpenLineage run history for an analysis window from the digest tables of a PostgreSQL OpenLineage log store (written by the PostgreSQL OpenLineage log store integration connector). Rather than parsing every event, it reads one row per run, one per dataset each run read or wrote, and the assertions and tests reported in the window - each an indexed range scan.

The history differs from the one built by OpenLineageLogStoreReader in one respect: a run that had an event in the window is read whole, so a run that started before the window and finished inside it has its start time, and so its duration.

  • Constructor Details

    • OpenLineageDatabaseLogStoreReader

      public OpenLineageDatabaseLogStoreReader()
  • Method Details

    • read

      public OpenLineageRunHistory read(DataSource dataSource, String schemaName, Date windowStart, Date windowEnd) throws SQLException
      Build the history for a window.
      Parameters:
      dataSource - source of database connections
      schemaName - name of the database schema holding the log store
      windowStart - start of the window (null for no lower bound)
      windowEnd - end of the window (null for no upper bound)
      Returns:
      history
      Throws:
      SQLException - problem reading the database