Package org.odpi.openmetadata.adapters.connectors.integration.openlineage.postgres

Class PostgresOpenLineageLogStoreWriter

java.lang.Object

org.odpi.openmetadata.adapters.connectors.integration.openlineage.postgres.PostgresOpenLineageLogStoreWriter

public class PostgresOpenLineageLogStoreWriter
extends Object

PostgresOpenLineageLogStoreWriter issues the SQL that maintains one PostgreSQL OpenLineage log store: it creates the schema and tables, stores each event with its contribution to the digest in one transaction, and purges old records. Each call takes its own connection from the data source and returns it before it returns.

Constructor Summary

Constructors

Constructor	Description
PostgresOpenLineageLogStoreWriter(DataSource dataSource, PostgresOpenLineageLogStoreSchema schema)	Constructor

Method Summary

Modifier and Type	Method	Description
void	createSchema()	Create the schema, its tables and indexes, if they do not exist.
List<Integer>	purge(Date cutOff)	Delete the records older than a cut-off time in one transaction.
boolean	store(PostgresOpenLineageEventDigest digest, String jsonEvent)	Store an event and its contribution to the digest in one transaction.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

PostgresOpenLineageLogStoreWriter

public PostgresOpenLineageLogStoreWriter(DataSource dataSource,
 PostgresOpenLineageLogStoreSchema schema)

Constructor

Parameters:

dataSource - source of pooled connections (auto-commit disabled)

schema - definitions of the log store

Method Details

createSchema

public void createSchema()
                  throws SQLException

Create the schema, its tables and indexes, if they do not exist.

Throws:

SQLException - problem with the database

store

public boolean store(PostgresOpenLineageEventDigest digest,
 String jsonEvent)
              throws Exception

Store an event and its contribution to the digest in one transaction. An event that is already stored (because it has been delivered twice) is skipped, so the digest counts it once.

Parameters:

digest - digest of the event

jsonEvent - event as JSON

Returns:

true if the event was stored; false if it was already there

Throws:

Exception - problem with the database or the event

purge

public List<Integer> purge(Date cutOff)
                    throws SQLException

Delete the records older than a cut-off time in one transaction.

Parameters:

cutOff - records before this time are deleted

Returns:

number of events, runs, assertions and tests deleted (in that order)

Throws:

SQLException - problem with the database