Class FileBasedOpenLineageLogStoreConnector
java.lang.Object
org.odpi.openmetadata.frameworks.connectors.Connector
org.odpi.openmetadata.frameworks.connectors.ConnectorBase
org.odpi.openmetadata.frameworks.integration.connectors.IntegrationConnectorBase
org.odpi.openmetadata.integrationservices.lineage.connector.LineageIntegratorConnector
org.odpi.openmetadata.adapters.connectors.integration.openlineage.OpenLineageLogStoreConnectorBase
org.odpi.openmetadata.adapters.connectors.integration.openlineage.FileBasedOpenLineageLogStoreConnector
- All Implemented Interfaces:
OpenLineageLogStore
,AuditLoggingComponent
,SecureConnectorExtension
,VirtualConnectorExtension
,IntegrationConnector
,LineageIntegratorOMISConnector
,OpenLineageEventListener
FileBasedOpenLineageLogStoreConnector provides a connector implementation for a file based open lineage log.
The open lineage log is stored in a directory and each open lineage event record is stored as a file with a filename built
from the record's unique identifier (runId), time and status. The record is stored in a subdirectory that is made from the namespace and job.
-
Nested Class Summary
Nested classes/interfaces inherited from class org.odpi.openmetadata.frameworks.connectors.ConnectorBase
ConnectorBase.ProtectedConnection
-
Field Summary
Fields inherited from class org.odpi.openmetadata.adapters.connectors.integration.openlineage.OpenLineageLogStoreConnectorBase
destinationName, myContext
Fields inherited from class org.odpi.openmetadata.frameworks.integration.connectors.IntegrationConnectorBase
auditLog, connectorName, embeddedConnectors, integrationContext
Fields inherited from class org.odpi.openmetadata.frameworks.connectors.ConnectorBase
connectedAssetProperties, connectionBean, connectionProperties, connectorInstanceId, secretsStoreConnectorMap
-
Constructor Summary
ConstructorsConstructorDescriptionDefault constructor used by the connector provider. -
Method Summary
Modifier and TypeMethodDescriptionvoid
start()
Set up the name of the file storevoid
storeEvent
(OpenLineageRunEvent openLineageEvent, String rawEvent) Store the open lineage event in the open lineage log store.Methods inherited from class org.odpi.openmetadata.adapters.connectors.integration.openlineage.OpenLineageLogStoreConnectorBase
getDestinationName, initialize, logNoRawEvent, processOpenLineageRunEvent, refresh
Methods inherited from class org.odpi.openmetadata.integrationservices.lineage.connector.LineageIntegratorConnector
getContext, setContext
Methods inherited from class org.odpi.openmetadata.frameworks.integration.connectors.IntegrationConnectorBase
disconnect, engage, getConnectorComponentDescription, initializeEmbeddedConnectors, setAuditLog, setConnectorName, setContext
Methods inherited from class org.odpi.openmetadata.frameworks.connectors.ConnectorBase
disconnectConnectors, equals, getConnectedAssetProperties, getConnection, getConnectorInstanceId, hashCode, initializeConnectedAssetProperties, initializeSecretsStoreConnector, isActive, toString
Methods inherited from class org.odpi.openmetadata.frameworks.connectors.Connector
clearStatisticProperty, clearStatisticTimestamp, getConnectorStatistics, getStatisticCounter, getStatisticProperty, getStatisticTimestamp, incrementStatisticCounter, initializeStatisticCounter, setStatisticProperty, setStatisticTimestamp
-
Constructor Details
-
FileBasedOpenLineageLogStoreConnector
public FileBasedOpenLineageLogStoreConnector()Default constructor used by the connector provider.
-
-
Method Details
-
start
Set up the name of the file store- Specified by:
start
in interfaceIntegrationConnector
- Overrides:
start
in classOpenLineageLogStoreConnectorBase
- Throws:
ConnectorCheckedException
- something went wrong
-
storeEvent
public void storeEvent(OpenLineageRunEvent openLineageEvent, String rawEvent) throws InvalidParameterException, PropertyServerException Store the open lineage event in the open lineage log store. If the raw event is null, a json version of the open lineage event is generated using the Egeria beans.- Specified by:
storeEvent
in interfaceOpenLineageLogStore
- Specified by:
storeEvent
in classOpenLineageLogStoreConnectorBase
- Parameters:
openLineageEvent
- event formatted using Egeria beansrawEvent
- event in Json form from the originator - may have facets that are not known to Egeria- Throws:
InvalidParameterException
- indicates that the openLineageEvent parameter is invalid.PropertyServerException
- indicates that the log store is not available or has an error.
-