Package org.odpi.openmetadata.adapters.connectors.integration.openlineage

Class APIBasedOpenLineageLogStoreConnector

java.lang.Object

org.odpi.openmetadata.frameworks.connectors.Connector

org.odpi.openmetadata.frameworks.connectors.ConnectorBase

org.odpi.openmetadata.frameworks.integration.connectors.IntegrationConnectorBase

org.odpi.openmetadata.integrationservices.lineage.connector.LineageIntegratorConnector

org.odpi.openmetadata.adapters.connectors.integration.openlineage.OpenLineageLogStoreConnectorBase

org.odpi.openmetadata.adapters.connectors.integration.openlineage.APIBasedOpenLineageLogStoreConnector

All Implemented Interfaces:

OpenLineageLogStore, AuditLoggingComponent, SecureConnectorExtension, VirtualConnectorExtension, IntegrationConnector, LineageIntegratorOMISConnector, OpenLineageEventListener

public class APIBasedOpenLineageLogStoreConnector
extends OpenLineageLogStoreConnectorBase

APIBasedOpenLineageLogStoreConnector provides a connector implementation for an API based open lineage log. Each lineage record is passed to the supplied lineage API.

Nested Class Summary

Nested classes/interfaces inherited from class org.odpi.openmetadata.frameworks.connectors.ConnectorBase

ConnectorBase.ProtectedConnection

Field Summary

Fields inherited from class org.odpi.openmetadata.adapters.connectors.integration.openlineage.OpenLineageLogStoreConnectorBase

destinationAddresses, distributorName, myContext

Fields inherited from class org.odpi.openmetadata.frameworks.integration.connectors.IntegrationConnectorBase

auditLog, catalogTargetsManager, connectorName, integrationContext, propertyHelper

Fields inherited from class org.odpi.openmetadata.frameworks.connectors.ConnectorBase

connectedAssetDetails, connectionBean, connectionDetails, connectorInstanceId, embeddedConnectors, messageFormatter, secretsStoreConnectorMap

Constructor Summary

Constructors

Constructor	Description
APIBasedOpenLineageLogStoreConnector()	Default constructor used by the connector provider.

Method Summary

Modifier and Type	Method	Description
protected void	newDestinationIdentified(String destinationAddress)	Informs the subclasses that there is a new destination - in case they need to do special setup.
void	start()	Set up the name of the file store
void	storeEvent(OpenLineageRunEvent openLineageEvent, String rawEvent, String logStoreURL)	Store the open lineage event in the open lineage log store.

Methods inherited from class org.odpi.openmetadata.adapters.connectors.integration.openlineage.OpenLineageLogStoreConnectorBase

initialize, logNoRawEvent, processOpenLineageRunEvent, refresh

Methods inherited from class org.odpi.openmetadata.integrationservices.lineage.connector.LineageIntegratorConnector

getContext, setContext

Methods inherited from class org.odpi.openmetadata.frameworks.integration.connectors.IntegrationConnectorBase

disconnect, engage, getConnectorComponentDescription, getNetworkAddress, refreshCatalogTargets, registerCatalogTargetChangeListener, setAuditLog, setConnectorName, setContext

Methods inherited from class org.odpi.openmetadata.frameworks.connectors.ConnectorBase

combineConfigurationProperties, disconnectConnectors, equals, getArrayConfigurationProperty, getArrayConfigurationProperty, getArrayConfigurationProperty, getAssetTypeName, getBooleanConfigurationProperty, getConnectedAssetProperties, getConnection, getConnectorInstanceId, getDateConfigurationProperty, getIntConfigurationProperty, getLongConfigurationProperty, getNetworkAddresses, getStringConfigurationProperty, getStringConfigurationProperty, getSuppliedPlaceholderProperties, hashCode, initializeConnectedAssetProperties, initializeEmbeddedConnectors, initializeSecretsStoreConnector, isActive, logExceptionRecord, logRecord, logRecord, throwMissingConfigurationProperty, throwMissingResource, throwNoAsset, throwWrongTypeOfAsset, throwWrongTypeOfResource, throwWrongTypeOfRootSchema, toString

Methods inherited from class org.odpi.openmetadata.frameworks.connectors.Connector

clearStatisticProperty, clearStatisticTimestamp, getConnectorStatistics, getStatisticCounter, getStatisticProperty, getStatisticTimestamp, incrementStatisticCounter, initializeStatisticCounter, setStatisticProperty, setStatisticTimestamp

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.odpi.openmetadata.frameworks.connectors.VirtualConnectorExtension

initializeEmbeddedConnectors

Constructor Details

APIBasedOpenLineageLogStoreConnector

public APIBasedOpenLineageLogStoreConnector()

Default constructor used by the connector provider.

Method Details

start

public void start()
           throws ConnectorCheckedException

Set up the name of the file store

Specified by:

start in interface IntegrationConnector

Overrides:

start in class OpenLineageLogStoreConnectorBase

Throws:

ConnectorCheckedException - something went wrong

newDestinationIdentified

protected void newDestinationIdentified(String destinationAddress)

Informs the subclasses that there is a new destination - in case they need to do special setup.

Specified by:

newDestinationIdentified in class OpenLineageLogStoreConnectorBase

Parameters:

destinationAddress - new destination

storeEvent

public void storeEvent(OpenLineageRunEvent openLineageEvent,
 String rawEvent,
 String logStoreURL)
                throws InvalidParameterException

Store the open lineage event in the open lineage log store. If the raw event is null, a json version of the open lineage event is generated using the Egeria beans.

Specified by:

storeEvent in interface OpenLineageLogStore

Specified by:

storeEvent in class OpenLineageLogStoreConnectorBase

Parameters:

openLineageEvent - event formatted using Egeria beans

rawEvent - event in Json form from the originator - may have facets that are not known to Egeria

logStoreURL - address to send the event

Throws:

InvalidParameterException - indicates that the openLineageEvent parameter is invalid