Class OpenLineageLogStoreConnectorBase
java.lang.Object
org.odpi.openmetadata.frameworks.connectors.Connector
org.odpi.openmetadata.frameworks.connectors.ConnectorBase
org.odpi.openmetadata.frameworks.integration.connectors.IntegrationConnectorBase
org.odpi.openmetadata.integrationservices.lineage.connector.LineageIntegratorConnector
org.odpi.openmetadata.adapters.connectors.integration.openlineage.OpenLineageLogStoreConnectorBase
- All Implemented Interfaces:
OpenLineageLogStore
,AuditLoggingComponent
,SecureConnectorExtension
,VirtualConnectorExtension
,IntegrationConnector
,LineageIntegratorOMISConnector
,OpenLineageEventListener
- Direct Known Subclasses:
APIBasedOpenLineageLogStoreConnector
,FileBasedOpenLineageLogStoreConnector
public abstract class OpenLineageLogStoreConnectorBase
extends LineageIntegratorConnector
implements OpenLineageLogStore, OpenLineageEventListener
OpenLineageLogStoreConnectorBase is the base class for connectors that support the OMRSAuditLog.
It has implementations of the query methods that throw "function not supported". This means that
log destinations that do not support queries can ignore these methods.
It also supports the start and stop method for the connector which only need to be
overridden if the connector has work to do at these times
-
Nested Class Summary
Nested classes/interfaces inherited from class org.odpi.openmetadata.frameworks.connectors.ConnectorBase
ConnectorBase.ProtectedConnection
-
Field Summary
Fields inherited from class org.odpi.openmetadata.frameworks.integration.connectors.IntegrationConnectorBase
auditLog, catalogTargetsManager, connectorName, integrationContext, propertyHelper
Fields inherited from class org.odpi.openmetadata.frameworks.connectors.ConnectorBase
connectedAssetProperties, connectionBean, connectionProperties, connectorInstanceId, embeddedConnectors, messageFormatter, secretsStoreConnectorMap
-
Constructor Summary
-
Method Summary
Modifier and TypeMethodDescriptionvoid
initialize
(String connectorInstanceId, ConnectionProperties connectionProperties) Call made by the ConnectorProvider to initialize the Connector with the base services.protected void
logNoRawEvent
(OpenLineageRunEvent openLineageEvent) Throws an invalid parameter exception.protected abstract void
newDestinationIdentified
(String destinationAddress) Informs the subclasses that there is a new destination - in case they need to do special setup.void
processOpenLineageRunEvent
(OpenLineageRunEvent event, String rawEvent) Called each time an open lineage run event is published to the Lineage Integrator OMIS.void
refresh()
Maintains the list of catalog targets.void
start()
Indicates that the connector is completely configured and can begin processing.abstract void
storeEvent
(OpenLineageRunEvent openLineageEvent, String rawEvent, String logDestinationAddress) Store the open lineage event in the open lineage log store.Methods inherited from class org.odpi.openmetadata.integrationservices.lineage.connector.LineageIntegratorConnector
getContext, setContext
Methods inherited from class org.odpi.openmetadata.frameworks.integration.connectors.IntegrationConnectorBase
disconnect, engage, getConnectorComponentDescription, getNetworkAddress, refreshCatalogTargets, registerCatalogTargetChangeListener, setAuditLog, setConnectorName, setContext
Methods inherited from class org.odpi.openmetadata.frameworks.connectors.ConnectorBase
combineConfigurationProperties, disconnectConnectors, equals, getArrayConfigurationProperty, getAssetTypeName, getBooleanConfigurationProperty, getConnectedAssetProperties, getConnection, getConnectorInstanceId, getIntConfigurationProperty, getLongConfigurationProperty, getNetworkAddresses, getStringConfigurationProperty, getSuppliedPlaceholderProperties, hashCode, initializeConnectedAssetProperties, initializeEmbeddedConnectors, initializeSecretsStoreConnector, isActive, logExceptionRecord, logRecord, logRecord, throwMissingConfigurationProperty, throwMissingResource, throwNoAsset, throwWrongTypeOfAsset, throwWrongTypeOfResource, throwWrongTypeOfRootSchema, toString
Methods inherited from class org.odpi.openmetadata.frameworks.connectors.Connector
clearStatisticProperty, clearStatisticTimestamp, getConnectorStatistics, getStatisticCounter, getStatisticProperty, getStatisticTimestamp, incrementStatisticCounter, initializeStatisticCounter, setStatisticProperty, setStatisticTimestamp
Methods inherited from class java.lang.Object
clone, finalize, getClass, notify, notifyAll, wait, wait, wait
Methods inherited from interface org.odpi.openmetadata.frameworks.connectors.VirtualConnectorExtension
initializeEmbeddedConnectors
-
Field Details
-
distributorName
-
myContext
-
destinationAddresses
-
-
Constructor Details
-
OpenLineageLogStoreConnectorBase
public OpenLineageLogStoreConnectorBase()Default constructor
-
-
Method Details
-
storeEvent
public abstract void storeEvent(OpenLineageRunEvent openLineageEvent, String rawEvent, String logDestinationAddress) throws InvalidParameterException, UserNotAuthorizedException, PropertyServerException Store the open lineage event in the open lineage log store. If the raw event is null, a json version of the open lineage event is generated using the Egeria beans.- Specified by:
storeEvent
in interfaceOpenLineageLogStore
- Parameters:
openLineageEvent
- event formatted using Egeria beansrawEvent
- event in Json form from the originator - may have facets that are not known to EgerialogDestinationAddress
- address to send the event to- Throws:
InvalidParameterException
- indicates that the openLineageEvent parameter is invalid.UserNotAuthorizedException
- indicates that the caller is not authorized to access the log store.PropertyServerException
- indicates that the log store is not available or has an error.
-
newDestinationIdentified
protected abstract void newDestinationIdentified(String destinationAddress) throws ConnectorCheckedException Informs the subclasses that there is a new destination - in case they need to do special setup.- Parameters:
destinationAddress
- new destination- Throws:
ConnectorCheckedException
- new destination not valid
-
initialize
Call made by the ConnectorProvider to initialize the Connector with the base services.- Overrides:
initialize
in classConnectorBase
- Parameters:
connectorInstanceId
- unique id for the connector instance useful for messages etcconnectionProperties
- POJO for the configuration used to create the connector.
-
start
Indicates that the connector is completely configured and can begin processing.- Specified by:
start
in interfaceIntegrationConnector
- Overrides:
start
in classIntegrationConnectorBase
- Throws:
ConnectorCheckedException
- there is a problem within the connector.
-
refresh
Maintains the list of catalog targets.- Specified by:
refresh
in interfaceIntegrationConnector
- Throws:
ConnectorCheckedException
- there is a problem with the connector. It is not able to refresh the catalog targets.
-
processOpenLineageRunEvent
Called each time an open lineage run event is published to the Lineage Integrator OMIS. The integration connector is able to work with the formatted event using the Egeria beans or reformat the open lineage run event using the supplied open lineage backend beans or another set of beans.- Specified by:
processOpenLineageRunEvent
in interfaceOpenLineageEventListener
- Parameters:
event
- run event formatted using Egeria supplied beans (null if egeria can not format the event)rawEvent
- json payload received for the event
-
logNoRawEvent
Throws an invalid parameter exception. Used by the subclasses when this class has failed to pass a raw event.- Parameters:
openLineageEvent
- supplied open lineage event - may also be null- Throws:
InvalidParameterException
- resulting exception
-