Package org.odpi.openmetadata.adapters.connectors.integration.openlineage

Class OpenLineageLogStoreConnectorBase

java.lang.Object
org.odpi.openmetadata.frameworks.connectors.Connector
org.odpi.openmetadata.frameworks.connectors.ConnectorBase
org.odpi.openmetadata.frameworks.integration.connectors.IntegrationConnectorBase
org.odpi.openmetadata.integrationservices.lineage.connector.LineageIntegratorConnector
org.odpi.openmetadata.adapters.connectors.integration.openlineage.OpenLineageLogStoreConnectorBase
All Implemented Interfaces:
OpenLineageLogStore, AuditLoggingComponent, SecureConnectorExtension, VirtualConnectorExtension, IntegrationConnector, LineageIntegratorOMISConnector, OpenLineageEventListener
Direct Known Subclasses:
APIBasedOpenLineageLogStoreConnector, FileBasedOpenLineageLogStoreConnector
public abstract class OpenLineageLogStoreConnectorBase extends LineageIntegratorConnector implements OpenLineageLogStore, OpenLineageEventListener
OpenLineageLogStoreConnectorBase is the base class for connectors that support the OMRSAuditLog. It has implementations of the query methods that throw "function not supported". This means that log destinations that do not support queries can ignore these methods. It also supports the start and stop method for the connector which only need to be overridden if the connector has work to do at these times

  • Field Details

  • Constructor Details

    • OpenLineageLogStoreConnectorBase

      public OpenLineageLogStoreConnectorBase()
      Default constructor

  • Method Details

    • getDestinationName

      public String getDestinationName()
      Return the name of this open lineage log destination.
      Specified by:
      getDestinationName in interface OpenLineageLogStore
      Returns:
      string display name suitable for messages.

    • storeEvent

      public abstract void storeEvent(OpenLineageRunEvent openLineageEvent, String rawEvent) throws InvalidParameterException, UserNotAuthorizedException, PropertyServerException
      Store the open lineage event in the open lineage log store. If the raw event is null, a json version of the open lineage event is generated using the Egeria beans.
      Specified by:
      storeEvent in interface OpenLineageLogStore
      Parameters:
      openLineageEvent - event formatted using Egeria beans
      rawEvent - event in Json form from the originator - may have facets that are not known to Egeria
      Throws:
      InvalidParameterException - indicates that the openLineageEvent parameter is invalid.
      UserNotAuthorizedException - indicates that the caller is not authorized to access the log store.
      PropertyServerException - indicates that the log store is not available or has an error.

    • initialize

      public void initialize(String connectorInstanceId, ConnectionProperties connectionProperties)
      Call made by the ConnectorProvider to initialize the Connector with the base services.
      Overrides:
      initialize in class ConnectorBase
      Parameters:
      connectorInstanceId - unique id for the connector instance useful for messages etc
      connectionProperties - POJO for the configuration used to create the connector.

    • start

      public void start() throws ConnectorCheckedException
      Indicates that the connector is completely configured and can begin processing.
      Specified by:
      start in interface IntegrationConnector
      Overrides:
      start in class ConnectorBase
      Throws:
      ConnectorCheckedException - there is a problem within the connector.

    • refresh

      public void refresh() throws ConnectorCheckedException
      Requests that the connector does a comparison of the metadata in the third party technology and open metadata repositories. Refresh is called when the integration connector first starts and then at intervals defined in the connector's configuration as well as any external REST API calls to explicitly refresh the connector.
      Specified by:
      refresh in interface IntegrationConnector
      Throws:
      ConnectorCheckedException - there is a problem with the connector. It is not able to refresh the metadata.

    • processOpenLineageRunEvent

      public void processOpenLineageRunEvent(OpenLineageRunEvent event, String rawEvent)
      Called each time an open lineage run event is published to the Lineage Integrator OMIS. The integration connector is able to work with the formatted event using the Egeria beans or reformat the open lineage run event using the supplied open lineage backend beans or another set of beans.
      Specified by:
      processOpenLineageRunEvent in interface OpenLineageEventListener
      Parameters:
      event - run event formatted using Egeria supplied beans (null if egeria can not format the event)
      rawEvent - json payload received for the event

    • logNoRawEvent

      protected void logNoRawEvent(OpenLineageRunEvent openLineageEvent, String methodName) throws InvalidParameterException
      Throws an invalid parameter exception. Used by the subclasses when this class has failed to pass a raw event.
      Parameters:
      openLineageEvent - supplied open lineage event - may also be null
      methodName - calling method
      Throws:
      InvalidParameterException - resulting exception