Package org.odpi.openmetadata.adapters.connectors.lineageinsight.openlineage

Class LovelaceOpenLineageDataScopeService

java.lang.Object

org.odpi.openmetadata.frameworks.connectors.Connector

org.odpi.openmetadata.frameworks.connectors.ConnectorBase

org.odpi.openmetadata.frameworks.opengovernance.GovernanceActionServiceConnector

org.odpi.openmetadata.frameworks.opengovernance.GeneralGovernanceActionService

org.odpi.openmetadata.adapters.connectors.lineageinsight.openlineage.LovelaceOpenLineageAnalysisServiceBase

org.odpi.openmetadata.adapters.connectors.lineageinsight.openlineage.LovelaceOpenLineageDataScopeService

All Implemented Interfaces:

AuditLoggingComponent, SecureConnectorExtension, VirtualConnectorExtension, GovernanceActionService

public class LovelaceOpenLineageDataScopeService
extends LovelaceOpenLineageAnalysisServiceBase

LovelaceOpenLineageDataScopeService analyses the series of writes to each dataset in an OpenLineage log store and refines the DataScope classification of the dataset's asset. The OpenLineage cataloguer extends the data collection window as each write arrives; this service looks at the whole series to decide whether the store is replaced or accumulates (from lifecycleStateChange and subset facets), sets the collection start time accordingly, and records the write and read pattern in the classification's additional properties.

Nested Class Summary

Nested classes/interfaces inherited from class org.odpi.openmetadata.frameworks.connectors.ConnectorBase

ConnectorBase.ProtectedConnection

Field Summary

Fields inherited from class org.odpi.openmetadata.adapters.connectors.lineageinsight.openlineage.LovelaceOpenLineageAnalysisServiceBase

ANALYSIS_SERVICE_PROPERTY, ANALYSIS_TIME_PROPERTY, ANALYSIS_WINDOW_END_PROPERTY, ANALYSIS_WINDOW_START_PROPERTY, analysisTime, windowEnd, windowStart

Fields inherited from class org.odpi.openmetadata.frameworks.opengovernance.GeneralGovernanceActionService

governanceContext

Fields inherited from class org.odpi.openmetadata.frameworks.opengovernance.GovernanceActionServiceConnector

governanceServiceName, propertyHelper

Fields inherited from class org.odpi.openmetadata.frameworks.connectors.ConnectorBase

auditLog, connectionBean, connectorInstanceId, embeddedConnectors, messageFormatter, secretsStoreConnectorMap

Constructor Summary

Constructors

Constructor	Description
LovelaceOpenLineageDataScopeService()	Default constructor

Method Summary

Modifier and Type	Method	Description
protected int	analyse(OpenLineageRunHistory history)	Refine the data scope of each dataset.
protected String	getAnalysisDescription()	Describe what the service produces.

Methods inherited from class org.odpi.openmetadata.adapters.connectors.lineageinsight.openlineage.LovelaceOpenLineageAnalysisServiceBase

addAnalysisProperties, findDataAsset, findProcess, formatDate, formatDuration, getWindowDays, put, put, start

Methods inherited from class org.odpi.openmetadata.frameworks.opengovernance.GeneralGovernanceActionService

disconnect, getActionTarget, getAllActionTargets, getProperty, setGovernanceContext

Methods inherited from class org.odpi.openmetadata.frameworks.opengovernance.GovernanceActionServiceConnector

getArrayRequestParameter, getBooleanRequestParameter, getConnectorComponentDescription, getIntRequestParameter, getLongRequestParameter, getStringRequestParameter, handleUnexpectedException, setAuditLog, setGovernanceServiceName, validateContext

Methods inherited from class org.odpi.openmetadata.frameworks.connectors.ConnectorBase

combineConfigurationProperties, disconnectConnectors, equals, fromCamelToCanonicalCase, fromCanonicalToCamelCase, fromCanonicalToKebabCase, fromCanonicalToSnakeCase, fromKebabToCanonicalCase, fromSnakeToCanonicalCase, getArrayConfigurationProperty, getArrayConfigurationProperty, getArrayConfigurationProperty, getBooleanConfigurationProperty, getConnection, getConnectorInstanceId, getDateConfigurationProperty, getIntConfigurationProperty, getLongConfigurationProperty, getSecretsCollectionName, getSecretsLocation, getSecretsProvider, getStringConfigurationProperty, getStringConfigurationProperty, getSuppliedPlaceholderProperties, hashCode, initialize, initializeEmbeddedConnectors, initializeSecretsStoreConnector, isActive, logExceptionRecord, logExceptionRecord, logRecord, logRecord, throwMissingConfigurationProperty, throwMissingElement, throwMissingEndpointAddress, throwMissingResource, throwNoAsset, throwWrongTypeOfAsset, throwWrongTypeOfResource, throwWrongTypeOfRootSchema, toString

Methods inherited from class org.odpi.openmetadata.frameworks.connectors.Connector

clearStatisticProperty, clearStatisticTimestamp, getConnectorStatistics, getStatisticCounter, getStatisticProperty, getStatisticTimestamp, incrementStatisticCounter, initializeStatisticCounter, refreshStatistics, setStatisticProperty, setStatisticTimestamp

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Details

LovelaceOpenLineageDataScopeService

public LovelaceOpenLineageDataScopeService()

Default constructor

Method Details

getAnalysisDescription

protected String getAnalysisDescription()

Describe what the service produces.

Specified by:

getAnalysisDescription in class LovelaceOpenLineageAnalysisServiceBase

Returns:

description

analyse

protected int analyse(OpenLineageRunHistory history)
               throws InvalidParameterException,
PropertyServerException,
UserNotAuthorizedException

Refine the data scope of each dataset.

Specified by:

analyse in class LovelaceOpenLineageAnalysisServiceBase

Parameters:

history - events in the analysis window

Returns:

number of assets updated

Throws:

InvalidParameterException - invalid parameter

PropertyServerException - repository problem

UserNotAuthorizedException - security problem