Package org.odpi.openmetadata.adapters.connectors.unitycatalog.sync

Class OSSUnityCatalogInsideCatalogSyncBase

java.lang.Object

org.odpi.openmetadata.adapters.connectors.unitycatalog.sync.OSSUnityCatalogInsideCatalogSyncBase

Direct Known Subclasses:

OSSUnityCatalogInsideCatalogSyncFunctions, OSSUnityCatalogInsideCatalogSyncRegisteredModels, OSSUnityCatalogInsideCatalogSyncSchema, OSSUnityCatalogInsideCatalogSyncTables, OSSUnityCatalogInsideCatalogSyncVolumes, OSSUnityCatalogServerSyncCatalog

public abstract class OSSUnityCatalogInsideCatalogSyncBase
extends Object

Common functions for the synchronizing between Egeria and Unity Catalog.

Field Summary

Fields

Modifier and Type	Field	Description
protected final AssetClient	assetClient
protected final AuditLog	auditLog
protected final String	catalogGUID
protected final String	catalogTargetName
protected final String	catalogTypeName
protected final Map<String,Object>	configurationProperties
protected final String	connectorName
protected final IntegrationContext	context
protected final UnityCatalogDeployedImplementationType	deployedImplementationType
protected final String	entityTypeName
protected final List<String>	excludeNames
protected final ExternalIdClient	externalIdClient
protected final List<String>	includeNames
protected final String	metadataCollectionQualifiedName
protected final OpenMetadataStore	openMetadataStore
protected final PropertyFacetClient	propertyFacetClient
protected final PropertyHelper	propertyHelper
protected final PermittedSynchronization	targetPermittedSynchronization
protected final String	templateGUID
protected final Map<String,String>	templates
protected final OSSUnityCatalogResourceConnector	ucConnector
protected final Map<String,String>	ucFullNameToEgeriaGUID
protected final String	ucServerEndpoint

Constructor Summary

Constructors

Constructor

Description

OSSUnityCatalogInsideCatalogSyncBase(String connectorName, IntegrationContext context, String catalogTargetName, String catalogGUID, String metadataCollectionQualifiedName, Map<String,String> ucFullNameToEgeriaGUID, PermittedSynchronization targetPermittedSynchronization, OSSUnityCatalogResourceConnector ucConnector, String ucServerEndpoint, String entityTypeName, UnityCatalogDeployedImplementationType deployedImplementationType, Map<String,String> templates, Map<String,Object> configurationProperties, List<String> excludeNames, List<String> includeNames, AuditLog auditLog)

Set up the schema synchronizer.

Method Summary

Modifier and Type	Method	Description
void	addExternalIdentifier(String openMetadataElementGUID, ElementBase ucElement, String schemaName, String elementName, String elementType, String id, PermittedSynchronization instanceSynchronizationDirection)	Add a new external identifier to an existing open metadata element.
protected void	addPropertyFacet(String parentGUID, String parentQualifiedName, BasicElementProperties basicElementProperties, Map<String,String> facetProperties)	Create the property facet for an Egeria element.
protected void	deleteElementInEgeria(MemberElement memberElement)	Delete an element from open metadata.
protected List<String>	getArrayConfigurationProperty(String propertyName, Map<String,Object> configurationProperties)	Retrieve a configuration property that is a comma-separated list of strings.
protected Date	getDateFromLong(long valueFromUC)	Convert the long value returned by UC into a date (or null).
protected ExternalIdLinkProperties	getExternalIdLinkProperties(ElementBase ucElement, String schemaName, String elementName, PermittedSynchronization instanceSynchronizationDirection)	Populate and return the external identifier properties for a UC Schema.
protected ExternalIdProperties	getExternalIdProperties(ElementBase ucElement, String elementType, String id)	Populate and return the external identifier properties for a UC Schema.
protected String	getPathNameFromStorageLocation(String storageLocation)	Convert the US format for the storage location into a proper pathname.
protected String	getQualifiedName(String fullName)	Return the qualified name of an element using UnityCatalog's full name.
protected String	getUCCatalogFromMember(MemberElement memberElement)	Extract the comment for a UC element from the description attribute in an egeria element.
protected String	getUCCommentFomMember(MemberElement memberElement)	Extract the comment for a UC element from the description attribute in an egeria element.
protected String	getUCNameFromMember(MemberElement memberElement)	Extract the name of a UC element from the name attribute in an egeria element (which is the full name).
protected Map<String,String>	getUCPropertiesFromMember(MemberElement memberElement)	Extract the properties for a UC element from the additionalProperties attribute in an egeria element.
protected String	getUCSchemaFromMember(MemberElement memberElement)	Extract the schema name for a UC element from the description attribute in an egeria element.
protected String	getUCStorageLocationFromMember(MemberElement memberElement)	Extract the storage location from the
protected boolean	noMismatchInExternalIdentifier(String thirdPartyExternalIdentifier, MemberElement memberElement)	Check that the name of the third party element has not changed with respect to Egeria.
protected abstract IntegrationIterator	refreshEgeria(String parentGUID, String parentRelationshipTypeName, RelationshipBeanProperties relationshipProperties)	Review all the elements for the subtype stored in Egeria.
protected abstract void	refreshUnityCatalog(String parentGUID, String parentRelationshipTypeName, RelationshipBeanProperties relationshipProperties, IntegrationIterator iterator)	Review all the elements for the subtype stored in UC.
protected void	throwBadBeanClass(OpenMetadataType expectedType, String methodName, OpenMetadataRootElement element)	Indicate that an element does not have properties of the expected bean class.
protected void	throwMissingPropertyValue(String missingPropertyName, String methodName, OpenMetadataRootElement element)	Throw an exception because a necessary property is missing.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

propertyHelper

protected final PropertyHelper propertyHelper

connectorName

protected final String connectorName

context

protected final IntegrationContext context

catalogGUID

protected final String catalogGUID

catalogTypeName

protected final String catalogTypeName

metadataCollectionQualifiedName

protected final String metadataCollectionQualifiedName

catalogTargetName

protected final String catalogTargetName

targetPermittedSynchronization

protected final PermittedSynchronization targetPermittedSynchronization

ucConnector

protected final OSSUnityCatalogResourceConnector ucConnector

ucServerEndpoint

protected final String ucServerEndpoint

entityTypeName

protected final String entityTypeName

deployedImplementationType

protected final UnityCatalogDeployedImplementationType deployedImplementationType

templates

protected final Map<String,String> templates

configurationProperties

protected final Map<String,Object> configurationProperties

auditLog

protected final AuditLog auditLog

openMetadataStore

protected final OpenMetadataStore openMetadataStore

externalIdClient

protected final ExternalIdClient externalIdClient

propertyFacetClient

protected final PropertyFacetClient propertyFacetClient

assetClient

protected final AssetClient assetClient

excludeNames

protected final List<String> excludeNames

includeNames

protected final List<String> includeNames

templateGUID

protected final String templateGUID

ucFullNameToEgeriaGUID

protected final Map<String,String> ucFullNameToEgeriaGUID

Constructor Details

OSSUnityCatalogInsideCatalogSyncBase

public OSSUnityCatalogInsideCatalogSyncBase(String connectorName,
 IntegrationContext context,
 String catalogTargetName,
 String catalogGUID,
 String metadataCollectionQualifiedName,
 Map<String,String> ucFullNameToEgeriaGUID,
 PermittedSynchronization targetPermittedSynchronization,
 OSSUnityCatalogResourceConnector ucConnector,
 String ucServerEndpoint,
 String entityTypeName,
 UnityCatalogDeployedImplementationType deployedImplementationType,
 Map<String,String> templates,
 Map<String,Object> configurationProperties,
 List<String> excludeNames,
 List<String> includeNames,
 AuditLog auditLog)
                                     throws UserNotAuthorizedException,
InvalidParameterException

Set up the schema synchronizer.

Parameters:

connectorName - name of this connector

context - context for the connector

catalogTargetName - the catalog target name

catalogGUID - guid of the catalog

metadataCollectionQualifiedName - name of the metadata collection for this UC server

ucFullNameToEgeriaGUID - map of full names from UC to the GUID of the entity in Egeria.

targetPermittedSynchronization - the policy that controls the direction of metadata exchange

ucConnector - connector for accessing UC

entityTypeName - type name of asset entity used to represent this type of element

deployedImplementationType - technology type

ucServerEndpoint - the server endpoint used to constructing the qualified names

templates - templates supplied through the catalog target

configurationProperties - configuration properties supplied through the catalog target

excludeNames - list of catalogs to ignore (and include all others)

includeNames - list of catalogs to include (and ignore all others) - overrides excludeCatalogs

auditLog - logging destination

Throws:

UserNotAuthorizedException - connector disconnected

InvalidParameterException - missing template

Method Details

refreshEgeria

protected abstract IntegrationIterator refreshEgeria(String parentGUID,
 String parentRelationshipTypeName,
 RelationshipBeanProperties relationshipProperties)
                                              throws InvalidParameterException,
PropertyServerException,
UserNotAuthorizedException,
ConnectorCheckedException

Review all the elements for the subtype stored in Egeria.

Parameters:

parentGUID - unique identifier of the parent

parentRelationshipTypeName - relationship type between parent and elements to iterate through

relationshipProperties - optional properties for the relationship

Returns:

iterator

Throws:

InvalidParameterException - parameter error

PropertyServerException - repository error

UserNotAuthorizedException - security error

ConnectorCheckedException - logic error in properties

refreshUnityCatalog

protected abstract void refreshUnityCatalog(String parentGUID,
 String parentRelationshipTypeName,
 RelationshipBeanProperties relationshipProperties,
 IntegrationIterator iterator)
                                     throws InvalidParameterException,
PropertyServerException,
UserNotAuthorizedException,
ConnectorCheckedException

Review all the elements for the subtype stored in UC.

Parameters:

parentGUID - unique identifier of the parent

parentRelationshipTypeName - relationship type between parent and elements to iterate through

relationshipProperties - optional properties for the relationship

iterator - iterator of choice

Throws:

InvalidParameterException - parameter error

PropertyServerException - repository error

UserNotAuthorizedException - security error

ConnectorCheckedException - logic error in properties

getDateFromLong

protected Date getDateFromLong(long valueFromUC)

Convert the long value returned by UC into a date (or null).

Parameters:

valueFromUC - long

Returns:

date

getQualifiedName

protected String getQualifiedName(String fullName)

Return the qualified name of an element using UnityCatalog's full name.

Parameters:

fullName - dotted name

Returns:

qualified name

deleteElementInEgeria

protected void deleteElementInEgeria(MemberElement memberElement)
                              throws InvalidParameterException,
PropertyServerException,
UserNotAuthorizedException

Delete an element from open metadata.

Parameters:

memberElement - element to delete

Throws:

InvalidParameterException - parameter error

PropertyServerException - repository error

UserNotAuthorizedException - authorization error

addExternalIdentifier

public void addExternalIdentifier(String openMetadataElementGUID,
 ElementBase ucElement,
 String schemaName,
 String elementName,
 String elementType,
 String id,
 PermittedSynchronization instanceSynchronizationDirection)
                           throws InvalidParameterException,
UserNotAuthorizedException,
PropertyServerException

Add a new external identifier to an existing open metadata element.

Parameters:

openMetadataElementGUID - unique identifier (GUID) of the element in the open metadata ecosystem

ucElement - values from Unity Catalog element

schemaName - name of the schema

elementName - element type (from UC)

instanceSynchronizationDirection - direction the metadata is flowing

Throws:

InvalidParameterException - one of the parameters is invalid

UserNotAuthorizedException - user not authorized to issue this request

PropertyServerException - problem accessing the property server

getExternalIdProperties

protected ExternalIdProperties getExternalIdProperties(ElementBase ucElement,
 String elementType,
 String id)

Populate and return the external identifier properties for a UC Schema.

Parameters:

ucElement - values from Unity Catalog element

elementType - element type (from UC)

id - element id (from UC)

Returns:

external identifier properties

getExternalIdLinkProperties

protected ExternalIdLinkProperties getExternalIdLinkProperties(ElementBase ucElement,
 String schemaName,
 String elementName,
 PermittedSynchronization instanceSynchronizationDirection)

Populate and return the external identifier properties for a UC Schema.

Parameters:

ucElement - values from Unity Catalog element

schemaName - name of the schema

elementName - element type (from UC)

instanceSynchronizationDirection - direction the metadata is flowing

Returns:

external identifier properties

addPropertyFacet

protected void addPropertyFacet(String parentGUID,
 String parentQualifiedName,
 BasicElementProperties basicElementProperties,
 Map<String,String> facetProperties)
                         throws InvalidParameterException,
PropertyServerException,
UserNotAuthorizedException

Create the property facet for an Egeria element. This holds the vendor specific properties for the element.

Parameters:

parentGUID - the parent (and anchor) unique name

parentQualifiedName - qualifiedName of the parent element

basicElementProperties - common properties of an element

facetProperties - these are the specialist property for the linked element.

Throws:

InvalidParameterException

PropertyServerException

UserNotAuthorizedException

noMismatchInExternalIdentifier

protected boolean noMismatchInExternalIdentifier(String thirdPartyExternalIdentifier,
 MemberElement memberElement)

Check that the name of the third party element has not changed with respect to Egeria.

Parameters:

thirdPartyExternalIdentifier - id from Unity Catalog

memberElement - element from Egeria

Returns:

boolean

throwMissingPropertyValue

protected void throwMissingPropertyValue(String missingPropertyName,
 String methodName,
 OpenMetadataRootElement element)
                                  throws ConnectorCheckedException

Throw an exception because a necessary property is missing. The description portrays this a s logic error.

Parameters:

missingPropertyName - missing property name

methodName - calling method

element - element in error

Throws:

ConnectorCheckedException - resulting exception

throwBadBeanClass

protected void throwBadBeanClass(OpenMetadataType expectedType,
 String methodName,
 OpenMetadataRootElement element)
                          throws ConnectorCheckedException

Indicate that an element does not have properties of the expected bean class. This is documented as a logic error.

Parameters:

expectedType - open metadata type information

methodName - calling method

element - element in error

Throws:

ConnectorCheckedException - resulting exception

getUCCatalogFromMember

protected String getUCCatalogFromMember(MemberElement memberElement)
                                 throws ConnectorCheckedException

Extract the comment for a UC element from the description attribute in an egeria element.

Parameters:

memberElement - element from Egeria

Returns:

catalog for UC

Throws:

ConnectorCheckedException - logic error in properties

getUCSchemaFromMember

protected String getUCSchemaFromMember(MemberElement memberElement)
                                throws ConnectorCheckedException

Extract the schema name for a UC element from the description attribute in an egeria element.

Parameters:

memberElement - element from Egeria

Returns:

schema for UC

Throws:

ConnectorCheckedException - logic error in properties

getUCNameFromMember

protected String getUCNameFromMember(MemberElement memberElement)
                              throws ConnectorCheckedException

Extract the name of a UC element from the name attribute in an egeria element (which is the full name).

Parameters:

memberElement - element from Egeria

Returns:

name for UC

Throws:

ConnectorCheckedException - logic error in properties

getUCCommentFomMember

protected String getUCCommentFomMember(MemberElement memberElement)
                                throws ConnectorCheckedException

Extract the comment for a UC element from the description attribute in an egeria element.

Parameters:

memberElement - element from Egeria

Returns:

comment for UC

Throws:

ConnectorCheckedException - logic error in properties

getUCPropertiesFromMember

protected Map<String,String> getUCPropertiesFromMember(MemberElement memberElement)
                                                throws ConnectorCheckedException

Extract the properties for a UC element from the additionalProperties attribute in an egeria element.

Parameters:

memberElement - element from Egeria

Returns:

properties for UC

Throws:

ConnectorCheckedException

getUCStorageLocationFromMember

protected String getUCStorageLocationFromMember(MemberElement memberElement)

Extract the storage location from the

Parameters:

memberElement - elements from egeria

Returns:

storage location (may be null

getPathNameFromStorageLocation

protected String getPathNameFromStorageLocation(String storageLocation)

Convert the US format for the storage location into a proper pathname.

Parameters:

storageLocation - storage location from UC

Returns:

pathname

getArrayConfigurationProperty

protected List<String> getArrayConfigurationProperty(String propertyName,
 Map<String,Object> configurationProperties)

Retrieve a configuration property that is a comma-separated list of strings.

Parameters:

propertyName - name of property

configurationProperties - configuration properties

Returns:

list of strings or null if not set