Package org.odpi.openmetadata.adapters.connectors.apacheatlas.integration.modules

Class DatabaseIntegrationModuleBase

java.lang.Object

org.odpi.openmetadata.adapters.connectors.apacheatlas.integration.modules.AtlasIntegrationModuleBase

org.odpi.openmetadata.adapters.connectors.apacheatlas.integration.modules.AtlasRegisteredIntegrationModuleBase

org.odpi.openmetadata.adapters.connectors.apacheatlas.integration.modules.DatabaseIntegrationModuleBase

All Implemented Interfaces:

RegisteredIntegrationModule

Direct Known Subclasses:

ApacheHiveIntegrationModule, RDBMSIntegrationModule

public abstract class DatabaseIntegrationModuleBase
extends AtlasRegisteredIntegrationModuleBase

DatabaseIntegrationModuleBase abstracts the process of synchronizing relational metadata from Apache Atlas, so it is independent of the actual Apache Atlas types. The subclasses supply the types. This is because Apache Atlas has multiple type definitions for this type of metadata.

Field Summary

Fields inherited from class org.odpi.openmetadata.adapters.connectors.apacheatlas.integration.modules.AtlasRegisteredIntegrationModuleBase

propertyHelper

Fields inherited from class org.odpi.openmetadata.adapters.connectors.apacheatlas.integration.modules.AtlasIntegrationModuleBase

atlasAssetProperties, atlasClient, atlasCreateTimePropertyName, atlasDataFileProperties, atlasDescriptionPropertyName, atlasDisplayNamePropertyName, atlasModifiedTimePropertyName, atlasNamePropertyName, atlasOwnerPropertyName, atlasPathPropertyName, atlasQualifiedNamePropertyName, atlasUserDescriptionPropertyName, auditLog, classificationManagerClient, connectionDetails, connectorName, dataAssetClient, egeriaGUIDPropertyName, egeriaOwnedPropertyName, egeriaOwnerPropertyName, egeriaOwnerTypeName, egeriaQualifiedNamePropertyName, egeriaTypeNamePropertyName, embeddedConnectors, externalIdClient, glossaryClient, glossaryTermClient, informalTagClient, moduleName, myContext, openMetadataAssociatedElementPropertyName, openMetadataCorrelationLinkTypeName, openMetadataCorrelationPropertyName, openMetadataCorrelationTypeName, openMetadataGlossaryCorrelationLinkTypeName, openMetadataStore, schemaAttributeClient, schemaTypeClient, targetRootURL

Constructor Summary

Constructors

Constructor

Description

DatabaseIntegrationModuleBase(String connectorName, String moduleName, String atlasDatabaseTypeName, String atlasDatabaseTablesPropertyName, String atlasDatabaseTableTypeName, String atlasDatabaseColumnsPropertyName, String atlasDatabaseColumnTypeName, Connection connectionDetails, AuditLog auditLog, IntegrationContext myContext, String targetRootURL, ApacheAtlasRESTConnector atlasClient, List<Connector> embeddedConnectors)

Constructor for the module is supplied with the runtime context in order to operate.

Method Summary

Modifier and Type	Method	Description
protected void	augmentAtlasDatabaseColumnInEgeria(AtlasEntityWithExtInfo atlasDatabaseColumnEntity, String egeriaDatabaseColumnGUID)	Allow a subclass to attach additional information to the database column.
protected void	augmentAtlasDatabaseInEgeria(AtlasEntityWithExtInfo atlasDatabaseEntity, String egeriaDatabaseGUID)	Allow a subclass to attach additional information to the database.
protected void	augmentAtlasDatabaseTableInEgeria(AtlasEntityWithExtInfo atlasDatabaseTableEntity, String egeriaDatabaseTableGUID)	Allow a subclass to attach additional information to the database table.
protected String	createAtlasDatabaseColumnInEgeria(AtlasEntityWithExtInfo atlasDatabaseColumnEntity, String egeriaDatabaseGUID, String egeriaDatabaseTableGUID)	Create the database in the open metadata ecosystem.
protected String	createAtlasDatabaseInEgeria(AtlasEntityWithExtInfo atlasDatabaseEntity)	Create the database in the open metadata ecosystem.
protected String	createAtlasDatabaseTableInEgeria(AtlasEntityWithExtInfo atlasDatabaseTableEntity, String egeriaDatabaseGUID)	Create the database table in the open metadata ecosystem.
protected abstract RelationalColumnProperties	getEgeriaDatabaseColumnProperties(AtlasEntity atlasEntity, String egeriaSchemaAttributeTypeName)	Map the properties from the entity retrieved from Apache Atlas to the Egeria properties for the open metadata entity.
protected abstract TypeEmbeddedAttributeProperties	getEgeriaDatabaseColumnTypeProperties(AtlasEntity atlasEntity, String egeriaSchemaTypeTypeName)	Map the properties from the entity retrieved from Apache Atlas to the Egeria properties for the open metadata entity.
protected abstract DataAssetProperties	getEgeriaDatabaseProperties(AtlasEntity atlasEntity, String egeriaTypeName)	Map the properties from the entity retrieved from Apache Atlas to the Egeria properties for the open metadata entity.
protected abstract RelationalTableProperties	getEgeriaDatabaseTableProperties(AtlasEntity atlasEntity, String egeriaSchemaAttributeTypeName)	Map the properties from the entity retrieved from Apache Atlas to the Egeria properties for the open metadata entity.
protected TypeEmbeddedAttributeProperties	getEgeriaDatabaseTableTypeProperties()	Map the properties from the entity retrieved from Apache Atlas to the Egeria properties for the open metadata entity.
void	processEvent(OpenMetadataOutTopicEvent event)	Process an event that was published by the Asset Manager OMAS.
void	refresh()	Requests that the connector does a comparison of the metadata in the third party technology and open metadata repositories.
protected void	updateAtlasDatabaseColumnInEgeria(AtlasEntityWithExtInfo atlasDatabaseColumnEntity, String egeriaDatabaseColumnGUID, OpenMetadataRootElement egeriaDatabaseColumn)	Update the properties of an open metadata database with the current properties from Apache Atlas.
protected void	updateAtlasDatabaseInEgeria(AtlasEntityWithExtInfo atlasDatabaseEntity, String egeriaDatabaseGUID)	Update the properties of an open metadata database with the current properties from Apache Atlas.
protected void	updateAtlasDatabaseTableInEgeria(AtlasEntityWithExtInfo atlasDatabaseTableEntity, String egeriaDatabaseTableGUID)	Update the properties of an open metadata database table with the current properties from Apache Atlas.

Methods inherited from class org.odpi.openmetadata.adapters.connectors.apacheatlas.integration.modules.AtlasRegisteredIntegrationModuleBase

getListenForTypes, getModuleName, getSupportedEntityTypes

Methods inherited from class org.odpi.openmetadata.adapters.connectors.apacheatlas.integration.modules.AtlasIntegrationModuleBase

addPropertyMapToAdditionalProperties, addRemainingPropertiesToAdditionalProperties, addStringArrayToAdditionalProperties, atlasUpdateRequired, createAtlasDataSetAsDataSetInEgeria, egeriaUpdateRequired, ensureAtlasExternalIdentifier, getAtlasBooleanProperty, getAtlasGUID, getAtlasGUID, getAtlasPropertyMap, getAtlasStringArray, getAtlasStringProperty, getDataAssetProperties, getEgeriaDataFileProperties, getEgeriaDataSetProperties, getEgeriaGUID, getExternalIdentifier, getExternalIdLink, getProcessProperties, getSchemaAttributeProperties, getSchemaTypeProperties, getValidAtlasGUIDs, isAtlasOwnedElement, isEgeriaOwned, isEgeriaOwned, isEgeriaOwned, removeEgeriaGUID, saveEgeriaGUIDInAtlas, setOwner, syncAtlasDataSetAsDataSet, syncAtlasDataSetsAsDataSets, updateAtlasDataSetAsDataSetInEgeria, updateExternalIdentifierAfterAtlasUpdate

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

DatabaseIntegrationModuleBase

public DatabaseIntegrationModuleBase(String connectorName,
 String moduleName,
 String atlasDatabaseTypeName,
 String atlasDatabaseTablesPropertyName,
 String atlasDatabaseTableTypeName,
 String atlasDatabaseColumnsPropertyName,
 String atlasDatabaseColumnTypeName,
 Connection connectionDetails,
 AuditLog auditLog,
 IntegrationContext myContext,
 String targetRootURL,
 ApacheAtlasRESTConnector atlasClient,
 List<Connector> embeddedConnectors)
                              throws UserNotAuthorizedException

Constructor for the module is supplied with the runtime context in order to operate.

Parameters:

connectorName - name of the connector (for messages)

moduleName - name of this module

atlasDatabaseTypeName - name of type in atlas used for the database

atlasDatabaseTablesPropertyName - name of the property used to navigate from the database to its tables.

atlasDatabaseTableTypeName - name of the type used to represent a database table in atlas

atlasDatabaseColumnsPropertyName - name of the property used to navigate from a database table to its columns

atlasDatabaseColumnTypeName - name of the type used to represent a database column in atlas

connectionDetails - connection properties used to start the connector

auditLog - logging destination

myContext - integration context

targetRootURL - URL to connect to Apache Atlas

atlasClient - client to connect to Apache Atlas

embeddedConnectors - list of any embedded connectors (such as secrets connector and topic connector

Throws:

UserNotAuthorizedException - security problem

Method Details

refresh

public void refresh()
             throws ConnectorCheckedException

Requests that the connector does a comparison of the metadata in the third party technology and open metadata repositories. Refresh is called when the integration connector first starts and then at intervals defined in the connector's configuration as well as any external REST API calls to explicitly refresh the connector.

Throws:

ConnectorCheckedException - there is a problem with the connector. It is not able to refresh the metadata.

processEvent

public void processEvent(OpenMetadataOutTopicEvent event)

Process an event that was published by the Asset Manager OMAS. The listener is only registered if metadata is flowing from the open metadata ecosystem to Apache Atlas.

Parameters:

event - event object

createAtlasDatabaseInEgeria

protected String createAtlasDatabaseInEgeria(AtlasEntityWithExtInfo atlasDatabaseEntity)
                                      throws InvalidParameterException,
UserNotAuthorizedException,
PropertyServerException

Create the database in the open metadata ecosystem.

Parameters:

atlasDatabaseEntity - entity retrieved from Apache Atlas

Returns:

unique identifier of the database entity in open metadata

Throws:

InvalidParameterException - invalid parameter - probably a logic error

UserNotAuthorizedException - security problem

PropertyServerException - unable to communicate with Egeria

updateAtlasDatabaseInEgeria

protected void updateAtlasDatabaseInEgeria(AtlasEntityWithExtInfo atlasDatabaseEntity,
 String egeriaDatabaseGUID)
                                    throws InvalidParameterException,
UserNotAuthorizedException,
PropertyServerException

Update the properties of an open metadata database with the current properties from Apache Atlas.

Parameters:

atlasDatabaseEntity - entity retrieved from Apache Atlas

egeriaDatabaseGUID - unique identifier of the equivalent entity in the open metadata ecosystem

Throws:

InvalidParameterException - invalid parameter - probably a logic error

UserNotAuthorizedException - security problem

PropertyServerException - unable to communicate with Egeria

augmentAtlasDatabaseInEgeria

protected void augmentAtlasDatabaseInEgeria(AtlasEntityWithExtInfo atlasDatabaseEntity,
 String egeriaDatabaseGUID)
                                     throws InvalidParameterException,
UserNotAuthorizedException,
PropertyServerException

Allow a subclass to attach additional information to the database.

Parameters:

atlasDatabaseEntity - entity retrieved from Apache Atlas

egeriaDatabaseGUID - unique identifier of the equivalent entity in the open metadata ecosystem

Throws:

InvalidParameterException - invalid parameter - probably a logic error

UserNotAuthorizedException - security problem

PropertyServerException - unable to communicate with Egeria

createAtlasDatabaseTableInEgeria

protected String createAtlasDatabaseTableInEgeria(AtlasEntityWithExtInfo atlasDatabaseTableEntity,
 String egeriaDatabaseGUID)
                                           throws InvalidParameterException,
UserNotAuthorizedException,
PropertyServerException

Create the database table in the open metadata ecosystem.

Parameters:

atlasDatabaseTableEntity - entity retrieved from Apache Atlas

egeriaDatabaseGUID - unique identifier of the database in the open metadata ecosystem

Returns:

unique identifier of the database entity in open metadata

Throws:

InvalidParameterException - invalid parameter - probably a logic error

UserNotAuthorizedException - security problem

PropertyServerException - unable to communicate with Egeria

updateAtlasDatabaseTableInEgeria

protected void updateAtlasDatabaseTableInEgeria(AtlasEntityWithExtInfo atlasDatabaseTableEntity,
 String egeriaDatabaseTableGUID)
                                         throws InvalidParameterException,
UserNotAuthorizedException,
PropertyServerException

Update the properties of an open metadata database table with the current properties from Apache Atlas.

Parameters:

atlasDatabaseTableEntity - entity retrieved from Apache Atlas

egeriaDatabaseTableGUID - unique identifier of the equivalent entity in the open metadata ecosystem

Throws:

InvalidParameterException - invalid parameter - probably a logic error

UserNotAuthorizedException - security problem

PropertyServerException - unable to communicate with Egeria

augmentAtlasDatabaseTableInEgeria

protected void augmentAtlasDatabaseTableInEgeria(AtlasEntityWithExtInfo atlasDatabaseTableEntity,
 String egeriaDatabaseTableGUID)
                                          throws InvalidParameterException,
UserNotAuthorizedException,
PropertyServerException

Allow a subclass to attach additional information to the database table.

Parameters:

atlasDatabaseTableEntity - entity retrieved from Apache Atlas

egeriaDatabaseTableGUID - unique identifier of the equivalent entity in the open metadata ecosystem

Throws:

InvalidParameterException - invalid parameter - probably a logic error

UserNotAuthorizedException - security problem

PropertyServerException - unable to communicate with Egeria

createAtlasDatabaseColumnInEgeria

protected String createAtlasDatabaseColumnInEgeria(AtlasEntityWithExtInfo atlasDatabaseColumnEntity,
 String egeriaDatabaseGUID,
 String egeriaDatabaseTableGUID)
                                            throws InvalidParameterException,
UserNotAuthorizedException,
PropertyServerException

Create the database in the open metadata ecosystem.

Parameters:

atlasDatabaseColumnEntity - entity retrieved from Apache Atlas

egeriaDatabaseGUID - unique identifier of the database (anchor)

egeriaDatabaseTableGUID - unique identifier of the database table to connect column to

Returns:

unique identifier of the database entity in open metadata

Throws:

InvalidParameterException - invalid parameter - probably a logic error

UserNotAuthorizedException - security problem

PropertyServerException - unable to communicate with Egeria

updateAtlasDatabaseColumnInEgeria

protected void updateAtlasDatabaseColumnInEgeria(AtlasEntityWithExtInfo atlasDatabaseColumnEntity,
 String egeriaDatabaseColumnGUID,
 OpenMetadataRootElement egeriaDatabaseColumn)
                                          throws InvalidParameterException,
UserNotAuthorizedException,
PropertyServerException

Update the properties of an open metadata database with the current properties from Apache Atlas.

Parameters:

atlasDatabaseColumnEntity - entity retrieved from Apache Atlas

egeriaDatabaseColumnGUID - unique identifier of the equivalent entity in the open metadata ecosystem

egeriaDatabaseColumn - retrieved entity from the open metadata ecosystem

Throws:

InvalidParameterException - invalid parameter - probably a logic error

UserNotAuthorizedException - security problem

PropertyServerException - unable to communicate with Egeria

augmentAtlasDatabaseColumnInEgeria

protected void augmentAtlasDatabaseColumnInEgeria(AtlasEntityWithExtInfo atlasDatabaseColumnEntity,
 String egeriaDatabaseColumnGUID)
                                           throws InvalidParameterException,
UserNotAuthorizedException,
PropertyServerException

Allow a subclass to attach additional information to the database column.

Parameters:

atlasDatabaseColumnEntity - entity retrieved from Apache Atlas

egeriaDatabaseColumnGUID - unique identifier of the equivalent entity in the open metadata ecosystem

Throws:

InvalidParameterException - invalid parameter - probably a logic error

UserNotAuthorizedException - security problem

PropertyServerException - unable to communicate with Egeria

getEgeriaDatabaseProperties

protected abstract DataAssetProperties getEgeriaDatabaseProperties(AtlasEntity atlasEntity,
 String egeriaTypeName)

Map the properties from the entity retrieved from Apache Atlas to the Egeria properties for the open metadata entity.

Parameters:

atlasEntity - retrieve entity from Apache Atlas

egeriaTypeName - name of the type used in the open metadata ecosystem

Returns:

properties to pass to Egeria

getEgeriaDatabaseTableProperties

protected abstract RelationalTableProperties getEgeriaDatabaseTableProperties(AtlasEntity atlasEntity,
 String egeriaSchemaAttributeTypeName)

Map the properties from the entity retrieved from Apache Atlas to the Egeria properties for the open metadata entity.

Parameters:

atlasEntity - retrieve entity from Apache Atlas

egeriaSchemaAttributeTypeName - name of the type used in the open metadata ecosystem

Returns:

properties to pass to Egeria

getEgeriaDatabaseTableTypeProperties

protected TypeEmbeddedAttributeProperties getEgeriaDatabaseTableTypeProperties()

Map the properties from the entity retrieved from Apache Atlas to the Egeria properties for the open metadata entity.

Returns:

properties to pass to Egeria

getEgeriaDatabaseColumnProperties

protected abstract RelationalColumnProperties getEgeriaDatabaseColumnProperties(AtlasEntity atlasEntity,
 String egeriaSchemaAttributeTypeName)

Map the properties from the entity retrieved from Apache Atlas to the Egeria properties for the open metadata entity.

Parameters:

atlasEntity - retrieve entity from Apache Atlas

egeriaSchemaAttributeTypeName - name of the type used in the open metadata ecosystem

Returns:

properties to pass to Egeria

getEgeriaDatabaseColumnTypeProperties

protected abstract TypeEmbeddedAttributeProperties getEgeriaDatabaseColumnTypeProperties(AtlasEntity atlasEntity,
 String egeriaSchemaTypeTypeName)

Map the properties from the entity retrieved from Apache Atlas to the Egeria properties for the open metadata entity.

Parameters:

atlasEntity - retrieve entity from Apache Atlas

egeriaSchemaTypeTypeName - name of the type used in the open metadata ecosystem

Returns:

properties to pass to Egeria