Package org.odpi.openmetadata.adapters.connectors.apacheatlas.integration.modules

Class AtlasIntegrationModuleBase

java.lang.Object
org.odpi.openmetadata.adapters.connectors.apacheatlas.integration.modules.AtlasIntegrationModuleBase
Direct Known Subclasses:
AtlasInformalTagsIntegrationModule, AtlasLineageIntegrationModule, AtlasReferenceClassificationsIntegrationModule, AtlasRegisteredIntegrationModuleBase, AtlasRelatedElementsIntegrationModule
public abstract class AtlasIntegrationModuleBase extends Object
AtlasIntegrationModuleBase defines the interface that classes that support the synchronization of particular types of metadata with Apache Atlas.

  • Field Details

  • Constructor Details

    • AtlasIntegrationModuleBase

      public AtlasIntegrationModuleBase(String connectorName, String moduleName, ConnectionProperties connectionProperties, AuditLog auditLog, CatalogIntegratorContext myContext, String targetRootURL, ApacheAtlasRESTConnector atlasClient, List<Connector> embeddedConnectors) throws UserNotAuthorizedException
      Initialize the common properties for an integration module.
      Parameters:
      connectorName - name of this connector
      moduleName - name of this module
      connectionProperties - supplied connector used to configure the connector
      auditLog - logging destination
      myContext - integration context assigned to the connector
      targetRootURL - host name and port of Apache Atlas
      atlasClient - client to call Apache Atlas
      embeddedConnectors - any connectors embedded in the connector (such as the secrets connector or Kafka connector)
      Throws:
      UserNotAuthorizedException - the data asset service has not been enabled.

  • Method Details

    • isAtlasOwnedElement

      protected boolean isAtlasOwnedElement(ElementHeader elementHeader)
      Return a flag indicating whether an element retrieved from the open metadata ecosystem.
      Parameters:
      elementHeader - header from open metadata ecosystem
      Returns:
      boolean flag

    • isEgeriaOwned

      protected boolean isEgeriaOwned(AtlasGlossaryBaseProperties properties)
      Return a flag to indicate whether an atlas glossary element is owned by Egeria (open metadata ecosystem) or is owned by Apache Atlas. This becomes important around delete scenarios.
      Parameters:
      properties - properties of the Atlas element
      Returns:
      boolean - true means the element originated in the open metadata ecosystem

    • isEgeriaOwned

      protected boolean isEgeriaOwned(AtlasEntityWithExtInfo atlasEntityWithExtInfo) throws PropertyServerException
      Return a flag to indicate whether an atlas glossary element is owned by Egeria (open metadata ecosystem) or is owned by Apache Atlas. This becomes important around delete scenarios.
      Parameters:
      atlasEntityWithExtInfo - properties of the Atlas element
      Returns:
      boolean - true means the element originated in the open metadata ecosystem
      Throws:
      PropertyServerException - problem communicating with Apache Atlas

    • isEgeriaOwned

      protected boolean isEgeriaOwned(AtlasEntity atlasEntity) throws PropertyServerException
      Return a flag to indicate whether an atlas glossary element is owned by Egeria (open metadata ecosystem) or is owned by Apache Atlas. This becomes important around delete scenarios.
      Parameters:
      atlasEntity - properties of the Atlas element
      Returns:
      boolean - true means the element originated in the open metadata ecosystem
      Throws:
      PropertyServerException - problem communicating with Apache Atlas

    • saveEgeriaGUIDInAtlas

      protected void saveEgeriaGUIDInAtlas(String atlasEntityGUID, String egeriaEntityGUID, String egeriaQualifiedName, String egeriaTypeName, boolean isAtlasGlossaryEntity, boolean egeriaOwned) throws PropertyServerException
      Add a new OpenMetadataCorrelation entity to apache atlas and link it to the supplied atlas entity.
      Parameters:
      atlasEntityGUID - unique identifier of the atlas element that the open metadata correlation entity should be connected to
      egeriaEntityGUID - unique identifier from the open metadata ecosystem
      egeriaQualifiedName - unique name from the open metadata ecosystem
      egeriaTypeName - name of type from the open metadata ecosystem
      isAtlasGlossaryEntity - is this a glossary entity?
      egeriaOwned - is this element owned by Egeria?
      Throws:
      PropertyServerException - problems communicating with Apache Atlas.

    • getAtlasGUID

      protected String getAtlasGUID(MetadataElement metadataElement) throws InvalidParameterException
      Return the unique identifier for the equivalent element in Apache Atlas.
      Parameters:
      metadataElement - retrieved metadata element
      Returns:
      string guid
      Throws:
      InvalidParameterException - retrieved element is null

    • egeriaUpdateRequired

      protected boolean egeriaUpdateRequired(String egeriaGUID, String egeriaTypeName, MetadataElement egeriaMetadataElement, AtlasEntity atlasEntity) throws InvalidParameterException, UserNotAuthorizedException, PropertyServerException
      Confirm whether there have been changes since the last refresh().
      Parameters:
      egeriaGUID - unique identifier (GUID) of this element in open metadata
      egeriaTypeName - type name for the open metadata element
      egeriaMetadataElement - list of external identifiers used to manage metadata exchange
      atlasEntity - atlas entity that is the source of the metadata
      Throws:
      InvalidParameterException - one of the parameters is invalid
      UserNotAuthorizedException - user not authorized to issue this request
      PropertyServerException - problem accessing the property server

    • atlasUpdateRequired

      protected boolean atlasUpdateRequired(String egeriaGUID, String egeriaTypeName, MetadataElement egeriaMetadataElement, AtlasEntity atlasEntity) throws InvalidParameterException, PropertyServerException
      Confirm whether there have been changes since the last synchronization.
      Parameters:
      egeriaGUID - unique identifier (GUID) of this element in open metadata
      egeriaTypeName - type name for the open metadata element
      egeriaMetadataElement - list of external identifiers used to manage metadata exchange
      atlasEntity - atlas entity that is the source of the metadata
      Throws:
      InvalidParameterException - one of the parameters is invalid
      PropertyServerException - problem accessing the property server

    • updateExternalIdentifierAfterAtlasUpdate

      protected void updateExternalIdentifierAfterAtlasUpdate(String egeriaGUID, String egeriaTypeName, AtlasEntity atlasEntity) throws InvalidParameterException, UserNotAuthorizedException, PropertyServerException
      Update the correlation properties for an Egeria element after an atlas entity has been updated.
      Parameters:
      egeriaGUID - unique identifier (GUID) of this element in open metadata
      egeriaTypeName - type name for the open metadata element
      atlasEntity - atlas entity that is the source of the metadata
      Throws:
      InvalidParameterException - one of the parameters is invalid
      UserNotAuthorizedException - user not authorized to issue this request
      PropertyServerException - problem accessing the property server

    • getEgeriaGUID

      protected String getEgeriaGUID(AtlasEntityWithExtInfo atlasEntity) throws PropertyServerException
      Extract the unique identifier for the corresponding open metadata entity to the supplied Apache Atlas entity. A null value returned means that an open metadata ecosystem entity has not yet been created.
      Parameters:
      atlasEntity - entity retrieved from Apache Atlas
      Returns:
      string guid
      Throws:
      PropertyServerException - there is a problem calling Apache Atlas

    • removeEgeriaGUID

      protected void removeEgeriaGUID(AtlasEntityWithExtInfo atlasEntity) throws PropertyServerException
      Remove the requested entity.
      Parameters:
      atlasEntity - entity retrieved from Apache Atlas
      Throws:
      PropertyServerException - there is a problem calling Apache Atlas

    • getExternalIdentifier

      protected ExternalIdentifierProperties getExternalIdentifier(String atlasGUID, String atlasTypeName, String externalInstanceCreatedBy, Date externalInstanceCreationTime, String externalInstanceLastUpdatedBy, Date externalInstanceLastUpdateTime, long externalInstanceVersion, SynchronizationDirection synchronizationDirection)
      Return an external identifier properties object for an Atlas GUID.
      Parameters:
      atlasGUID - guid to encode
      atlasTypeName - name of the atlas type
      externalInstanceCreatedBy - the username of the person or process that created the instance in the external system
      externalInstanceCreationTime - the date/time when the instance in the external system was created
      externalInstanceLastUpdatedBy - the username of the person or process that last updated the instance in the external system
      externalInstanceLastUpdateTime - the date/time that the instance in the external system was last updated
      externalInstanceVersion - the latest version of the element in the external system
      synchronizationDirection - the permitted synchronization direction
      Returns:
      properties object

    • getExternalIdentifier

      protected ExternalIdentifierProperties getExternalIdentifier(String atlasGUID, String atlasTypeName, String atlasName, String lastKnownEgeriaDisplayName, String externalInstanceCreatedBy, Date externalInstanceCreationTime, String externalInstanceLastUpdatedBy, Date externalInstanceLastUpdateTime, long externalInstanceVersion, SynchronizationDirection synchronizationDirection)
      Return an external identifier properties object for an Atlas GUID. This creates additional mapping properties for glossary terms and categories.
      Parameters:
      atlasGUID - guid to encode
      atlasTypeName - name of the atlas type
      externalInstanceCreatedBy - the username of the person or process that created the instance in the external system
      externalInstanceCreationTime - the date/time when the instance in the external system was created
      externalInstanceLastUpdatedBy - the username of the person or process that last updated the instance in the external system
      externalInstanceLastUpdateTime - the date/time that the instance in the external system was last updated
      externalInstanceVersion - the latest version of the element in the external system
      synchronizationDirection - the permitted synchronization direction
      Returns:
      properties object

    • ensureAtlasExternalIdentifier

      protected void ensureAtlasExternalIdentifier(MetadataElement egeriaElement, String egeriaDisplayName, AtlasEntity atlasEntity, String atlasGUID, String atlasTypeName, String atlasName) throws InvalidParameterException, UserNotAuthorizedException, PropertyServerException
      Check that the Atlas GUID is correct stored in the open metadata element.
      Parameters:
      egeriaElement - element to validate and potentially update
      egeriaDisplayName - associated display name - used for glossary members
      atlasEntity - equivalent entity from Apache Atlas
      atlasGUID - Unique identifier from Apache Atlas
      atlasTypeName - name of the atlas type
      atlasName - atlas name property
      Throws:
      InvalidParameterException - invalid parameter - probably a logic error
      UserNotAuthorizedException - security problem
      PropertyServerException - problem connecting with Egeria

    • getValidAtlasGUIDs

      protected List<String> getValidAtlasGUIDs(List<AtlasEntityWithExtInfo> atlasEntities)
      Return the list of unique identifiers from a list of entities from Apache Atlas.
      Parameters:
      atlasEntities - list of entities retrieved from Apache Atlas
      Returns:
      list of guids (or empty list but not null)

    • getAtlasStringProperty

      protected String getAtlasStringProperty(Map<String,Object> attributes, String propertyName)
      Retrieve a string property from and Apache Atlas entity's attributes.
      Parameters:
      attributes - attribute map
      propertyName - name of property to extract
      Returns:
      extracted string or null

    • getAtlasBooleanProperty

      protected boolean getAtlasBooleanProperty(Map<String,Object> attributes, String propertyName)
      Retrieve a boolean property from and Apache Atlas entity's attributes.
      Parameters:
      attributes - attribute map
      propertyName - name of property to extract
      Returns:
      extracted boolean or null

    • getAtlasPropertyMap

      protected Map<String,String> getAtlasPropertyMap(Map<String,Object> attributes, String propertyName)
      Retrieve a string property map from and Apache Atlas entity's attributes.
      Parameters:
      attributes - attribute map
      propertyName - name of property to extract
      Returns:
      extracted string or null

    • getAtlasStringArray

      protected List<String> getAtlasStringArray(Map<String,Object> attributes, String propertyName)
      Retrieve a string array property from and Apache Atlas entity's attributes.
      Parameters:
      attributes - attribute map
      propertyName - name of property to extract
      Returns:
      extracted string or null

    • addRemainingPropertiesToAdditionalProperties

      protected Map<String,String> addRemainingPropertiesToAdditionalProperties(Map<String,Object> atlasAttributes, List<String> skipPropertyNames)
      Add a property from Apache Atlas to Egeria's additionalProperties map since it has nowhere else to go.
      Parameters:
      atlasAttributes - all the attributes retrieved from Apache Atlas
      skipPropertyNames - names of the properties that have already been extracted
      Returns:
      additionalProperties populated with properties from Apache Atlas

    • addStringArrayToAdditionalProperties

      protected void addStringArrayToAdditionalProperties(Map<String,Object> atlasAttributes, String propertyName, Map<String,String> egeriaAdditionalProperties)
      Add a property from Apache Atlas to Egeria's additionalProperties map since it has nowhere else to go. Since the property is a map of properties, each entry is added as separate property using a dotted notation mapPropertyName.mapPropertyKey.mapPropertyValue.
      Parameters:
      atlasAttributes - all the attributes retrieved from Apache Atlas
      propertyName - name of the property to extract
      egeriaAdditionalProperties - additionalProperties to map to populate

    • addPropertyMapToAdditionalProperties

      protected void addPropertyMapToAdditionalProperties(Map<String,Object> atlasAttributes, String propertyName, Map<String,String> egeriaAdditionalProperties)
      Add a property from Apache Atlas to Egeria's additionalProperties map since it has nowhere else to go. Since the property is a map of properties, each entry is added as separate property using a dotted notation mapPropertyName.mapPropertyKey.mapPropertyValue.
      Parameters:
      atlasAttributes - all the attributes retrieved from Apache Atlas
      propertyName - name of the property to extract
      egeriaAdditionalProperties - additionalProperties to map to populate

    • syncAtlasDataSetsAsDataSets

      protected void syncAtlasDataSetsAsDataSets(String atlasTypeName, String egeriaTypeName) throws InvalidParameterException, UserNotAuthorizedException, PropertyServerException
      Synchronize all the data set entities from Apache Atlas to the open metadata ecosystem.
      Parameters:
      atlasTypeName - name of the type of entity to synchronize from Apache Atlas
      egeriaTypeName - name of the type of entity to synchronize to the open metadata ecosystem
      Throws:
      InvalidParameterException - invalid parameter - probably a logic error
      UserNotAuthorizedException - security problem
      PropertyServerException - unable to communicate with Egeria

    • syncAtlasDataSetAsDataSet

      protected String syncAtlasDataSetAsDataSet(AtlasEntityWithExtInfo atlasDataSetEntity, String atlasTypeName, String egeriaTypeName) throws InvalidParameterException, UserNotAuthorizedException, PropertyServerException
      Copy the contents of the Atlas data set entity into open metadata as a type of DataSet.
      Parameters:
      atlasDataSetEntity - entity retrieved from Apache Atlas
      atlasTypeName - name of the type of entity in Apache Atlas to synchronize
      egeriaTypeName - name of the type of entity in the open metadata ecosystem to synchronize
      Returns:
      unique identifier of the data set
      Throws:
      InvalidParameterException - invalid parameter - probably a logic error
      UserNotAuthorizedException - security problem
      PropertyServerException - unable to communicate with Egeria

    • createAtlasDataSetAsDataSetInEgeria

      protected String createAtlasDataSetAsDataSetInEgeria(AtlasEntityWithExtInfo atlasDataSetEntity, String atlasTypeName, String egeriaTypeName) throws InvalidParameterException, UserNotAuthorizedException, PropertyServerException
      Create the data set in the open metadata ecosystem.
      Parameters:
      atlasDataSetEntity - entity retrieved from Apache Atlas
      atlasTypeName - name of the type of entity to synchronize from Apache Atlas
      egeriaTypeName - name of the type of entity to synchronize to the open metadata ecosystem
      Returns:
      unique identifier of the data set
      Throws:
      InvalidParameterException - invalid parameter - probably a logic error
      UserNotAuthorizedException - security problem
      PropertyServerException - unable to communicate with Egeria

    • updateAtlasDataSetAsDataSetInEgeria

      protected void updateAtlasDataSetAsDataSetInEgeria(AtlasEntityWithExtInfo atlasDataSetEntity, String egeriaDataSetGUID, String atlasTypeName, String egeriaTypeName) throws InvalidParameterException, UserNotAuthorizedException, PropertyServerException
      Update the properties of an open metadata data set with the current properties from Apache Atlas.
      Parameters:
      atlasDataSetEntity - entity retrieved from Apache Atlas
      egeriaDataSetGUID - unique identifier of the equivalent entity in the open metadata ecosystem
      atlasTypeName - name of the type of entity to synchronize from Apache Atlas
      egeriaTypeName - name of the type of entity to synchronize to the open metadata ecosystem
      Throws:
      InvalidParameterException - invalid parameter - probably a logic error
      UserNotAuthorizedException - security problem
      PropertyServerException - unable to communicate with Egeria

    • getEgeriaDataSetProperties

      protected DataAssetProperties getEgeriaDataSetProperties(AtlasEntity atlasEntity, String egeriaTypeName)
      Map the properties from the entity retrieved from Apache Atlas to the Egeria properties for the open metadata entity.
      Parameters:
      atlasEntity - retrieve entity from Apache Atlas
      egeriaTypeName - name of the type used in the open metadata ecosystem
      Returns:
      properties to pass to Egeria

    • getDataAssetProperties

      protected DataAssetProperties getDataAssetProperties(AtlasEntity atlasEntity, String egeriaTypeName)
      Add the standard properties for an Atlas asset entity into an Egeria DataAssetProperties. This is suitable for a DataSet.
      Parameters:
      atlasEntity - asset entity retrieved from Apache Atlas
      egeriaTypeName - name of the type used in the open metadata ecosystem
      Returns:
      data asset properties used to maintaining data assets in the open metadata ecosystem

    • getEgeriaDataFileProperties

      protected DataAssetProperties getEgeriaDataFileProperties(AtlasEntity atlasFSPathEntity, String egeriaTypeName)
      Map the properties from the entity retrieved from Apache Atlas to the Egeria properties for the open metadata entity.
      Parameters:
      atlasFSPathEntity - retrieve fs_path entity from Apache Atlas
      egeriaTypeName - name of the type used in the open metadata ecosystem
      Returns:
      properties to pass to Egeria

    • getProcessProperties

      protected ProcessProperties getProcessProperties(AtlasEntity atlasEntity, String egeriaTypeName)
      Add the standard properties for an Atlas asset entity into an Egeria DataAssetProperties. This is suitable for a DataSet.
      Parameters:
      atlasEntity - asset entity retrieved from Apache Atlas
      egeriaTypeName - name of the type used in the open metadata ecosystem
      Returns:
      data asset properties used to maintaining data assets in the open metadata ecosystem

    • getSchemaAttributeProperties

      protected SchemaAttributeProperties getSchemaAttributeProperties(AtlasEntity atlasEntity, String egeriaSchemaAttributeTypeName, String egeriaSchemaTypeTypeName)
      Add the standard properties for an Atlas asset entity into an Egeria SchemaAttributeProperties.
      Parameters:
      atlasEntity - asset entity retrieved from Apache Atlas
      egeriaSchemaAttributeTypeName - type name for the schema attribute
      egeriaSchemaTypeTypeName - type name for the schema type
      Returns:
      data asset properties used to maintaining data assets in the open metadata ecosystem

    • setOwner

      protected void setOwner(AtlasEntityWithExtInfo atlasEntityWithExtInfo, String egeriaGUID) throws InvalidParameterException, UserNotAuthorizedException, PropertyServerException
      Set up the Ownership classification in open metadata if the Atlas entity has the owner property set.
      Parameters:
      atlasEntityWithExtInfo - entity retrieved from Apache Atlas
      egeriaGUID - unique identifier of the element in open metadata
      Throws:
      InvalidParameterException - invalid parameter - probably a logic error
      UserNotAuthorizedException - security problem
      PropertyServerException - problem communicating with Egeria.