Package org.odpi.openmetadata.adapters.connectors.apachekafka.integration

Class KafkaTopicIntegrationConnector

java.lang.Object

org.odpi.openmetadata.frameworks.connectors.Connector

org.odpi.openmetadata.frameworks.connectors.ConnectorBase

org.odpi.openmetadata.frameworks.integration.connectors.IntegrationConnectorBase

org.odpi.openmetadata.integrationservices.topic.connector.TopicIntegratorConnector

org.odpi.openmetadata.adapters.connectors.apachekafka.integration.KafkaTopicIntegrationConnector

All Implemented Interfaces:

AuditLoggingComponent, SecureConnectorExtension, VirtualConnectorExtension, CatalogTargetIntegrator, IntegrationConnector, TopicIntegratorOMISConnector

public class KafkaTopicIntegrationConnector
extends TopicIntegratorConnector
implements CatalogTargetIntegrator

KafkaTopicIntegrationConnector catalogues active topics in a kafka broker.

Nested Class Summary

Nested classes/interfaces inherited from class org.odpi.openmetadata.frameworks.connectors.ConnectorBase

ConnectorBase.ProtectedConnection

Field Summary

Fields inherited from class org.odpi.openmetadata.frameworks.integration.connectors.IntegrationConnectorBase

auditLog, catalogTargetsManager, connectorName, integrationContext

Fields inherited from class org.odpi.openmetadata.frameworks.connectors.ConnectorBase

connectedAssetProperties, connectionBean, connectionProperties, connectorInstanceId, embeddedConnectors, messageFormatter, secretsStoreConnectorMap

Constructor Summary

Constructors

Constructor	Description
KafkaTopicIntegrationConnector()

Method Summary

Modifier and Type	Method	Description
void	disconnect()	Shutdown kafka monitoring
void	integrateCatalogTarget(RequestedCatalogTarget requestedCatalogTarget)	Perform the required integration logic for the assigned catalog target.
void	refresh()	Requests that the connector does a comparison of the metadata in the third party technology and open metadata repositories.
void	refreshEndpointTarget(ApacheKafkaAdminConnector apacheKafkaAdminConnector)	Processes any event broker configured in this connector's connection.
void	refreshEventBroker(String targetRootURL, String templateGUID, String templateQualifiedName)	Requests that the connector does a comparison of the metadata in the third party technology and open metadata repositories.

Methods inherited from class org.odpi.openmetadata.integrationservices.topic.connector.TopicIntegratorConnector

getContext, setContext

Methods inherited from class org.odpi.openmetadata.frameworks.integration.connectors.IntegrationConnectorBase

engage, getConnectorComponentDescription, getNetworkAddress, refreshCatalogTargets, registerCatalogTargetChangeListener, setAuditLog, setConnectorName, setContext, start

Methods inherited from class org.odpi.openmetadata.frameworks.connectors.ConnectorBase

combineConfigurationProperties, disconnectConnectors, equals, getArrayConfigurationProperty, getAssetTypeName, getBooleanConfigurationProperty, getConnectedAssetProperties, getConnection, getConnectorInstanceId, getDateConfigurationProperty, getIntConfigurationProperty, getLongConfigurationProperty, getNetworkAddresses, getStringConfigurationProperty, getSuppliedPlaceholderProperties, hashCode, initialize, initializeConnectedAssetProperties, initializeEmbeddedConnectors, initializeSecretsStoreConnector, isActive, logExceptionRecord, logRecord, logRecord, throwMissingConfigurationProperty, throwMissingResource, throwNoAsset, throwWrongTypeOfAsset, throwWrongTypeOfResource, throwWrongTypeOfRootSchema, toString

Methods inherited from class org.odpi.openmetadata.frameworks.connectors.Connector

clearStatisticProperty, clearStatisticTimestamp, getConnectorStatistics, getStatisticCounter, getStatisticProperty, getStatisticTimestamp, incrementStatisticCounter, initializeStatisticCounter, setStatisticProperty, setStatisticTimestamp

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.odpi.openmetadata.frameworks.integration.connectors.CatalogTargetIntegrator

getNewRequestedCatalogTargetSkeleton

Methods inherited from interface org.odpi.openmetadata.frameworks.connectors.VirtualConnectorExtension

initializeEmbeddedConnectors

Constructor Details

KafkaTopicIntegrationConnector

public KafkaTopicIntegrationConnector()

Method Details

refresh

public void refresh()
             throws ConnectorCheckedException

Requests that the connector does a comparison of the metadata in the third party technology and open metadata repositories. Refresh is called when the integration connector first starts and then at intervals defined in the connector's configuration as well as any external REST API calls to explicitly refresh the connector. This method performs two sweeps. It first retrieves the topics from the event broker (Kafka) and validates that are in the catalog - adding or updating them if necessary. The second sweep is to ensure that all the topics catalogued actually exist in the event broker.

Specified by:

refresh in interface IntegrationConnector

Throws:

ConnectorCheckedException - there is a problem with the connector. It is not able to refresh the metadata.

refreshEndpointTarget

public void refreshEndpointTarget(ApacheKafkaAdminConnector apacheKafkaAdminConnector)
                           throws ConnectorCheckedException

Processes any event broker configured in this connector's connection.

Throws:

ConnectorCheckedException - there is a problem within the connector.

integrateCatalogTarget

public void integrateCatalogTarget(RequestedCatalogTarget requestedCatalogTarget)
                            throws ConnectorCheckedException

Perform the required integration logic for the assigned catalog target.

Specified by:

integrateCatalogTarget in interface CatalogTargetIntegrator

Parameters:

requestedCatalogTarget - the catalog target

Throws:

ConnectorCheckedException - there is an unrecoverable error and the connector should stop processing.

refreshEventBroker

public void refreshEventBroker(String targetRootURL,
 String templateGUID,
 String templateQualifiedName)
                        throws ConnectorCheckedException

Requests that the connector does a comparison of the metadata in the third party technology and open metadata repositories. Refresh is called when the integration connector first starts and then at intervals defined in the connector's configuration as well as any external REST API calls to explicitly refresh the connector. This method performs two sweeps. It first retrieves the topics from the event broker (Kafka) and validates that are in the catalog - adding or updating them if necessary. The second sweep is to ensure that all the topics catalogued actually exist in the event broker.

Parameters:

targetRootURL - URL to the Kafka Broker

templateGUID - optional template to use when creating new topics

templateQualifiedName - qualifiedName for template - only set if templateGUID is set

Throws:

ConnectorCheckedException - there is a problem with the connector. It is not able to refresh the metadata.

disconnect

public void disconnect()
                throws ConnectorCheckedException

Shutdown kafka monitoring

Specified by:

disconnect in interface IntegrationConnector

Overrides:

disconnect in class IntegrationConnectorBase

Throws:

ConnectorCheckedException - something failed in the super class