Package org.odpi.openmetadata.adapters.connectors.integration.dcat

Class DcatFilesReceiverIntegrationConnector

java.lang.Object

org.odpi.openmetadata.frameworks.connectors.Connector

org.odpi.openmetadata.frameworks.connectors.ConnectorBase

org.odpi.openmetadata.frameworks.integration.connectors.IntegrationConnectorBase

org.odpi.openmetadata.frameworks.integration.connectors.DynamicIntegrationConnectorBase

org.odpi.openmetadata.adapters.connectors.integration.dcat.DcatFilesReceiverIntegrationConnector

All Implemented Interfaces:

AuditLoggingComponent, SecureConnectorExtension, VirtualConnectorExtension, CatalogTargetFactory, IntegrationConnector

public class DcatFilesReceiverIntegrationConnector
extends DynamicIntegrationConnectorBase

DcatFilesReceiverIntegrationConnector monitors file directories for Data Catalog Vocabulary (DCAT) documents. The directories are typically drop folders, or the downloads from a data portal, where the descriptions of catalogs, datasets and data services are placed. A directory is identified either by the network address of the connection's endpoint, or by a FileFolder catalog target. On each refresh, the connector scans each directory tree for JSON-LD files (.jsonld and .json) that describe DCAT resources and publishes the new and changed documents to the DCAT listeners registered in the same integration daemon (for example the cataloguer connector).

Nested Class Summary

Nested classes/interfaces inherited from class org.odpi.openmetadata.frameworks.connectors.ConnectorBase

ConnectorBase.ProtectedConnection

Field Summary

Fields inherited from class org.odpi.openmetadata.frameworks.integration.connectors.IntegrationConnectorBase

catalogTargetsManager, connectorName, integrationContext, propertyHelper

Fields inherited from class org.odpi.openmetadata.frameworks.connectors.ConnectorBase

auditLog, connectionBean, connectorInstanceId, embeddedConnectors, messageFormatter, secretsStoreConnectorMap

Constructor Summary

Constructors

Constructor	Description
DcatFilesReceiverIntegrationConnector()	Default constructor

Method Summary

Modifier and Type	Method	Description
RequestedCatalogTarget	getNewRequestedCatalogTargetSkeleton(CatalogTarget retrievedCatalogTarget, CatalogTargetContext catalogTargetContext, Connector connectorToTarget)	Create a new catalog target processor (typically inherits from CatalogTargetProcessorBase).
void	refresh()	Requests that the connector does a comparison of the metadata in the third party technology and open metadata repositories.
void	start()	Indicates that the connector is completely configured and can begin processing.

Methods inherited from class org.odpi.openmetadata.frameworks.integration.connectors.DynamicIntegrationConnectorBase

processEvent

Methods inherited from class org.odpi.openmetadata.frameworks.integration.connectors.IntegrationConnectorBase

disconnect, engage, getConnectorComponentDescription, getNetworkAddress, passEventToCatalogTargets, refreshCatalogTargets, setAuditLog, setConnectorName, setContext

Methods inherited from class org.odpi.openmetadata.frameworks.connectors.ConnectorBase

combineConfigurationProperties, disconnectConnectors, equals, fromCamelToCanonicalCase, fromCanonicalToCamelCase, fromCanonicalToKebabCase, fromCanonicalToSnakeCase, fromKebabToCanonicalCase, fromSnakeToCanonicalCase, getArrayConfigurationProperty, getArrayConfigurationProperty, getArrayConfigurationProperty, getArrayValue, getBooleanConfigurationProperty, getConnection, getConnectorInstanceId, getDateConfigurationProperty, getIntConfigurationProperty, getLongConfigurationProperty, getSecretsCollectionName, getSecretsLocation, getSecretsProvider, getStringConfigurationProperty, getStringConfigurationProperty, getSuppliedPlaceholderProperties, hashCode, initialize, initializeEmbeddedConnectors, initializeSecretsStoreConnector, isActive, logExceptionRecord, logExceptionRecord, logRecord, logRecord, throwMissingConfigurationProperty, throwMissingElement, throwMissingEndpointAddress, throwMissingResource, throwNoAsset, throwWrongTypeOfAsset, throwWrongTypeOfResource, throwWrongTypeOfRootSchema, toString

Methods inherited from class org.odpi.openmetadata.frameworks.connectors.Connector

clearStatisticProperty, clearStatisticTimestamp, getConnectorStatistics, getStatisticCounter, getStatisticProperty, getStatisticTimestamp, incrementStatisticCounter, initializeStatisticCounter, refreshStatistics, setStatisticProperty, setStatisticTimestamp

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.odpi.openmetadata.frameworks.connectors.VirtualConnectorExtension

initializeEmbeddedConnectors

Constructor Details

DcatFilesReceiverIntegrationConnector

public DcatFilesReceiverIntegrationConnector()

Default constructor

Method Details

start

public void start()
           throws ConnectorCheckedException,
UserNotAuthorizedException

Indicates that the connector is completely configured and can begin processing. This picks up the optional directory named in the connection's endpoint.

Specified by:

start in interface IntegrationConnector

Overrides:

start in class IntegrationConnectorBase

Throws:

ConnectorCheckedException - there is a problem within the connector.

UserNotAuthorizedException - the connector was disconnected before/during start

refresh

public void refresh()
             throws ConnectorCheckedException,
UserNotAuthorizedException

Requests that the connector does a comparison of the metadata in the third party technology and open metadata repositories. The endpoint's directory is scanned and then the catalog targets are refreshed by the dynamic base class, which calls refresh() on each catalog target processor.

Specified by:

refresh in interface IntegrationConnector

Overrides:

refresh in class DynamicIntegrationConnectorBase

Throws:

ConnectorCheckedException - there is a problem with the connector. It is not able to refresh the metadata.

UserNotAuthorizedException - the connector was disconnected before/during start

getNewRequestedCatalogTargetSkeleton

public RequestedCatalogTarget getNewRequestedCatalogTargetSkeleton(CatalogTarget retrievedCatalogTarget,
 CatalogTargetContext catalogTargetContext,
 Connector connectorToTarget)
                                                            throws ConnectorCheckedException,
UserNotAuthorizedException

Create a new catalog target processor (typically inherits from CatalogTargetProcessorBase). FileFolder catalog targets are scanned for DCAT documents; other catalog targets are ignored.

Specified by:

getNewRequestedCatalogTargetSkeleton in interface CatalogTargetFactory

Specified by:

getNewRequestedCatalogTargetSkeleton in class DynamicIntegrationConnectorBase

Parameters:

retrievedCatalogTarget - the catalog target information retrieved from the open metadata repository

catalogTargetContext - specialized integration context for this catalog target

connectorToTarget - connector to access the target resource

Returns:

new processor based on the catalog target information

Throws:

ConnectorCheckedException - problem setting up the processor

UserNotAuthorizedException - the connector was disconnected before/during start