Package org.odpi.openmetadata.adapters.connectors.duckdb.catalog

Class DuckDBDatabaseCatalogTargetProcessor

java.lang.Object

org.odpi.openmetadata.frameworks.openmetadata.properties.RelationshipProperties

org.odpi.openmetadata.frameworks.openmetadata.properties.RelationshipBeanProperties

org.odpi.openmetadata.frameworks.openmetadata.properties.assets.processes.connectors.CatalogTargetProperties

org.odpi.openmetadata.frameworks.opengovernance.properties.CatalogTarget

org.odpi.openmetadata.frameworks.integration.properties.RequestedCatalogTarget

org.odpi.openmetadata.frameworks.integration.connectors.CatalogTargetProcessorBase

org.odpi.openmetadata.adapters.connectors.duckdb.catalog.DuckDBDatabaseCatalogTargetProcessor

public class DuckDBDatabaseCatalogTargetProcessor
extends CatalogTargetProcessorBase

DuckDBDatabaseCatalogTargetProcessor is the most novel of the DuckDB connectors. A DuckDB database is embedded (a single ".duckdb" file, or an in-memory ":memory:" session) so there is no equivalent of the "server" catalog target used by the other database connector suites for the connector to be pointed at - the catalog target configured for this processor may be any convenient, already-catalogued element (for example, a FileFolder for the directory the database file lives in, or a previous run's own DuckDB database asset); the actual identity of the DuckDB database is always (re)computed from the DuckDBConfigurationProperty.DATABASE_PATH configuration property using the files-integration-connector's FileClassifier, exactly the way DataFilesMonitorForTarget.catalogFile() computes the identity of a data file.
DESIGN NOTE (judgement call): CatalogTargetProcessorBase's contract assumes the catalog target element already exists, but the whole point of this processor is that it can materialise the RelationalDatabase asset for a DuckDB file the first time it is refreshed. This processor therefore always looks the database asset up by its FileClassifier-derived qualifiedName first (which will typically find the catalog target element itself, once it has been catalogued once) and only creates a brand-new self-anchored asset if no match is found - see getOrCreateDuckDBDatabaseAsset(String, Map).
Once the database asset's identity is settled, this processor does two more things on every refresh:

1. hands off cataloguing of the database's native schemas/tables/columns to the "friendship" JDBC integration connector, using exactly the same catalog target hand-off mechanism as the other four vendor connector suites;
2. catalogues DuckDB's federation relationships (databases ATTACH-ed to this one, and views that scan external files/object-store resources) - see catalogFederationRelationships(String, String, Connection). This reuses DuckDBFederationExtractor's discovery queries (its SurveyActionServiceConnector constructor parameter is unused by its query logic, so null is passed here) rather than duplicating the SQL/regex matching a second time.

Field Summary

Fields inherited from class org.odpi.openmetadata.frameworks.integration.connectors.CatalogTargetProcessorBase

auditLog, connectorName, metadataCollectionGUID, metadataCollectionName

Fields inherited from class org.odpi.openmetadata.frameworks.integration.properties.RequestedCatalogTarget

connectorToTarget, integrationContext

Fields inherited from class org.odpi.openmetadata.frameworks.openmetadata.properties.RelationshipBeanProperties

typeName

Constructor Summary

Constructors

Constructor	Description
DuckDBDatabaseCatalogTargetProcessor(CatalogTarget catalogTarget, CatalogTargetContext integrationContext, Connector connectorToTarget, String connectorName, AuditLog auditLog)	Constructor

Method Summary

Modifier and Type	Method	Description
void	disconnect()	Free up any resources held since the connector is no longer needed.
void	refresh()	Requests that the connector does a comparison of the metadata in the third party technology and open metadata repositories.
void	start()	Indicates that the connector is completely configured and can begin processing.

Methods inherited from class org.odpi.openmetadata.frameworks.integration.connectors.CatalogTargetProcessorBase

getArrayConfigurationProperty, getArrayConfigurationProperty, getArrayConfigurationProperty, getBooleanConfigurationProperty, getDateConfigurationProperty, getHostIdentifier, getIntConfigurationProperty, getLongConfigurationProperty, getPortNumber, getStringConfigurationProperty, getSuppliedPlaceholderProperties, setUpMetadataSource, setUpSoftwareCapability, throwBadBeanClass, throwMissingConnectionInfo, throwMissingPropertyValue, throwWrongTypeOfCatalogTarget, throwWrongTypeOfResourceConnector

Methods inherited from class org.odpi.openmetadata.frameworks.integration.properties.RequestedCatalogTarget

equals, getConnectorToTarget, getIntegrationContext, getNetworkAddress, hashCode, toString

Methods inherited from class org.odpi.openmetadata.frameworks.opengovernance.properties.CatalogTarget

getCatalogTargetElement, getRelationshipGUID, getRelationshipVersions, setCatalogTargetElement, setRelationshipGUID, setRelationshipVersions

Methods inherited from class org.odpi.openmetadata.frameworks.openmetadata.properties.assets.processes.connectors.CatalogTargetProperties

getCatalogTargetName, getConfigurationProperties, getConnectionName, getDeleteMethod, getMetadataCollectionQualifiedName, getMetadataSourceQualifiedName, getPermittedSynchronization, getTemplates, setCatalogTargetName, setConfigurationProperties, setConnectionName, setDeleteMethod, setMetadataCollectionQualifiedName, setMetadataSourceQualifiedName, setPermittedSynchronization, setTemplates

Methods inherited from class org.odpi.openmetadata.frameworks.openmetadata.properties.RelationshipBeanProperties

getExtendedProperties, getTypeName, setExtendedProperties, setTypeName

Methods inherited from class org.odpi.openmetadata.frameworks.openmetadata.properties.RelationshipProperties

getEffectiveFrom, getEffectiveTo, setEffectiveFrom, setEffectiveTo

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Details

DuckDBDatabaseCatalogTargetProcessor

public DuckDBDatabaseCatalogTargetProcessor(CatalogTarget catalogTarget,
 CatalogTargetContext integrationContext,
 Connector connectorToTarget,
 String connectorName,
 AuditLog auditLog)

Constructor

Parameters:

catalogTarget - catalog target information

integrationContext - specialized context for this catalog target

connectorToTarget - connector to access the target resource

connectorName - name of this integration connector

auditLog - logging destination

Method Details

start

public void start()
           throws ConnectorCheckedException,
UserNotAuthorizedException

Indicates that the connector is completely configured and can begin processing.

Overrides:

start in class CatalogTargetProcessorBase

Throws:

ConnectorCheckedException - the connector detected a problem.

UserNotAuthorizedException - the connector was disconnected before/during start

refresh

public void refresh()
             throws ConnectorCheckedException

Requests that the connector does a comparison of the metadata in the third party technology and open metadata repositories.

Overrides:

refresh in class CatalogTargetProcessorBase

Throws:

ConnectorCheckedException - a problem with the connector. It is not able to refresh the metadata.

disconnect

public void disconnect()
                throws ConnectorCheckedException

Free up any resources held since the connector is no longer needed.

Overrides:

disconnect in class CatalogTargetProcessorBase

Throws:

ConnectorCheckedException - the connector detected a problem.