Package org.odpi.openmetadata.adapters.connectors.duckdb.survey

Class DuckDBFederationExtractor

java.lang.Object
org.odpi.openmetadata.adapters.connectors.duckdb.survey.DuckDBFederationExtractor
public class DuckDBFederationExtractor extends Object
DuckDBFederationExtractor discovers the federation relationships that a DuckDB database makes use of:
  • other databases that have been ATTACH-ed to it, discovered via DuckDB's duckdb_databases() table function
  • views that scan external file/object-store resources (via functions like read_parquet(), read_csv(), read_csv_auto(), read_json(), iceberg_scan() or delta_scan()), discovered via DuckDB's duckdb_views() table function
Both queries are wrapped defensively - not every DuckDB database makes use of federation features, and older DuckDB versions may not support every table function used here, so failures are logged and processing continues rather than failing the whole survey/catalog pass.

  • Constructor Details

    • DuckDBFederationExtractor

      public DuckDBFederationExtractor(SurveyActionServiceConnector surveyActionServiceConnector, AuditLog auditLog, String connectorName)
      Constructor.
      Parameters:
      surveyActionServiceConnector - calling connector (used to format JSON properties)
      auditLog - logging destination
      connectorName - name of the calling connector - used in audit log messages

  • Method Details

    • discoverAttachedSources

      public void discoverAttachedSources(Connection databaseJDBCConnection, String primaryDatabaseName)
      Discover databases that have been ATTACH-ed to the DuckDB database.
      Parameters:
      databaseJDBCConnection - connection to the DuckDB database
      primaryDatabaseName - name of the database being surveyed/catalogued - used only for audit log messages. This is NOT used to identify (and exclude) the "self" row in duckdb_databases(): it is Egeria's configured display name for the asset, which is independent of DuckDB's own internal database_name (always derived from the file's base name, eg "egeria_test" for "egeria_test.duckdb") - the two only match by coincidence. current_database() is queried instead to reliably identify the self row.

    • discoverExternalFileSources

      public void discoverExternalFileSources(Connection databaseJDBCConnection, String primaryDatabaseName)
      Discover views that scan external file/object-store resources.
      Parameters:
      databaseJDBCConnection - connection to the DuckDB database
      primaryDatabaseName - name of the database being surveyed/catalogued - used only for audit log messages

    • getAnnotations

      public List<AnnotationProperties> getAnnotations()
      Return the accumulated list of federation annotations discovered so far.
      Returns:
      list of annotations (never null, but may be empty)

    • getColumnsForTable

      public List<DuckDBFederationExtractor.DuckDBColumnInfo> getColumnsForTable(Connection databaseJDBCConnection, String tableName)
      Retrieve the ordered list of columns (name and DuckDB-reported data type) for a table or view known to DuckDB's own catalog. Used both for a DuckDB view's own columns (to model the "real" structure of the external file/object-store resource it scans) and, via getColumnsForAttachedTable(java.sql.Connection, java.lang.String, java.lang.String), for tables exposed through an ATTACH-ed network-backed data source.
      Parameters:
      databaseJDBCConnection - connection to the DuckDB database
      tableName - name of the table/view to retrieve columns for
      Returns:
      ordered list of columns - never null, may be empty

    • getTablesForAttachedDatabase

      public List<String> getTablesForAttachedDatabase(Connection databaseJDBCConnection, String databaseAlias)
      Retrieve the names of the tables exposed by an ATTACH-ed data source, as known to DuckDB's own catalog.
      Parameters:
      databaseJDBCConnection - connection to the DuckDB database (must already have re-issued the ATTACH statement that established the alias, since DuckDB does not persist ATTACH-ed data sources between sessions/connections)
      databaseAlias - alias the data source was ATTACH-ed under
      Returns:
      list of table names - never null, may be empty

    • getColumnsForAttachedTable

      public List<DuckDBFederationExtractor.DuckDBColumnInfo> getColumnsForAttachedTable(Connection databaseJDBCConnection, String databaseAlias, String tableName)
      Retrieve the ordered list of columns (name and DuckDB-reported data type) for one table exposed by an ATTACH-ed data source, filtering duckdb_columns() by both database_name and table_name so that a table name that happens to be duplicated across several ATTACH-ed sources is not confused with the wrong one.
      Parameters:
      databaseJDBCConnection - connection to the DuckDB database
      databaseAlias - alias the data source was ATTACH-ed under
      tableName - name of the table to retrieve columns for
      Returns:
      ordered list of columns - never null, may be empty