Package org.odpi.openmetadata.adapters.connectors.unitycatalog.survey

Class OSSUnityCatalogServerSurveyBase

java.lang.Object

org.odpi.openmetadata.frameworks.connectors.Connector

org.odpi.openmetadata.frameworks.connectors.ConnectorBase

org.odpi.openmetadata.frameworks.opensurvey.SurveyActionServiceConnector

org.odpi.openmetadata.adapters.connectors.unitycatalog.survey.OSSUnityCatalogServerSurveyBase

All Implemented Interfaces:

AuditLoggingComponent, SecureConnectorExtension, VirtualConnectorExtension, SurveyActionService

Direct Known Subclasses:

OSSUnityCatalogInsideCatalogSurveyService, OSSUnityCatalogInsideSchemaSurveyService, OSSUnityCatalogServerSurveyService

public class OSSUnityCatalogServerSurveyBase
extends SurveyActionServiceConnector

Common functions for the OSS Unity Catalog survey action services.

The surveys retrieve their information through the Unity Catalog REST API, one list call per type of resource in each schema. Each call is made independently: if one fails (for example, because the survey's userId may not read a schema, or the element was deleted while the survey was running), the failure is logged and the survey continues, so that one inaccessible element leaves a gap in the results rather than ending the survey.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
protected static class	OSSUnityCatalogServerSurveyBase.ResourceProperties	Information about a UC resource collected for the inventory.
protected static class	OSSUnityCatalogServerSurveyBase.SchemaContents	The resources found in the schemas surveyed, accumulated across the schemas.

Nested classes/interfaces inherited from class org.odpi.openmetadata.frameworks.connectors.ConnectorBase

ConnectorBase.ProtectedConnection

Field Summary

Fields inherited from class org.odpi.openmetadata.frameworks.opensurvey.SurveyActionServiceConnector

connector, csvFileConnectorTypeGUID, embeddedConnectors, objectMapper, propertyHelper, surveyActionServiceName, surveyContext

Fields inherited from class org.odpi.openmetadata.frameworks.connectors.ConnectorBase

auditLog, connectionBean, connectorInstanceId, messageFormatter, secretsStoreConnectorMap

Constructor Summary

Constructors

Constructor	Description
OSSUnityCatalogServerSurveyBase()

Method Summary

Modifier and Type	Method	Description
protected String	getCatalogName()	Return the name of the catalog to survey; or throw an exception if there is no catalog name.
protected ResourceProfileAnnotationProperties	getNameListAnnotation(AnnotationType annotationType, Map<String,OSSUnityCatalogServerSurveyBase.ResourceProperties> nameProperties)	Fill out a name list annotation.
protected OSSUnityCatalogServerSurveyBase.ResourceProperties	getResourceProperties(ElementBase element)	Build the inventory properties for a Unity Catalog element.
protected String	getSchemaName()	Return the name of the schema to survey; or throw an exception if there is no schema name.
protected void	logRetrievalFailure(String informationSought, String elementName, Exception error)	Record that a call to Unity Catalog failed.
protected void	surveySchemaContents(OSSUnityCatalogResourceConnector ucConnector, String catalogName, String schemaName, OSSUnityCatalogServerSurveyBase.SchemaContents schemaContents)	Gather the volumes, tables (with their columns), registered models and functions in a schema.
protected void	writeInventory(String inventoryName, Map<String,OSSUnityCatalogServerSurveyBase.ResourceProperties> catalogList, Map<String,OSSUnityCatalogServerSurveyBase.ResourceProperties> schemaList, Map<String,OSSUnityCatalogServerSurveyBase.ResourceProperties> functionList, Map<String,OSSUnityCatalogServerSurveyBase.ResourceProperties> tableList, Map<String,OSSUnityCatalogServerSurveyBase.ResourceProperties> volumeList)	Add an inventory annotation to the survey report.

Methods inherited from class org.odpi.openmetadata.frameworks.opensurvey.SurveyActionServiceConnector

disconnect, getArrayRequestOrConfigurationProperty, getConnectorComponentDescription, getEmbeddedSurveyActionServices, getJSONProperties, getRootSchemaType, getSurveyContext, handleUnexpectedException, hasReachedFinalAnalysisStep, isFinalAnalysisStep, performCheckAssetAnalysisStep, performCheckAssetAnalysisStep, setAuditLog, setCSVFileConnectorTypeGUID, setSurveyActionServiceName, setSurveyContext, setUpAnnotation, setUpAnnotation, setUpAnnotation, start, throwMissingSchemaType, throwNoAssetConnector, writeNameCountInventory, writeNameDescriptionInventory, writeNameListInventory, writePropertyListInventory

Methods inherited from class org.odpi.openmetadata.frameworks.connectors.ConnectorBase

combineConfigurationProperties, disconnectConnectors, equals, fromCamelToCanonicalCase, fromCanonicalToCamelCase, fromCanonicalToKebabCase, fromCanonicalToSnakeCase, fromKebabToCanonicalCase, fromSnakeToCanonicalCase, getArrayConfigurationProperty, getArrayConfigurationProperty, getArrayConfigurationProperty, getArrayValue, getBooleanConfigurationProperty, getConnection, getConnectorInstanceId, getDateConfigurationProperty, getIntConfigurationProperty, getLongConfigurationProperty, getSecretsCollectionName, getSecretsLocation, getSecretsProvider, getStringConfigurationProperty, getStringConfigurationProperty, getSuppliedPlaceholderProperties, hashCode, initialize, initializeEmbeddedConnectors, initializeSecretsStoreConnector, isActive, logExceptionRecord, logExceptionRecord, logRecord, logRecord, throwMissingConfigurationProperty, throwMissingElement, throwMissingEndpointAddress, throwMissingResource, throwNoAsset, throwWrongTypeOfAsset, throwWrongTypeOfResource, throwWrongTypeOfRootSchema, toString

Methods inherited from class org.odpi.openmetadata.frameworks.connectors.Connector

clearStatisticProperty, clearStatisticTimestamp, getConnectorStatistics, getStatisticCounter, getStatisticProperty, getStatisticTimestamp, incrementStatisticCounter, initializeStatisticCounter, refreshStatistics, setStatisticProperty, setStatisticTimestamp

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Details

OSSUnityCatalogServerSurveyBase

public OSSUnityCatalogServerSurveyBase()

Method Details

getNameListAnnotation

protected ResourceProfileAnnotationProperties getNameListAnnotation(AnnotationType annotationType,
 Map<String,OSSUnityCatalogServerSurveyBase.ResourceProperties> nameProperties)
                                                             throws UserNotAuthorizedException

Fill out a name list annotation.

Parameters:

annotationType - type of annotation

nameProperties - list of names with properties

Returns:

annotation

Throws:

UserNotAuthorizedException - if there is a problem with the user's authorization

getCatalogName

protected String getCatalogName()
                         throws ConnectorCheckedException,
UserNotAuthorizedException

Return the name of the catalog to survey; or throw an exception if there is no catalog name.

Returns:

name of the catalog to survey

Throws:

ConnectorCheckedException - the catalog name is not specified

UserNotAuthorizedException - connector is disconnected

getSchemaName

protected String getSchemaName()
                        throws ConnectorCheckedException,
UserNotAuthorizedException

Return the name of the schema to survey; or throw an exception if there is no schema name.

Returns:

name of the catalog to survey

Throws:

ConnectorCheckedException - the catalog name is not specified

UserNotAuthorizedException - connector is disconnected

getResourceProperties

protected OSSUnityCatalogServerSurveyBase.ResourceProperties getResourceProperties(ElementBase element)

Build the inventory properties for a Unity Catalog element.

Parameters:

element - element retrieved from Unity Catalog

Returns:

inventory properties

surveySchemaContents

protected void surveySchemaContents(OSSUnityCatalogResourceConnector ucConnector,
 String catalogName,
 String schemaName,
 OSSUnityCatalogServerSurveyBase.SchemaContents schemaContents)

Gather the volumes, tables (with their columns), registered models and functions in a schema. Each type of resource is retrieved with its own call to Unity Catalog; a call that fails is logged and the others still contribute their results.

Parameters:

ucConnector - connector to Unity Catalog

catalogName - name of the catalog that owns the schema

schemaName - name of the schema

schemaContents - accumulating results

logRetrievalFailure

protected void logRetrievalFailure(String informationSought,
 String elementName,
 Exception error)

Record that a call to Unity Catalog failed. The survey carries on without its results.

Parameters:

informationSought - description of the information the call was retrieving

elementName - description of the element the call was about

error - the failure

writeInventory

protected void writeInventory(String inventoryName,
 Map<String,OSSUnityCatalogServerSurveyBase.ResourceProperties> catalogList,
 Map<String,OSSUnityCatalogServerSurveyBase.ResourceProperties> schemaList,
 Map<String,OSSUnityCatalogServerSurveyBase.ResourceProperties> functionList,
 Map<String,OSSUnityCatalogServerSurveyBase.ResourceProperties> tableList,
 Map<String,OSSUnityCatalogServerSurveyBase.ResourceProperties> volumeList)
                       throws ConnectorCheckedException,
InvalidParameterException,
PropertyServerException,
UserNotAuthorizedException,
IOException

Add an inventory annotation to the survey report.

Parameters:

inventoryName - name of the inventory

catalogList - list of catalog names and descriptions

schemaList - list of schema names and descriptions

functionList - list of function names and descriptions

tableList - list of table names and descriptions

volumeList - list of volume names and descriptions

Throws:

IOException - problem writing file

InvalidParameterException - problem creating CSV file asset

PropertyServerException - repository problem creating CSV file asset

UserNotAuthorizedException - authorization problem creating CSV file asset

ConnectorCheckedException - survey has been asked to stop