Package org.odpi.openmetadata.adapters.connectors.postgres.tabulardatasource

Class PostgresTabularDataSetConnector

java.lang.Object

org.odpi.openmetadata.frameworks.connectors.Connector

org.odpi.openmetadata.frameworks.connectors.ConnectorBase

org.odpi.openmetadata.adapters.connectors.postgres.tabulardatasource.PostgresTabularDataSetConnector

All Implemented Interfaces:

SecureConnectorExtension, ReadableTabularDataSource, WritableTabularDataSource, VirtualConnectorExtension

Direct Known Subclasses:

PostgresTabularDataSetCollectionConnector

public class PostgresTabularDataSetConnector
extends ConnectorBase
implements ReadableTabularDataSource, WritableTabularDataSource

PostgresTabularDataSetConnector presents one PostgreSQL table as a simple table of data that can be both read and written. As a source, it describes the table's columns from the database's own catalog and reads its records in a stable order; as a destination, it creates the table from the column descriptions it is given and writes records into it. Column names cross the boundary in canonical form - capitalised words with spaces between them - and are held in the database in snake case, which is the naming convention PostgreSQL uses.

Nested Class Summary

Nested classes/interfaces inherited from class org.odpi.openmetadata.frameworks.connectors.ConnectorBase

ConnectorBase.ProtectedConnection

Field Summary

Fields

Modifier and Type	Field	Description
protected final PropertyHelper	propertyHelper
protected String	tableDescription
protected String	tableName

Fields inherited from class org.odpi.openmetadata.frameworks.connectors.ConnectorBase

auditLog, connectionBean, connectorInstanceId, embeddedConnectors, messageFormatter, secretsStoreConnectorMap

Constructor Summary

Constructors

Constructor	Description
PostgresTabularDataSetConnector()

Method Summary

Modifier and Type	Method	Description
void	appendRecord(List<String> dataValues)	Write the requested data record to the end of the data source.
void	deleteRecord(long rowNumber)	Remove the requested data record.
void	disconnect()	Close the file
List<TabularColumnDescription>	getColumnDescriptions()	Return the list of columns in this table, named in canonical form so that a destination can translate them into its own naming convention.
int	getColumnNumber(String columnName)	Locate the named column.
long	getRecordCount()	Return the number of records in the table.
String	getTableDescription()	Return the description for this data source.
String	getTableName()	Return the name of the table in canonical form - capitalised words with spaces between them - so that a destination can translate it into its own naming convention.
List<String>	readRecord(long rowNumber)	Return the requested data record.
void	setColumnDescriptions(List<TabularColumnDescription> columnDescriptions)	Set up the columns associated with this tabular data source, creating the schema and table to hold them if they do not already exist.
void	start()	Indicates that the connector is completely configured and can begin processing.
void	writeRecord(long requestedRowNumber, List<String> dataValues)	Write the requested data record.

Methods inherited from class org.odpi.openmetadata.frameworks.connectors.ConnectorBase

combineConfigurationProperties, disconnectConnectors, equals, fromCamelToCanonicalCase, fromCanonicalToCamelCase, fromCanonicalToKebabCase, fromCanonicalToSnakeCase, fromKebabToCanonicalCase, fromSnakeToCanonicalCase, getArrayConfigurationProperty, getArrayConfigurationProperty, getArrayConfigurationProperty, getBooleanConfigurationProperty, getConnection, getConnectorInstanceId, getDateConfigurationProperty, getIntConfigurationProperty, getLongConfigurationProperty, getSecretsCollectionName, getSecretsLocation, getSecretsProvider, getStringConfigurationProperty, getStringConfigurationProperty, getSuppliedPlaceholderProperties, hashCode, initialize, initializeEmbeddedConnectors, initializeSecretsStoreConnector, isActive, logExceptionRecord, logExceptionRecord, logRecord, logRecord, throwMissingConfigurationProperty, throwMissingElement, throwMissingEndpointAddress, throwMissingResource, throwNoAsset, throwWrongTypeOfAsset, throwWrongTypeOfResource, throwWrongTypeOfRootSchema, toString

Methods inherited from class org.odpi.openmetadata.frameworks.connectors.Connector

clearStatisticProperty, clearStatisticTimestamp, getConnectorStatistics, getStatisticCounter, getStatisticProperty, getStatisticTimestamp, incrementStatisticCounter, initializeStatisticCounter, refreshStatistics, setStatisticProperty, setStatisticTimestamp

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Field Details

tableName

protected String tableName

tableDescription

protected String tableDescription

propertyHelper

protected final PropertyHelper propertyHelper

Constructor Details

PostgresTabularDataSetConnector

public PostgresTabularDataSetConnector()

Method Details

start

public void start()
           throws UserNotAuthorizedException,
ConnectorCheckedException

Indicates that the connector is completely configured and can begin processing.

Overrides:

start in class ConnectorBase

Throws:

ConnectorCheckedException - the connector detected a problem.

UserNotAuthorizedException - the connector was disconnected before/during start

getRecordCount

public long getRecordCount()
                    throws ConnectorCheckedException

Return the number of records in the table.

Specified by:

getRecordCount in interface ReadableTabularDataSource

Specified by:

getRecordCount in interface WritableTabularDataSource

Returns:

count

Throws:

ConnectorCheckedException - problem accessing the data

getTableName

public String getTableName()
                    throws ConnectorCheckedException

Return the name of the table in canonical form - capitalised words with spaces between them - so that a destination can translate it into its own naming convention. The table is held in the database in snake case.

Specified by:

getTableName in interface ReadableTabularDataSource

Returns:

table name

Throws:

ConnectorCheckedException - data access problem

getTableDescription

public String getTableDescription()
                           throws ConnectorCheckedException

Return the description for this data source.

Specified by:

getTableDescription in interface ReadableTabularDataSource

Returns:

string

Throws:

ConnectorCheckedException - data access problem

getColumnDescriptions

public List<TabularColumnDescription> getColumnDescriptions()
                                                     throws ConnectorCheckedException

Return the list of columns in this table, named in canonical form so that a destination can translate them into its own naming convention. The column order is the table's own, which is also the order that the values of a record are returned in by readRecord(long).

Specified by:

getColumnDescriptions in interface ReadableTabularDataSource

Returns:

a list of column descriptions or null if not available.

Throws:

ConnectorCheckedException - data access problem

getColumnNumber

public int getColumnNumber(String columnName)
                    throws ConnectorCheckedException

Locate the named column. The name may be given in canonical form or as the database names it. A negative number means the column is not present.

Specified by:

getColumnNumber in interface ReadableTabularDataSource

Parameters:

columnName - name of the column

Returns:

column number, starting at 0

Throws:

ConnectorCheckedException - problem extracting the column descriptions

readRecord

public List<String> readRecord(long rowNumber)
                        throws ConnectorCheckedException

Return the requested data record. The first record is record 0.
A table has no row numbers of its own, so the record at a position is the one at that offset when the rows are put in the same repeatable order that deleteRecord(long) uses - by the columns that identify a record where the table has them, and by physical position where it does not. The values are returned in the table's column order, as text: PostgreSQL renders each value in its own type's text form, which is what a destination casts back on the way in.

Specified by:

readRecord in interface ReadableTabularDataSource

Parameters:

rowNumber - long

Returns:

List of strings, each string is the value from the column; a null value stays null.

Throws:

ConnectorCheckedException - a problem occurred accessing the data, or there is no such record.

setColumnDescriptions

public void setColumnDescriptions(List<TabularColumnDescription> columnDescriptions)
                           throws ConnectorCheckedException

Set up the columns associated with this tabular data source, creating the schema and table to hold them if they do not already exist.

Specified by:

setColumnDescriptions in interface WritableTabularDataSource

Parameters:

columnDescriptions - a list of column descriptions

Throws:

ConnectorCheckedException - data access problem

writeRecord

public void writeRecord(long requestedRowNumber,
 List<String> dataValues)
                 throws ConnectorCheckedException

Write the requested data record. The first data record is record 0.

A relational table has no row order to insert into, so the row number is not used; the record is written the same way as appendRecord(java.util.List<java.lang.String>): as an upsert keyed on the identifier columns when the column descriptions have been set and mark some, and as a plain insert otherwise. A plain insert was used here before, and it meant a data set delivered a second time to the same table - which is what a refresh subscription does - failed on every row already there.

Specified by:

writeRecord in interface WritableTabularDataSource

Parameters:

requestedRowNumber - long

dataValues - Map of column descriptions to strings, each string is the value for the column.

Throws:

ConnectorCheckedException - a problem occurred accessing the data.

appendRecord

public void appendRecord(List<String> dataValues)
                  throws ConnectorCheckedException

Write the requested data record to the end of the data source.

Specified by:

appendRecord in interface WritableTabularDataSource

Parameters:

dataValues - Map of column descriptions to strings, each string is the value for the column.

Throws:

ConnectorCheckedException - a problem occurred accessing the data.

deleteRecord

public void deleteRecord(long rowNumber)
                  throws ConnectorCheckedException

Remove the requested data record. The first data record is record 0.
A table has no row numbers of its own, so the record at a position is the one at that offset when the rows are put in a repeatable order - by the columns that identify a record where the table has them, and by physical position where it does not. That is the same order every other operation on this connector sees, which is what makes "the record at position n" mean the same thing to all of them.

Specified by:

deleteRecord in interface WritableTabularDataSource

Parameters:

rowNumber - long

Throws:

ConnectorCheckedException - a problem occurred accessing the data.

disconnect

public void disconnect()

Close the file

Overrides:

disconnect in class ConnectorBase