Package org.odpi.openmetadata.adapters.connectors.integration.dcat

Class DcatDirectoryScanner

java.lang.Object

org.odpi.openmetadata.adapters.connectors.integration.dcat.DcatDirectoryScanner

public class DcatDirectoryScanner
extends Object

DcatDirectoryScanner watches a directory tree (for example a drop folder) for DCAT documents. On each scan it looks for JSON-LD files that describe DCAT catalogs, datasets, dataset series or data services and publishes those that are new, or have changed since they were last published. Hidden directories (such as .git) are skipped. The scanner remembers the last modified time of each file it has processed so that a document is only published once per change. A file that is not a DCAT document (for example some other JSON file) is remembered too, so it is only re-examined when it changes.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static interface	DcatDirectoryScanner.DocumentPublisher	Callback used to publish a document found by the scanner.

Constructor Summary

Constructors

Constructor	Description
DcatDirectoryScanner(File directory, String sourceName)	Constructor.

Method Summary

Modifier and Type	Method	Description
File	getDirectory()	Return the directory being scanned.
List<File>	scan(DcatDirectoryScanner.DocumentPublisher publisher)	Scan the directory tree and publish the new and changed DCAT documents.
void	scanAndPublish(IntegrationContext integrationContext, AuditLog auditLog, String connectorName)	Scan the directory tree, publish the new and changed documents through the integration context and write the outcome to the audit log.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

DcatDirectoryScanner

public DcatDirectoryScanner(File directory,
 String sourceName)

Constructor.

Parameters:

directory - directory to scan

sourceName - where the directory name came from (for messages)

Method Details

getDirectory

public File getDirectory()

Return the directory being scanned.

Returns:

file

scan

public List<File> scan(DcatDirectoryScanner.DocumentPublisher publisher)
                throws IOException

Scan the directory tree and publish the new and changed DCAT documents.

Parameters:

publisher - callback that receives each document

Returns:

the files published by this scan

Throws:

IOException - the directory can not be read

scanAndPublish

public void scanAndPublish(IntegrationContext integrationContext,
 AuditLog auditLog,
 String connectorName)

Scan the directory tree, publish the new and changed documents through the integration context and write the outcome to the audit log. Problems reading the directory are logged rather than thrown so that the connector carries on with its other directories and retries on the next refresh.

Parameters:

integrationContext - context used to publish the documents

auditLog - logging destination

connectorName - name of the calling connector (for messages)