Package org.odpi.openmetadata.adapters.connectors.lineageinsight.openlineage

Class OpenLineageLogStoreReader

java.lang.Object

org.odpi.openmetadata.adapters.connectors.lineageinsight.openlineage.OpenLineageLogStoreReader

public class OpenLineageLogStoreReader
extends Object

OpenLineageLogStoreReader walks the directory tree written by the file-based OpenLineage log store integration connector ({directory}/{namespace}/{job}/{runId}-{timestamp}-{eventType}.openlineageevent), parses each run event and accumulates it into an OpenLineageRunHistory. Events outside the requested time window are skipped.

Constructor Summary

Constructors

Constructor	Description
OpenLineageLogStoreReader()

Method Summary

Modifier and Type	Method	Description
void	addEvent(OpenLineageRunEvent event, Date windowStart, Date windowEnd, OpenLineageRunHistory history)	Add a run event to the history.
static Date	parseTimestamp(String timestamp)	Parse an ISO-8601 timestamp from an OpenLineage event.
OpenLineageRunHistory	read(File logStoreDirectory, Date windowStart, Date windowEnd)	Read every event file below a directory into a history.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

OpenLineageLogStoreReader

public OpenLineageLogStoreReader()

Method Details

read

public OpenLineageRunHistory read(File logStoreDirectory,
 Date windowStart,
 Date windowEnd)

Read every event file below a directory into a history.

Parameters:

logStoreDirectory - root of the log store

windowStart - earliest event time to include (null for no limit)

windowEnd - latest event time to include (null for no limit)

Returns:

history

addEvent

public void addEvent(OpenLineageRunEvent event,
 Date windowStart,
 Date windowEnd,
 OpenLineageRunHistory history)

Add a run event to the history.

Parameters:

event - event

windowStart - earliest event time to include (null for no limit)

windowEnd - latest event time to include (null for no limit)

history - history to add to

parseTimestamp

public static Date parseTimestamp(String timestamp)

Parse an ISO-8601 timestamp from an OpenLineage event. Timestamps without a zone are treated as UTC.

Parameters:

timestamp - string timestamp

Returns:

date or null if the timestamp is missing or unparsable