Package org.odpi.openmetadata.adapters.connectors.lineageinsight.openlineage

Class OpenLineageRunHistory

java.lang.Object

org.odpi.openmetadata.adapters.connectors.lineageinsight.openlineage.OpenLineageRunHistory

public class OpenLineageRunHistory
extends Object

OpenLineageRunHistory is the in-memory model of a window of OpenLineage events read from a log store. It groups the events by job (into runs) and by dataset (into writes, reads and data quality results) so that the analysis services can work with a job's run series or a dataset's write series directly.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static final record	OpenLineageRunHistory.AssertionRecord	A data quality assertion evaluated against a dataset.
static class	OpenLineageRunHistory.DataSetHistory	The writes, reads and assertions seen for a dataset.
static class	OpenLineageRunHistory.JobHistory	The runs seen for a job.
static final record	OpenLineageRunHistory.Key	Identity of a job or dataset in OpenLineage.
static final record	OpenLineageRunHistory.ReadRecord	A read of a dataset by a run.
static class	OpenLineageRunHistory.RunRecord	The events seen for a single run of a job.
static final record	OpenLineageRunHistory.TestRecord	A test reported by the test run facet.
static final record	OpenLineageRunHistory.WriteRecord	A write to a dataset by a run.

Constructor Summary

Constructors

Constructor	Description
OpenLineageRunHistory()

Method Summary

Modifier and Type	Method	Description
void	countEvent(Date eventTime)	Record that an event has been added.
void	countUnreadableFile()	Record that a file could not be read.
OpenLineageRunHistory.DataSetHistory	getDataSet(OpenLineageRunHistory.Key dataset)	Return the history of a dataset, creating it if this is the first event for the dataset.
Map<OpenLineageRunHistory.Key,OpenLineageRunHistory.DataSetHistory>	getDataSets()	Return the datasets seen.
Date	getEarliestEvent()	Return the time of the earliest event in the history.
int	getEventCount()	Return the number of events in the history.
OpenLineageRunHistory.JobHistory	getJob(OpenLineageRunHistory.Key job)	Return the history of a job, creating it if this is the first event for the job.
Map<OpenLineageRunHistory.Key,OpenLineageRunHistory.JobHistory>	getJobs()	Return the jobs seen.
Date	getLatestEvent()	Return the time of the latest event in the history.
int	getUnreadableFiles()	Return the number of files that could not be read.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

OpenLineageRunHistory

public OpenLineageRunHistory()

Method Details

getJob

public OpenLineageRunHistory.JobHistory getJob(OpenLineageRunHistory.Key job)

Return the history of a job, creating it if this is the first event for the job.

Parameters:

job - job identity

Returns:

job history

getDataSet

public OpenLineageRunHistory.DataSetHistory getDataSet(OpenLineageRunHistory.Key dataset)

Return the history of a dataset, creating it if this is the first event for the dataset.

Parameters:

dataset - dataset identity

Returns:

dataset history

getJobs

public Map<OpenLineageRunHistory.Key,OpenLineageRunHistory.JobHistory> getJobs()

Return the jobs seen.

Returns:

map

getDataSets

public Map<OpenLineageRunHistory.Key,OpenLineageRunHistory.DataSetHistory> getDataSets()

Return the datasets seen.

Returns:

map

countEvent

public void countEvent(Date eventTime)

Record that an event has been added.

Parameters:

eventTime - time of the event

countUnreadableFile

public void countUnreadableFile()

Record that a file could not be read.

getEventCount

public int getEventCount()

Return the number of events in the history.

Returns:

count

getUnreadableFiles

public int getUnreadableFiles()

Return the number of files that could not be read.

Returns:

count

getEarliestEvent

public Date getEarliestEvent()

Return the time of the earliest event in the history.

Returns:

date or null

getLatestEvent

public Date getLatestEvent()

Return the time of the latest event in the history.

Returns:

date or null