Apache Beam Cheatsheet: Watermarks

Recommended reading: 4 minsCopy-paste ready Python code snippets

Track progress and event-time completeness in streaming pipelines.

TimestampedValue()

returns: TimestampedValue

Purpose & Description

Assigns event-time timestamps to elements before they enter temporal windows.

Syntax Signaturebeam.window.TimestampedValue(value, timestamp)

Executable Example

import apache_beam as beam

timestamped = records | "Add Timestamps" >> beam.Map(
    lambda x: beam.window.TimestampedValue(x, x["epoch_time"])
)

Used In

Reading elements from files or message fields lacking implicit timestamps.

Pro Tip

A Watermark is the runner's temporal completeness boundary; it guarantees that no elements with event-time t < T are expected.

More Free Data Engineering Cheatsheets (DataPlayArena Network)Interactive syntax references