CSV

Consume comma-separated values row by row, including support for custom delimiters.

# Config fields, showing default values
csv:
  custom_delimiter: "" # No default (optional)
  parse_header_row: true
  lazy_quotes: false
  continue_on_error: false

Metadata

This scanner adds the following metadata to each message:

• csv_row The index of each row, beginning at 0.

Fields

custom_delimiter

Use a provided custom delimiter instead of the default comma.

Type: string

parse_header_row

Whether to reference the first row as a header row. If set to true the output structure for messages will be an object where field keys are determined by the header row. Otherwise, each message will consist of an array of values from the corresponding CSV row.

Type: bool
Default: true

lazy_quotes

If set to true, a quote may appear in an unquoted field and a non-doubled quote may appear in a quoted field.

Type: bool
Default: false

continue_on_error

If a row fails to parse due to any error emit an empty message marked with the error and then continue consuming subsequent rows when possible. This can sometimes be useful in situations where input data contains individual rows which are malformed. However, when a row encounters a parsing error it is impossible to guarantee that following rows are valid, as this indicates that the input data is unreliable and could potentially emit misaligned rows.

Type: bool
Default: false