STDIO Adapter

The stdio adapter allows reading points from stdin and writing points to stdout.

STDIO Adapter
- read stdio
- write stdio

read stdio

Supported formats are JSON array, JSON lines, column-aligned text, and CSV; see examples below.

read stdio [-format <format>] [-timeField <fieldname>]

Parameter	Description	Required?
`-format`	Input file format, supports: `csv` `json` `jsonl` or `grok`for text parseable by grok	No; defaults to `json`
`-timeField`	Name of the field in the data which contains a valid timestamp	No; defaults to `time`
`-pattern`	When `-format 'grok'`you can specify the grok matching pattern here. More information on grok here	No
`-separator`	When `-format 'csv'`is used, you can specify the separator between columns in a CSV file.	No: defaults to `,`
`-commentSymbol`	When `-format 'csv'`is used, you can specify the comment character that prefixes comment lines in a CSV file.	No: defaults to `,`
`-ignoreEmptyLines`	When `-format 'csv'`is used, you can skip empty lines in a CSV file.	No: defaults to `false`
`-allowIncompleteLines`	When `-format 'csv'`is used, you can allow for parsing of incomplete lines in a CSV file.	No: defaults to `false`

The data is assumed to contain valid timestamps in a field named time by default; a different name for the time field may be specified with -timeField option. If the data contains fields time and another named field specified with -timeField, the original contents of field time will be overwritten by the valid timestamp from timeField. Timeless data, which contains no timestamps, is acceptable; however certain operations which expect time to be present in the points, such as reduce -every :interval:, will execute with warnings or error out. Timeless data can be joined in the Juttle flowgraph with other data which contains timestamps; a common use case for reading timeless data from a file or another backend is to join it with streaming data for enrichment.

The stdio adapter does not support any kind of filtering (neither filter expressions, nor full text search). In order to filter the data from the read stdio, pipe to the filter proc.

Example: redirect a JSON file to your juttle program

This is an example using the CLI since this is where the stdio adapter comes in handy:

cat myfile.json | juttle -e "read stdio | filter hostname = 'lemoncake' | view table"

Example: count the processes holding the most open files

This is an example using column-aligned text formatting that comes as the output of UNIX commands like lsof.

lsof | juttle -e "read stdio -format 'columns' | reduce count() by COMMAND | sort count -desc | head 10 | view table"

write stdio

Supported formats are JSON array, JSON lines and CSV; see examples below.

... | write stdio [-format <format>]

Parameter	Description	Required?
`-format`	Input input format: `csv`for CSV data, `json`for JSON data, or `jsonl`for JSON lines data	No; defaults to `json`

Example: writing csv data to stdout

emit -from :2015-01-01: -limit 2
| put name = 'write_me', value = count()
| write stdio -format 'csv'

The resulting output will look like so:

time,name,value
2015-01-01T00:00:00.000Z,write_me,1
2015-01-01T00:00:01.000Z,write_me,2