Routing your service logs to multiple destinations has never been easier.

Fluentbit is a powerful tool to manage and route your logs.

• It’s usually run as a sidecar to your service(s).

• Fluentbit ingests, parses, and routes logs to the destinations you define.

• Save costs — send a subset of logs to an indexing platform (Datadog, Splunk, etc.), the rest to an archive like S3.

Logs are crucial for operating your service. You may need to react to errors quickly but also shelve them away for historical analysis.

To that end, I want to configure Fluentbit to send:

• ERROR logs to Datadog

• All logs to S3

# fluent-bit.conf

[SERVICE]
    Flush 1
    Log_Level TRACE

[FILTER]
    Name rewrite_tag
    Match_Regex ^(?!index_in_datadog).*
    Rule $log .*ERROR.* index_in_datadog false

[OUTPUT]
    Name datadog
    Match index_in_datadog
    TLS on
    apikey ${DD_API_KEY}
    dd_service ${SERVICE}
    dd_tags env:production

[OUTPUT]
    Name s3
    Match *
    Region us-east-1
    bucket my_log_bucket
    s3_key_format /${SERVICE}/%Y/%m/%d/%H/%M-%S-$UUID
    upload_timeout 30s
    retry_limit 3

Fluentbit processes logs in a pipeline:

• Filter stage: matches ERROR logs and attaches the index_in_datadog on them

• First Output: send logs to Datadog if index_in_datadog tag is on the log

• Second Output: send all logs to S3

I find this workflow helpful for testing Fluentbit config locally:

• Create a file with logs to test

• Add a tail input to the config

• Add a stdout output to the config

• Run the Fluentbit Docker container using the config

• Observe output

[INPUT]
    Name tail
    Path /fluent-bit-conf/my_log.log
    Read_from_head true

...
...
[OUTPUT]
    Name stdout
    Match *

Docker command:

docker run --rm -it \
    -v $PWD/:/fluent-bit-conf/ \
    -p 24224:24224 \
    -e DD_API_KEY=XXXXXXX \
    fluent/fluent-bit \
    -c /fluent-bit-conf/fluent-bit.conf

Happy logging! 🌲🪓🪵🎉