# Routing logs to 2+ destinations with Fluentbit

By [Brian Luong](https://paragraph.com/@bluong) · 2023-07-10

---

> **Routing your service logs to multiple destinations has never been easier.**

[Fluentbit](https://fluentbit.io/) is a powerful tool to manage and route your logs.

*   It’s usually run as a sidecar to your service(s).
    
*   Fluentbit ingests, parses, and routes logs to the destinations you define.
    

![hummin’ bird](https://storage.googleapis.com/papyrus_images/92fc2f53581d9709138cf5c1646a28791806eaa46f78eecda75dc0c201317435.png)

hummin’ bird

### Why it matters:

*   Save costs — send a subset of logs to an indexing platform (Datadog, Splunk, etc.), the rest to an archive like S3.
    

* * *

Logs are crucial for operating your service. You may need to react to errors quickly but also shelve them away for historical analysis.

To that end, I want to configure Fluentbit to send:

*   `ERROR` logs to Datadog
    
*   All logs to S3
    

    # fluent-bit.conf
    
    [SERVICE]
        Flush 1
        Log_Level TRACE
    
    [FILTER]
        Name rewrite_tag
        Match_Regex ^(?!index_in_datadog).*
        Rule $log .*ERROR.* index_in_datadog false
    
    [OUTPUT]
        Name datadog
        Match index_in_datadog
        TLS on
        apikey ${DD_API_KEY}
        dd_service ${SERVICE}
        dd_tags env:production
    
    [OUTPUT]
        Name s3
        Match *
        Region us-east-1
        bucket my_log_bucket
        s3_key_format /${SERVICE}/%Y/%m/%d/%H/%M-%S-$UUID
        upload_timeout 30s
        retry_limit 3
    

Fluentbit processes logs in a pipeline:

*   `Filter` stage: matches `ERROR` logs and attaches the `index_in_datadog` on them
    
*   First `Output`: send logs to Datadog if `index_in_datadog` tag is on the log
    
*   Second `Output`: send all logs to S3
    

![Fluentbit pipeline](https://storage.googleapis.com/papyrus_images/86c90919d12dd6de7827e2164aab225965671eb29bff1a681e5d29be0bb9b97a.png)

Fluentbit pipeline

### Local testing

I find this workflow helpful for testing Fluentbit config locally:

*   Create a file with logs to test
    
*   Add a `tail` input to the config
    
*   Add a `stdout` output to the config
    
*   Run the Fluentbit Docker container using the config
    
*   Observe output
    

    [INPUT]
        Name tail
        Path /fluent-bit-conf/my_log.log
        Read_from_head true
    
    ...
    ...
    [OUTPUT]
        Name stdout
        Match *
    

  
Docker command:

    
    docker run --rm -it \
        -v $PWD/:/fluent-bit-conf/ \
        -p 24224:24224 \
        -e DD_API_KEY=XXXXXXX \
        fluent/fluent-bit \
        -c /fluent-bit-conf/fluent-bit.conf
    

* * *

Happy logging! 🌲🪓🪵🎉

---

*Originally published on [Brian Luong](https://paragraph.com/@bluong/routing-logs-to-2-destinations-with-fluentbit)*