Fastly-Tempo🚀: A Real-Time Data Pipeline for Fastly’s CDN

4 min readApr 10, 2021

Monitor, Alert, and Display all your Fastly metrics in Real-Time!

Full Disclosure: This article is not sponsored (or promoted) by Fastly, New Relic, or any other organization. The sole purpose of this article is support the opensource community.

About 💡

Fastly-Tempo is an opensource project that allows you to stream real-time aggregated metrics from Fastly into New Relic and other monitoring services.

This is based off the New Relic blessed way to get your Fastly metrics into New Relic Insights, packaged as a Docker container image for ease of use!

This project is opensource and hosted on GitHub.

Getting Started 💻

Getting started and using the Fastly-to-Insights project is incredibly easy! In fact, it can be used by running a single command.

Before you get started, make sure that you have a Fastly API Key and a New Relic Insert Key.

Then run the command below:

docker run \
  -e ACCOUNT_ID='yourNewRelicAccountId' \
  -e FASTLY_KEY='yourFastlyKey' \
  -e INSERT_KEY='yourNewRelicInsertKey' \
  -e SERVICES='ServiceId1 ServiceId2 ...' \
  grantbirki/fastly-tempo:latest

Grab the image from DockerHub 🐳

Boom! You should now have metrics from all your specified Fastly services streaming in real-time into New Relic.

The next step is to create some dashboards to visualize this data.

Dashboard Creation 🗺️

To start visualizing your data, you will need to create a dashboard.

I have created a pre-made template which can be imported through a simply copy & paste in the New Relic UI. For more instructions on importing a dashboard through JSON, see New Relic’s documentation.

Tip: Make sure to “find and replace” “accountId”: 1234567 in the JSON template to your own New Relic accountId. Also ensure to set your dashboard permissions appropriately.

Steps:

Save the JSON dashboard template available here
“Find and Replace” “accountId”: 1234567 with your own New Relic accountId
Import the dashboard to your account through the New Relic UI — docs

Congrats! You should now have some pretty slick dashboards to visualize your Fastly metrics in real-time.

Dashboard Examples:

Exported Values 🧮

There are over 100 values are are exported from the Fastly-to-Insights project for your to create dashboards on, visualize and set alerts for. Here is the complete list of usable values:

service
num_requests
num_tls
num_http2
num_logs
num_pci
num_video
ipv6
pipe
uncacheable
shield
shield_resp_header_bytes
shield_resp_body_bytes
otfp
otfp_shield_time
otfp_deliver_time
otfp_manifests
otfp_shield_resp_header_bytes
otfp_shield_resp_body_bytes
otfp_resp_header_bytes
otfp_resp_body_bytes
bandwidth
resp_header_bytes
header_size
resp_body_bytes
body_size
req_body_bytes
req_header_bytes
bereq_header_bytes
bereq_body_bytes
billed_header_bytes
billed_body_bytes
status_2xx
status_3xx
status_4xx
status_5xx
status_200
status_204
status_301
status_304
status_400
status_401
status_403
status_404
status_500
status_501
status_502
status_503
status_504
status_505
status_1xx
waf_logged
waf_blocked
waf_passed
attack_req_body_bytes
attack_req_header_bytes
attack_logged_req_body_bytes
attack_logged_req_header_bytes
attack_blocked_req_body_bytes
attack_blocked_req_header_bytes
attack_passed_req_body_bytes
attack_passed_req_header_bytes
attack_resp_synth_bytes
hits
hit_ratio
miss
pass
pass_time
synth
errors
restarts
hits_time
miss_time
tls
tls_v10
tls_v11
tls_v12
tls_v13
imgopto
imgopto_resp_body_bytes
imgopto_resp_header_bytes
imgopto_shield_resp_body_bytes
imgopto_shield_resp_header_bytes
object_size_1k
object_size_10k
object_size_100k
object_size_1m
object_size_10m
object_size_100m
object_size_1g
recv_sub_time
recv_sub_count
hash_sub_time
hash_sub_count
deliver_sub_time
deliver_sub_count
hit_sub_time
hit_sub_count
prehash_sub_time
prehash_sub_count
predeliver_sub_time
predeliver_sub_count

Writing Custom Queries (NRQL)🔎

You can use any of the values above with New Relic’s Query Language to create your own visualizations and reports.

Here are several examples using NRQL to create a few sample visualizations:

Further Documentation 📚

For further documentation on how this project works, building your own images, and enabling more features, check out the GitHub repo!

Contributing 👩‍💻

If you like this project and want to support its development, you are free to do so! There are plans to expand Fastly Tempo to additional backends such as Graphite, Splunk, and Datadog.

JavaScript version 🔗

This project was forked and developed in Python from the Fastly-to-Insights project. Checkout the original version written in JavaScript by New Relic engineers.

Conclusion ⭐️

CDN’s are generally Tier 0 services that need robust and continuous monitoring. Fastly provides great real-time metrics available via their API or consumption. Using Fastly-Tempo, you can build out a centralized dashboard for real-time alerts, monitoring and service visualization for Fastly using their API. If you followed along, you should now have a data pipeline to visualize all your Fastly services and monitor their performance. Enjoy! 🎉