Workflow History Export in Temporal Cloud allows users to export closed workflow histories on a per-namespace basis to an AWS S3 cloud storage bucket.
Today in Temporal Cloud, the default retention period for workflow history data is 30-days. While users are able to extend this limit up to 90-days, those looking for longer-term data storage can take advantage of the workflow history export feature. By doing so, users can retain and preserve historical workflow data for compliance, auditing, and further analysis.
This feature has been available to a select group of users in Pre-release and we have already exported millions of workflows. Here’s what Betashares, a leading Australian provider of exchange-traded funds and Temporal Cloud customer, had to say about the feature.
“We’ve actually needed this feature for a long time. The 90-day retention window has been a real issue for us. I was really surprised to see how easy it was to set up, I went in and launched the AWS CloudFormation stack installer, created the role and it was ready to go.
One of the nice advantages with this feature is that two years from now we could re-run code in the debugger that would have run two years prior, and find out what went wrong! Given we’re a financial institution, the ability to have code-line level audit trails is something that gives us a great degree of safety and accountability.”
We’re excited to announce that as of today, this feature is now available to all users of Temporal Cloud in Public Preview, which means you can safely use the feature in production with data delivery guarantees.
There is a cost associated with this feature. Each workflow exported will accrue a single action, which will be rolled up and shown on your invoice as a separate line item per namespace.
To set up workflow export, users must first create an AWS S3 bucket in the same region as the namespace. Then through the Temporal Cloud UI or CLI, users grant Temporal permission to write data to their S3 bucket via AWS CloudFormation.
The image above is an example of setting up Workflow History Export in the Temporal Cloud UI
Once everything is set up correctly, Temporal will export history data from your namespace to S3 on an hourly basis. That’s pretty much it!
Here are some other useful resources for more details about the feature.
If you have any feedback, questions or concerns, please reach out to product@temporal.io!