Tools 56. Mozilla SOPS Trial Our advice when it comes to secrets management has always been to decouple it from source code. However, teams are often presented with a tradeoff between full automation (in the spirit of infrastructure as code) versus a few manual steps (using tools like vaults) for managing, seeding and rotating seed secrets. For instance, our teams use SOPS to manage seed credentials for bootstrapping infrastructure. In some situations, however, it’s impossible to remove secrets from legacy code repositories. For such needs, we found Mozilla SOPS to be a good choice for encrypting secrets in text files. SOPS integrates with cloud-managed keystores such as AWS and GCP Key Management Service (KMS) or Azure Key Vault as sources of encryption keys. It also works cross-platform and supports PGP keys. 57. Ru昀昀 Trial Ruff is a new linter for Python. For us, the question is not whether to use a linter or not but which linter to use, and there are several choices for Python. Ruff stands out for two reasons: its out-of-box experience and its speed. It has over 500 rules built in and readily replaces Flake8, including many of that linter’s plug-ins. The claims by the team behind Ruff about its performance are borne out by our experience. It really is at least an order of magnitude faster than other linters, which is a huge benefit because it helps reduce build times on large codebases. 58. Soda Core Trial Soda Core is an open-source data quality and observability tool. Our teams have used it to validate data as it arrives in a system, before and after transformations, and set up automated monitoring checks for anomalies. We’re happy with SodaCL, the DSL for writing data checks in Soda Core, as it helps team members other than data engineers write quality checks. Overall, our experience using Soda Core to find and resolve data issues at scale has been positive. 59. Steampipe Trial Steampipe is an open-source tool that lets you instantly query cloud services like AWS, Azure and GCP with SQL. With 100+ plugins and built-in support for creating dashboards, Steampipe makes it trivial to connect live cloud configuration data with internal or external data sets and create security or compliance dashboards. We’ve enjoyed working with Steampipe and created several such dashboards with AWS cloud configurations. © Thoughtworks, Inc. All Rights Reserved. 31