One of our teams at Meltwater was recently faced with a problem that required relatively simple tasks applied to a large volume of data. To solve this we experimented with a pattern we call micro-pipelines, which are a sequence of microservices that work together to create efficient, fault tolerant systems.
This post provides an example of how we designed and built such a micro-pipeline.
Benthos Lab is a website where users of the Benthos stream processor can write, format, execute and share their pipeline configurations. This app is now a critical part of our Benthos workflow, but it would never have existed were it not for Meltwater Hackathons.
This post explains how we built Benthos Lab and how the Meltwater Hackathons have enabled us to do so.
Meltwater’s Foundation team has invested heavily in Kubernetes to provide this platform as a service to other development teams. Many of those teams have migrated to Kubernetes already.
Naturally our Foundation team learned a lot about what can, and will, go wrong when running a big enough Kubernetes cluster. Recently they attended Cloud Native Rejekts in Barcelona, where they shared some of these learnings.
Meltwater has been using Kubernetes in production for over a year. The theme for this meetup was “difficulties and pitfalls” so we wanted to share our lessons learned from offering Kubernetes as an internal service to our engineering teams all around the globe.
It was the second edition of the CloudNativeGbg meetup and we hosted the meetup it in our Gothenburg office. A great attendance ensured very interesting discussions before, during and after the event. Thank you all for coming!
We open sourced drone-cache, a plugin for the popular Continuous Delivery platform Drone. It allows you to cache dependencies and interim files between builds to reduce your build times. This post explains why we are using Drone, why we needed a cache plugin, and what I learned while trying to release drone-cache as open source software.
Read on for the story behind drone-cache or if you want to jump into action directly, go to the github.com/meltwater/drone-cache, and try it for yourself.