Monitoring your System’s Heartbeat using Cloudwatch

Have you implemented a system that is supposed to perform tasks at regular intervals? Does the repeated failure of such a system pose a threat to your quality of service? If so, I am sure you would want to be alerted, if your system suddenly stops performing these tasks.

We at Meltwater’s Premium Content team had this exact requirement. In this post we will share how we used Cloudwatch, to monitor the heartbeat of our system.

JUGRI: The JUpyter - GRemlin Interface

Jupyter is a popular web framework used with Python to easily visualize and manipulate data. It can display the results of many databases using the Pandas library, but the popular Gremlin graph query language hasn’t been supported.

To solve this problem we created and open-sourced JUGRI to show your Gremlin query results in the Jupyter Notebook. So if you are a Data Scientist using Python, and want to visualize your Gremlin graph queries using Jupyter, then JUGRI can be a handy addition to your toolbox.

Risk-free Deployments with Immutable Web Apps

Today we are excited to share our Immutable Web Applications methodology with you. Immutable Web Applications is a framework-agnostic methodology for building and deploying static, single-page applications that minimizes the complexity of live releases and enables continuous delivery through simple, flexible, atomic deployments.

If you care about building web applications, and want to make deployments easier and less risky, then this blog post is for you.

Hosting the Elixir Berlin Meetup

In Meltwater’s Berlin office, we are enthusiastic users and advocates for Elixir and ruby. Hence we were excited to get the chance to host the Elixir Berlin meetup for the first time this November.

It was the #53’rd edition of the Elixir Berlin already, what a great streak!

Besides hosting the event and sponsoring food and drinks, we were happy to share some of our own Elixir learnings. For some of our presenters it was the first time on stage at a developer meetup, and we can only say that the Elixir Berlin crowd made it very easy for us, so thank you for that!

Optimal Shard Placement in a Petabyte Scale Elasticsearch Cluster

At the heart of Meltwater’s and Fairhair.ai’s information retrieval systems lies a collection of Elasticsearch clusters containing billions of social media posts and editorial articles.

The index shards in our clusters vary greatly in their access pattern, workload and size which presents some very interesting challenges.

This blog post describes how we use Linear Optimization modeling for distributing search and indexing workload as evenly as possible across all nodes in our clusters.