| Comments

Optimal Shard Placement in a Petabyte Scale Elasticsearch Cluster

At the heart of Meltwater’s and Fairhair.ai’s information retrieval systems lies a collection of Elasticsearch clusters containing billions of social media posts and editorial articles.

The index shards in our clusters vary greatly in their access pattern, workload and size which presents some very interesting challenges.

This blog post describes how we use Linear Optimization modeling for distributing search and indexing workload as evenly as possible across all nodes in our clusters.

| Comments

Increase Diversity by Reducing Biases in your Hiring Process

Would you agree that your biases are affecting your recruitment process? We have been thinking about it and we were especially curious how we can improve our recruitment process by working with our biases and learning how to disarm those when hiring.

In this post we are sharing the tools and processes that we found useful. You can try them too!

| Comments

Using Machine Learning to Load Balance Elasticsearch Queries

Meltwater recently launched the Fairhair.ai data science platform. Part of this platform are several large Elasticsearch clusters, which serve insights over billions of social media posts and editorial articles. The nature of the searches that our customers need to run against this data quickly make the default load balancing behaviour of Elasticsearch insufficient.

In this post we explain how we built a custom search router using machine learning, that helps us to address the shortcomings of Elasticsearch’s default round-robin approach, and greatly improves search performance and fault tolerance.

| Comments

Meltwater is Sponsoring Brewing Agile 2018

Meltwater is excited to sponsor Brewing Agile in Gothenburg on October 12-13, 2018. This is the 4th year in a row that Meltwater is supporting Brewing Agile, so you can tell that we are honestly excited about this event.

This is the only conference about Agile in Gothenburg, and there are still tickets available, so don’t wait and sign up quickly.