My Internship Project – Improving Anti-Affinity in Sahara

It’s been around 3 weeks since my internship started and trust me every single day had been an entirely new learning experience.

My internship started on 7th December. The very first day, we had a meeting with the fellow OpenStack interns and mentors all around the world. It was kind of warm up for the internship. We discussed what and how we are going to achieve our goals set for the internship.

Project:

Improving anti-affinity behaviour for Sahara cluster creation

Sahara is a data processing component of OpenStack. It provisions the clusters (e.g a Hadoop cluster) into OpenStack cloud.

Since the nodes in the cluster are provisioned on virtual machines, we can’t be sure on which hypervisor they go. To provide high availability, hadoop ensures that there are replicas for the nodes. In the worst case, if all the replicas go onto the same hypervisor and the hypervisor goes down, high availability is not ensured. So to avoid this, Sahara provides the user with capability of defining anti-affinity policies. In the policy, the user is expected to define a node group, nodes of which should be anti-affine.

Sahara ensures that the nodes in the anti-affinity group are not on a single hypervisor. It provisions each of the anti-affine node on a different hypervisor. The problem with the current implementation is that, if the hypervisors are less than the number of nodes in the anti-affinity group, Sahara throws an error which can be a valid  use-case sometimes. My task involves me to understand the current cluster allocation strategy and improve it so as to support the above use case.

Mentor:

My mentor for the project is Michael McCune (elmiko). He is very supportive and encouraging. His “Keep up the good work!” comment always inspires me to do more and more. I would love to meet him in person some day. We discuss the status of the project every alternate day.

Work so far:

On the work front, I have installed DevStack with Sahara enabled (a blog post coming up soon on how to install it). I am exploring the Sahara codebase and pdb comes very handy. I am learning how to use screens in DevStack. So far, it has been an amazing experience.

I have fallen in love with OpenStack. I am learning new things each day. 🙂

Advertisements

One thought on “My Internship Project – Improving Anti-Affinity in Sahara

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s