Tag Archives: opennebula

Building Private and Hybrid Clouds with Ubuntu 9.04

Ubuntu 9.04 (Jaunty Jackalope) has been released today bringing highly interesting new features, specially in the Cloud Computing and Virtualization area. The new Ubuntu server distribution includes two complementary cloud tools, OpenNebula and Eucalyptus, so providing the technology required to build the three types of Cloud architectures, namely private, hybrid and public clouds.

Eucalyptus can be used to transform an existing infrastructure into an IaaS public cloud, being compatible with Amazon’s EC2 interface. Eucalyptus is fully functional with respect to providing cloud-like interfaces and higher-level cloud functionality for security, contextualization and image management. OpenNebula, on the other hand, is a virtual infrastructure engine that enables the dynamic and scalable deployment and re-placement of groups of interconnected virtual machines within and across sites. OpenNebula can be primarily used as a virtualization tool to manage a distributed virtual infrastructure in the datacenter or cluster. This application is usually referred as private cloud, and  OpenNebula can also dynamically scale the local infrastructure using external clouds, so building hybrid clouds. OpenNebula provides dynamic “cloudbursting” to any cloud with Amazon EC2 interfaces, including Eucalyptus-based clouds.

OpenNebula is building an ecosytem with tools extending its functionality, such as the Haizea lease management system, a libvirt implementation on top of OpenNebula or a VM consolidation scheduler fro GreenIT. The project provides support to host the development of the new ecosystem projects.

Moreover, because OpenNebula is one of the technologies being enhanced in RESERVOIR, flagship European research initiative in virtualized infrastructures and cloud computing, in few months there will be available several new components complementing its functionality for service elasticity management, VM placement to meet SLA commitments, supporting public cloud interfaces…

Ignacio M. Llorente

OpenNebula Wins the Best Demo Award at OGF25/4th EGEE-UF

We are happy to announce that our demo “Scaling out EGEE sites on Amazon EC2 with OpenNebula” has won the best demo award in the 4th EGEE User Forum/OGF 25 and OGF Europe’s 2nd International Event. The demos were evaluated by the EGEE External Advisory Committee.

The demonstration showed how virtualization can be used to transform a physical cluster into a flexible and elastic virtual infrastructure, separating resource provisioning from job execution management, and supporting the dynamic adaptation of a virtual EGEE site to the users’ computational demands. The virtual infrastructure, managed by the OpenNebula Virtual Infrastructure Engine, ran on local and Cloud resources, so automatically scaling out the local infrastructure in order to meet peak demands. This demo shows how Grid and Cloud are complementary technologies that will coexist and cooperate at different levels of abstraction in future e-infrastructures. In this particular use case, clouds help to overcome many of the barriers to grid adoption. GridCast recorded a video with the demo.

The OpenNebula Team

Interoperation between Cloud Infrastructures

A Distributed Virtual Infrastructure (VI) Manager is responsible for the efficient management of the virtual infrastructure as a whole, by providing functionality for deployment, control and monitoring of groups of interconnected Virtual Machines (VMs) across a pool of resources. An added functionality of these management tools is the dynamic scaling of the virtual infrastructure with resources from remote providers, so seamless integrating remote Cloud resources with in-house infrastructures. This novel functionality allows to add and remove capacity in order to meet peak or fluctuating service demands, so providing the foundation for interoperation between Cloud infrastructures. The distributed virtual infrastructure would run on top of a geographically distributed physical infrastructure consisting of resources from the private cloud and several external cloud providers.

Following the terminology defined by the Grid community for getting Grids to work together, we use the term interoperation for the set of techniques to get production Cloud infrastructures to work together using adapters and gateways. While interoperability would refer to the ability of Cloud infrastructures to interact directly via common open standards.

Since release 1.0, OpenNebula distribution includes the plugins required to supplement local resources with Amazon EC2 resources to satisfy peak or fluctuating demands. This novel feature has been illustrated in several use cases for computing clusters and web servers. The open and flexible architecture of OpenNebula makes quite simple to create new plugins to access other cloud providers. In order to illustrate this and to provide KVM users with an utility access to remote resources, the OpenNebula team has just released the plugins required to dynamically grow the infrastructure using ElasticHosts resources. ElasticHosts is the world’s first public cloud based upon KVM, providing scalable and flexible virtual server capacity for cloud hosting. An interesting result is that a private infrastructure could dynamically grow using resources from different Cloud providers according to provisioning policies based on resource availability, performance, costs, availability…

The release of these new plugins represents a new step towards an open-source framework for cloud infrastructure federation, which is one of the main goasl of the Reservoir project, European research initiative in virtualized infrastructures and cloud computing.

Ignacio Martín Llorente

Release of OpenNebula Cloud Plug-in for ElasticHosts

The OpenNebula Team is releasing a new plug-in to interface the ElasticHosts cloud provider, so it can be used to dynamically increase capacity of your virtualized infrastructure to meet fluctuating or peak demands. This can happen when the local fabric runs out of capacity to spawn a new virtual machine, therefore it may be interesting to add capacity using cloud providers.

Cloud bursting with OpenNebula and ElasticHosts

ElasticHosts offers KVM based virtualized hosts in a cloud like fashion, i.e., à la Amazon EC2, using a very neat RESTful API. Uploading images (drives, in ElasticHosts speak) previously configured with the service that needs to meet a increased demand would allow the cloudbursting described above through OpenNebula.

Information on how to download and install the ElasticHosts plug-in can be found in the OpenNebula Trac.

Tino Vazquez

Haizea Technology Preview 1.3 Released

Borja Sotomayor has just announced the release of a new version of the Haizea Lease Manager. Technology Preview 1.3 now includes support for OpenNebula 1.2 (released one week ago), and enhanced stability and robustness. This is a new step towards TP2.0, which will include a policy engine and several novel scheduling features. The detailed list of changes is available in the project changelog.

Ignacio Martín Llorente

The Emerging Ecosystem of Cloud Components

MIT Technology Review has just published an interesting article entitled “Openning the Cloud” about open-source technological components to build a cloud-like infrastructure. The article focuses on the IaaS (Infrastructure as a Service) paradigm, describing the components required to develop a solution to provide virtualized resources as a service. The article briefly describes the following technologies: OpenNebula, Globus Nimbus, and Eucalyptus.

In the OpenNenula project, we strongly believe that a complete Cloud solution requires the integration of several of the available components, with each component focused on a niche. The open architecture and interfaces of the OpenNevula VM Manager allow its integration with third-party tools, such as capacity managers, cloud interfaces, service adapters, VM image managers…; so supporting a complete solution for the deployment of flexible and efficient virtual infrastructures. We maintain an Ecosystem web page with information about third-party tools to extend the functionality provided by OpenNebula.

Ignacio Martín Llorente

The OpenNebula Engine for Data Center Virtualization and Cloud Solutions

Virtualization has opened up avenues for new resource management techniques within the data center. Probably, the most important characteristic is its ability to dynamically shape a given hardware infrastructure to support different services with varying workloads. Therefore, effectively decoupling the management of the service (for example a web server or a computing cluster) from the management of the infrastructure (e.g. the resources allocated to each service or the interconnection network).

A key component in this scenario is the virtual machine manager. A VM manager is responsible for the efficient management of the virtual infrastructure as a whole, by providing basic functionality for the deployment, control and monitoring of VMs on a distributed pool of resources. Usually, these VM managers also offer high availability capabilities and scheduling policies for VM placement and physical resource selection. Taking advantage of the underlying virtualization technologies and according to a set of predefined policies, the VM manager is able to adapt the physical infrastructure to the services it supports and their current load. This adaptation usually involves the deployment of new VMs or the migration of running VMs to optimize their placement.

The dsa-research group at the Universidad Complutense de Madrid has released under the terms of the Apache License, Version 2.0, the first stable version of the OpenNebula Virtual Infrastructure Engine. OpenNebula enables the dynamic allocation of virtual machines on a pool of physical resources, so extending the benefits of existing virtualization platforms from a single physical resource to a pool of resources, decoupling the server not only from the physical infrastructure but also from the physical location. OpenNebula is a component being enhanced within the context of the RESERVOIR European Project.

The new VM manger differentiates from existing VM managers in its highly modular and open architecture designed to meet the requirements of cluster administrators. OpenNebula 1.0 supports Xen and KVM virtualization platforms to provide several features and capabilities for VM dynamic management, such as centralized management, efficient resource management, powerful API and CLI interfaces for monitoring and controlling VMs and physical resources, fault tolerant design… Two of the outstanding new features are its support for advance reservation leases and on-demand access to remote cloud provider

Support for Advance Reservation Leases

Haizea is an open source lease management architecture that OpenNebula can use as a scheduling backend. Haizea uses leases as a fundamental resource provisioning abstraction, and implements those leases as virtual machines, taking into account the overhead of using virtual machines (e.g., deploying a disk image for a VM) when scheduling leases. Using OpenNebula with Haizea allows resource providers to lease their resources, using potentially complex lease terms, instead of only allowing users to request VMs that must start immediately.

Support to Access on-Demand to Amazon EC2 resources

Recently, virtualization has also brought about a new utility computing model, called cloud computing, for the on-demand provision of virtualized resources as a service. The Amazon Elastic Compute Cloudi s probably the best example of this new paradigm for the elastic capacity providing. Thanks to virtualization, the clouds can be used efficiently to supplement local capacity with outsourced resources. The joint use of these two technologies, VM managers and clouds, will change arguably the structure and economics of current data centers. OpenNebula provides support to access Amazon EC2 resources to supplement local resources with cloud resources to satisfy peak or fluctuating demands.

Scale-out of Computing Clusters with OpenNebula and Amazon EC2

As use case to illustrate the new capabilities provided by OpenNebula, the release includes documentation about the application of this new paradigm (i.e. the combination of VM managers and cloud computing) to a computing cluster, a typical data center service. The use of a new virtualization layer between the computing cluster and the physical infrastructure extends the classical benefits of VMs to the computing cluster, so providing cluster consolidation, cluster partitioning and support for heterogeneous workloads. Moreover, the integration of the cloud in this layer allows the cluster to grow on-demand with additional computational resources to satisfy peak demands.

Ignacio Martín Llorente

Release of OpenNebula 1.0 for Data Center Virtualization & Cloud Solutions

The dsa-research group is pleased to announce that a stable release (v1.0) of the OpenNebula (ONE) Virtual Infrastructure Engine is available for download under the terms of the Apache License, Version 2.0. ONE enables the dynamic allocation of virtual machines on a pool of physical resources, so extending the benefits of existing virtualization platforms from a single physical resource to a pool of resources, decoupling the server not only from the physical infrastructure but also from the physical location.

Main Features

The OpenNebula Virtual Infrastructure Engine differentiates from existing VM managers in its highly modular and open architecture designed to meet the requirements of cluster administrators. The last version supports Xen and KVM virtualization platforms to provide the following features and capabilities:

  • Centralized management, a single access point to manage a pool of VMs and physical resources.
  • Efficient resource management, including support to build any capacity provision policy and for advance reservation of capacity through the Haizea lease manager
  • Powerful API and CLI interfaces for monitoring and controlling VMs and physical resources
  • Easy 3rd party software integration to provide a complete solution for the deployment of flexible and efficient virtual infrastructures
  • Fault tolerant design, state is kept in a SQLite database.
  • Open and flexible architecture to add new infrastructure metrics and parameters or even to support new Hypervisors.
  • Support to access Amazon EC2 resources to supplement local resources with cloud resources to satisfy peak or fluctuating demands.
  • Ease of installation and administration on UNIX clusters
  • Open source software released under Apache license v2.0
  • As engine for the dynamic management of VMs, OpenNebula is being enhanced in the context of the RESERVOIR project (EU grant agreement 215605) to address the requirements of several business use cases.

More details at http://www.opennebula.org/doku.php?id=documentation:rn-rel1.0

Relevant Links

  • Benefits and Features: http://www.opennebula.org/doku.php?id=about
  • Documentation: http://www.opennebula.org/doku.php?id=documentation
  • Release Notes: http://www.opennebula.org/doku.php?id=documentation:rn-rel1.0
  • Download: http://www.opennebula.org/doku.php?id=software
  • Ecosystem: http://www.opennebula.org/doku.php?id=ecosystem

First Technology Preview of the Haizea Lease Manager

I would like to give a warm welcome to Haizea to the virtualization ecosystem. The new technological component is an open-source VM-based lease management architecture, which can be used

  • As a platform for experimenting with scheduling algorithms that depend on VM deployment or on the leasing abstraction.
  • In combination with the OpenNebula virtual infrastructure manager, to manage a Xen or KVM cluster, allowing you to deploy different types of leases that are instantiated as virtual machines (VMs).

Its full integration with OpenNebula will be part of the next Technoloy Preview (TP1.1), due mid-july. Haizea is being developed by Borja Sotomayor, a PhD student at the University of Chicago, who is now visiting our research group partially funded by the European Union’s FP7 Reservoir project (“Resources and Services Virtualization without Barriers”).

Ignacio Martín Llorente

HPC, Grid and Cloud Computing in Cetraro

I am attending the INTERNATIONAL ADVANCED RESEARCH WORKSHOP ON HIGH PERFORMANCE COMPUTING AND GRIDS in Cetraro (Italy). This is the 9th edition of the workshop organized by Prof. Lucio Grandinetti. I have to say the venue of the workshop, at Grand Hotel San Michele, is just perfect. The panel of speakers includes representatives of the more relevant Grid and HPC research initiatives and technologies around the world. The abstracts of the presentations are available online at the workshop site.

Cloud Computing for on-Demand Resource Provisioning

This is the title of the talk that I gave in the Workshop. The aim of the presentation was to show the benefits of the separation of resource provisioning from job execution management in different deployment scenarios. Within an organization, the incorporation of a new virtualization layer under existing Cluster and HPC middleware stacks decouples the execution of the computing services from the physical infrastructure. The dynamic execution of working nodes, on virtual resources supported by virtual machine managers such as the OpenNEbula Virtual Infrastructure Engine, provides multiple benefits, such as cluster consolidation, cluster partitioning and heterogeneous workload execution. When the computing platform is part of a Grid Infrastructure, this approach additionally provides generic execution support, allowing Grid sites to dynamically adapt to changing VO demands, so overcoming many of the obstacles for Grid adoption.

The previous scenario can be modified so the computing services are executed on a remote virtual infrastructure. This is the resource provision paradigm implemented by some commercial and scientific infrastructure Cloud Computing solutions, such as Globus VWS or Amazon EC2, which provide remote interfaces for control and monitoring of virtual resources. In this way a computing platform could scale out using resources provided on-demand by a provider, so supplementing local physical computing services to satisfy peak or unusual demands. Cloud interfaces can also provide support for the federation of virtualization infrastructures, so allowing virtual machine managers to access resources from remote resources providers or Cloud systems in order to meet fluctuating demands. The OpenNEbula Virtual Infrastructure Engine is being enhanced to access on-demand resources from EC2 and Globus-based clouds. This scenario is being studied in the context of the RESERVOIR– Resources and Services Virtualization without Barriers — EU-funded initiative

Download the slides

Towards a New Model for the Infrastructure Grid

This is the title of my contribution in the Panel “From Grids to Cloud Services”, chaired by Charlie Catlett, in the Workshop. The aim of the presentation was to introduce the discussion on the future of compute grid infrastructures, from infrastructures for the sharing of basic resource services to infrastructures for the sharing of hardware resources. A widely distributed virtual infrastructure, inspired in the federation of cloud systems as providers of virtualized resources (hardware) as a service, would not require end users to learn new interfaces and port their applications to the expected runtime environment. The sharing of resources would be performed at resource level, so local job managers could scale out to partner or commercial clouds, transparently to end users. This new model provides additional benefits, such as the support to any service, seamless integration with any service middleware stack…; at the cost of the virtualization overhead in the execution of the jobs.

It was very interesting to share this position on cloud computing with other researchers from Grid and HPC fields. So the question is: Are the existing compute Grid Infrastructures going to evolve to Grids of Clouds?. In other words, Which model is better for end users and site administrators?, to share basic infrastructure services or the physical infrastructure?.

Download the slides

Ignacio Martín Llorente