Tag Archives: Globus

GridWay 5.10 is now available!

Dear GridWay users,

A new stable release of the GridWay Metascheduler is available for download. It comes with improved drivers for CREAM and BES,the possibility to perform a FHS-compliant installation, changes in the scheduler to enqueue jobs, and other minor features and bugfixes. You can find more information in the release notes and the documentation.

Again, these developments have been funded by the IGE project. Packages for the GridWay core and the different drivers will be included in IGE 2.1.

Try it and send us your feedback!

The GridWay team.

Request for Requirements from EU Globus Users

Dear GridWay users,

The Initiative for Globus in Europe (IGE) is an EU project aimed at meeting the needs of the European Globus Community. As such, we are very interested in hearing about any requirements that you may have regarding Globus technologies, so we can consider them in our future development cycles. We’re here to help you, so if you need something from Globus, this is your chance to let us know!

If you’re aware of others using Globus (perhaps you know they have requirements?), please feel free to forward this message accordingly.

These requirements may originate within your projects or within your wider research communities, and might include (but are not limited to) requests for new features, enhancements of existing features or requests for bug fixes.

If you have any requirements for Globus feel free to register them through our Community Requirements page at http://www.ige-project.eu/hub/rt, or alternatively, email support@ige-project.eu. In either case we’ll get back to you.

We look forward to hearing from you!

Best Regards,

The Initiative for Globus in Europe

GridWay 5.8 released!

A new stable release of the GridWay Metascheduler is available for download.

This release comes with a new installation procedure, improvements in job polling, logging with syslog, forwarding of resource requirements to LRMSs, automatic disposal of jobs, a new execution driver for CREAM (Computing Resource Execution And Management) as used in gLite, a technology preview of an execution driver for OGSA-BES (Open Grid Services Architecture-Basic Execution Service) and, as always, some bug fixing. You can find more information in the release notes.

This stable release is based on a previous development one, 5.7, which has been thoroughly tested. It has been funded by the IGE project (Initiative for Globus in Europe), which aims to support and enhance the provision of Globus Grid software to European e-Infrastructures, like EGI (European Grid Infrastructure). Packages for the GridWay core and its GT5 drivers are included in the IGE Tech Preview.

IGE logo

Enjoy it! Your feedback is more than welcome.

The GridWay team.

Newly released Globus Toolkit 4.2 includes GridWay 5.4!

Globus Toolkit logo

Recently, Charles Bacon announced, on behalf of the Globus Toolkit development team, the release of Globus Toolkit 4.2, containing an upgrade to the web services specifications used by the toolkit as well as new features in all services.

Starting from Globus Toolkit 4.0.5, GridWay 5.2 was included as a contribution for the GT4.0 final distribution, but contributions are not supported by Globus Toolkit and have very limited documentation. The new Globus Toolkit 4.2 includes a new stable release, GridWay 5.4, as a true Globus component, well documented and fully integrated in Globus installation, building and testing procedures.

This new stable release of GridWay is the result of the previous development release, GridWay 5.3, which was released on December 2007 and has been thoroughly tested and documented since then. In few days, GridWay 5.4 will be also available from GridWay’s web page.

RELEVANT LINKS

Grid Interoperability and Interoperation

The high expectations raised by grid computing have favored the development and deployment of a growing number of grid infrastructures and middlewares. However, the interaction between these grids is still limited, so reducing the potential large-scale application of grid technology, in spite of efforts made by grid community. In this sense, the Open Grid Forum (OGF) is developing open standards for grid software interoperability, while the OGF’s Grid Interoperation Now Community Group (GIN-CG) is coordinating a set of interoperation efforts among production grids. It is therefore clear that, according to OGF (as Laurence Field explains in his article entitled “Getting Grids to work together: interoperation is key to sharing”), there is a big difference between these two terms:

  • Interoperability is the native ability of grids and grid technologies to interact directly via common open standards.
  • Interoperation is a set of techniques to get production grid infrastructures to work together in the short term.

Since most common open standards to provide grid interoperability are still being defined and only a few have been consolidated, grid interoperation techniques, like adapters and gateways, are needed. An adapter is, according to different dictionaries of computer terms, “a device that allows one system to connect to and work with another”. On the other hand, a gateway is conceptually similar to an adapter, but it is implemented as an independent service, acting as a bridge between two systems. The main drawback of adapters is that grid middleware or tools must be modified to insert the adapters. Gateways can be accessed without changes on grid middleware or tools, but they can become a single point of failure or a scalability bottleneck.

GridWay provides support for some of the few established standards like DRMAA, JSDL or WSRF to achieve interoperability but, in the meanwhile, it also provides components to allow interoperation, like Middleware Access Drivers (MADs) acting as adapters for different grid services, and the GridGateWay, which is a WSRF GRAM service encapsulating an instance of GridWay, thus providing a gateway for resource management services.

GridWay 4.0.2, coinciding with the release of Globus Toolkit 4 and its new WS GRAM service, introduced an architecture for the execution manager module based on a MAD (Middleware Access Driver) to interface several grid execution services, like pre-WS GRAM and WS GRAM, even simultaneously. That architecture was presented in the paper entitled “A modular meta-scheduling architecture for interfacing with pre-WS and WS Grid resource management services” (E. Huedo, R. S. Montero and I. M. Llorente). GridWay 5.0 took advantage of this modular architecture to implement an information manager module with a MAD to interface several grid information services, and a transfer manager module with a MAD to interface several grid data services. Moreover, the scheduling process was decoupled from the dispatch manager through the use of an external and selectable scheduler module.

GridWay components

The resulting architecture, which is shown above, provides direct interoperation between different middleware stacks. In fact, we demonstrated at OGF22 the interoperation of three important grid infrastructures, namely EGEE (gLite-based), TeraGrid and OSG (both Globus-based), being coordinately used through a single GridWay instance by means of the appropriate adapters. To set an example, the application was written using the DRMAA OGF standard. GridWay documentation provides a lot of information on how to integrate GridWay in the main middleware stacks, like gLite, pre-WS and WS Globus, or ARC, and provides information on how to develop new drivers for other middlewares.

OGF22 interoperation demo

Regarding the GridGateWay, it is being used for provisioning resources from several infrastructures. For example, the German Astronomy Community Grid (GACG or AstroGrid-D) uses a GridGateWay as a central resource broker, providing metascheduling functionality to Globus-based submission tools (e.g. for workflow execution) without modification. GridAustralia also uses a GridGateWay as a WSRF interface for its central GridWay Metascheduler instance, allowing reliable, remote job submission.

 

 

 

Astrogrid-D metascheduling architecture
Picture by AstroGrid-D

More information about the GridGateWay component is provided in its web page, as well as in this blog entry, which shows how to build Utility Computing infrastructures with this Globus-based gateway technology.

Eduardo Huedo

Grid Gateways: Building Utility Computing Infrastructures with Globus

While research institutions are interested in Partner Grids that provide access to a higher computing performance to satisfy peak demands and support to face collaborative projects; enterprises understand grid computing as a way to address the changing service needs in an organization. They are interested in in-house resource sharing, to achieve a better return from their information technology investment, supplemented by outsourced resources, to satisfy peak or unusual demands. An Outsourced/Utility Grid would provide pay-per-use computational power when Enterprise Grid resources are overloaded. Such hierarchical grid organization may be extended recursively to federate a higher number of Partner or Outsourced Grid infrastructures with consumer/provider relationships. This would allow supplying resources on demand, making resource provision more agile and adaptive. It would offer, therefore, access to a potentially unlimited computational capacity, causing IT costs to transform from fixed to variable.

 fetch.png

In the context of the GridWay project we have developed a Grid Gateway that exposes a WSRF interface to a metascheduling instance, so enabling the creation of hierarchical grid structures. GridGateWay consists of a set of Globus services hosting a GridWay Metascheduler, thus providing a uniform, standard interface for the secure and reliable submission, monitoring and control of jobs. Most functionality is provided through GRAM (Grid Resource Allocation and Management), while scheduling information is provided through MDS (Monitoring and Discovery Service). The security requirement at the user level is addressed by GSI (Globus Security Infrastructure).

The new technology allows different layers of metaschedulers to be arranged in a hierarchical structure. In this arrangement, each target grid is handled as another resource, that is, the underlying grid is characterized as a single resource in the source grid, by means of grid gateways. This strategy encourages companies to federate their grids in order to have a better return of IT investment, and also satisfy peak demands of computation. Furthermore, this solution allows for gradual deployment (from fully in-house to fully outsourced), in order to deal with the obstacles for grid technology adoption, such as enterprise scepticism and IT staff resistance.

This approach also provides the components required for interoperability between existing Grid infrastructures. It is clear that we can’t wait for a single global grid to arise or to become predominant. Instead, we should work to build a seamless integration of the existing grids, which may eventually constitute the ultimate, capital-letter Grid, Grid of grids, or InterGrid, in the same way that the Internet was born. Grid interoperability can be achieved by means of common, ideally standard, grid interfaces, whose existence is an important (if not essential) characteristic of grid systems. Unfortunately, common interfaces (and even less standard ones) are not always available for given services. Then, the use of grid adapters and gateways becomes necessary. In particular, an interoperability solution based on grid gateways provides the infrastructures with significant benefits in terms of autonomy, scalability, deployment and security.

Well, what are you waiting for?, components are open-source, license is Apache v2.0, and we are willing to collaborate with you.

Ignacio Martin Llorente