Category Archives: cloud

CloudMix: Generating Diverse and Reducible Workloads for Cloud Systems

Our latest contribution to the 10th IEEE International Conference Cloud Computing (CLOUD 2017) is available online and can be accessed here.

The prosperity of cloud computing offers common infrastructures to a wide range of applications. Understanding these applications’ workload behaviors is the premise of designing, managing, and optimizing cloud systems. Considering the heterogeneity and diversity of cloud workloads, for the sake of fairness, cloud benchmarks must be able to accurately replicate their behaviors in cloud systems, including both the usages of cloud resources and the microarchitectural behaviors beyond the virtualization layer. Furthermore, workloads spanning long durations are usually required to achieve representativeness in evaluation. Hence the more challenging issue is to significantly reduce the evaluation duration while still preserving their workload characteristics.

This paper presents our efforts towards generating cloud workloads of diverse behaviors and reducible durations. Our benchmark tool, CloudMix, employs a repository of reducible workload blocks (RWBs) as the high level abstraction of workload behaviors, including usages of the two most important cloud resources (CPU and memory) and their pairing microarchitectural operations. CloudMix further introduces an efficient methodology to combine RWBs to synthesize and replicate diverse cloud workloads in real-world traces. The effectiveness of CloudMix is demonstrated by generating a variety of reducible workloads according to a Google cluster trace and by applying these workloads in job scheduling optimization on Hadoop YARN. The evaluation results show: (i) when the workload durations are reduced by 100 times, the replication errors of workload behaviors are smaller than 2.08%; (ii) when providing fast evaluations (workload durations are reduced by 10 to 100 times) to recommend the optimal setting in YARN job scheduling, the performance degradation in the recommended setting is just 0.69% compared to that of the actual optimal setting.

J.L. Vázquez-Poletti

Performance study of a signal-extraction algorithm using different parallelisation strategies for the Cherenkov Telescope Array’s real-time-analysis software

Concurrency and Computation: Practice and Experience just published our latest work on parallelisation strategies in the context of the the Cherenkov Telescope Array project. This is a result of an ongoing collaboration with CIEMAT (Spain) and INAF (Italy) and it can be accessed here.

In this work, a signal-extraction algorithm pertaining to the Cherenkov Telescope Array’s real-time-analysis pipeline has been parallelised using SSE, POSIX Threads and CUDA. Because of the observatory’s constraints, the online analysis has to be conducted on site, on hardware located at the telescopes, and compels a search for efficient computing solutions to handle the huge amount of measured data. This work is framed in a series of studies which benchmark several algorithms of the real-time-analysis pipeline on different architectures to gain an insight into the suitability and performance of each platform.

J.L. Vázquez-Poletti

Synopsis-Based Approximate Request Processing for Low Latency and Small Correctness Loss in Cloud Online Services

The International Journal of Parallel Programming has just made available online our latest work on approximate request processing in cloud online services. This is the result of our collaboration with the Institute of Computing Technology from the Chinese Academy of Sciences and it can be accessed here.

SARP: Synopsis-Based Approximate Request Processing for Low Latency and Small Correctness Loss in Cloud Online Services

Despite the importance of providing quick responsiveness to user requests for online services, such request processing is very resource expensive when dealing with large-scale service datasets. These often exceed the service providers’ budget when services are deployed on a cloud, in which resources are charged in monetary terms. Providing approximate processing results in request processing is a feasible solution for such problem that trades off result correctness (e.g. prediction or query accuracy) for response time reduction. However, existing techniques in this area either use parts of datasets or skip expensive computations to produce approximate results, thus resulting in large losses in result correctness on a tight resource budget. In this paper, we propose Synopsis-based Approximate Request Processing (SARP), a SARP framework to produce approximate results with small correctness losses even using small amount of resources. To achieve this, SARP conducts computations over synopses, which aggregate the statistical information of the entire service dataset at different approximation levels, based on two key ideas: (1) offline synopsis management that generates and maintains a set of synopses that represent the aggregation information of the dataset at different approximation levels. (2) Online synopsis selection that considers both the current resource allocation and the workload status so as to select the synopsis with the maximal length that can be processed within the required response time. We demonstrate the effectiveness of our approach by testing the recommendation services in e-commerce sites using a large, real-world dataset. Using prediction accuracy as the result correctness metric, the results demonstrate: (i) SARP achieves significant response time reduction with very small correctness losses compared to the exact processing results; (ii) using the same processing time, SARP demonstrates a considerable reduction in correctness loss compared to existing approximation techniques.

J.L. Vázquez-Poletti

A Cloud for Clouds: Weather Research and Forecasting on a Public Cloud Infrastructure

Springer has finally made available our latest paper in collaboration with the Spanish State Meteorological Agency. It can be accessed here.

A Cloud for Clouds: Weather Research and Forecasting on a Public Cloud Infrastructure

The Weather Research & Forecasting (WRF) Model is a high performance computing application used by many worldwide meteorological agencies. Its execution may benefit from the cloud computing paradigm and from public cloud infrastructures in particular, but only if the parameters are chosen wisely. An optimal infrastructure by means of cost can be instantiated for a given deadline, and an optimal infrastructure by means of performance can be instantiated for a given budget. With this in mind, we provide the optimal parameters for the execution of the WRF on a public cloud infrastructure such as Amazon Web Services.

J.L. Vázquez-Poletti

Special Issue on Cloud-based Simulations and Data Analysis

Scientific Programming (Hindawi, JCR:0.559) has just announced the call for papers for a Special Issue on cloud-based simulations and data analysis in which I’m participating as Guest Editor together with Dr. Fabrizio Messina (University of Catania, Catania, Italy) and Dr. Lars Braubach (Nordakademie, Elmshorn, Germany).

Cloud-based Simulations and Data AnalysisIn many areas, including commercial as well as scientific fields, the generation and storage of large amounts of data have become essential. Manufacturing and engineering companies use cloud-based high-performance computing technologies and simulation techniques to model, simulate, and predict behavior of complicated models, involving the preliminary analysis of existing data as well the generation of data during the simulations. Having large amounts of data the following question arises, how can it be efficiently processed? Cloud computing holds the promise of providing elastic computational resources thereby adapting towards the concrete application needs and thus is a promising base technology for such processing techniques. But, cloud computing itself is not the complete solution and in order to exploit its underlying power novel algorithms and techniques have to be conceived.

In this special issue we invite original contributions providing novel ideas towards simulation and data processing in the context of cloud computing approaches. The aim of this special issue is to assemble visions, ideas, experiences, and research achievements in these areas.

Potential topics include, but are not limited to:

  • Techniques for cloud-based simulations
  • Computational Intelligence for cloud-based simulations
  • Service composition for cloud-based simulations
  • Computational Intelligence for data analysis
  • Software architectures for cloud-based simulations
  • Cloud-based data mining
  • Big data analytics for predictive modeling

Authors can submit their manuscripts via the Manuscript Tracking System at

Manuscript Due: Friday, 29 April 2016
First Round of Reviews: Friday, 22 July 2016
Publication Date: Friday, 16 September 2016

J.L. Vázquez-Poletti

Special Issue on High Performance Computing for Big Data

I’m very happy to announce that I’m serving as Guest Editor at the Computers Journal for a Special Issue on “High Performance Computing for Big Data”. The deadline for submissions is March 31st, 2016.


Big Data is, right now one of the hottest topics in computing research. This is because of:

  • the numerous challenges that include (and are not limited to) capture, search, storage, sharing, transfer, representation and privacy of the data;
  • and the wide spectrum of areas covered, that range from Bioinformatics to Space Science, and are a research challenge by themselves.

New technologies and algorithms have emerged from Big Data to efficiently manage and process great quantities of data within reasonable elapsed times. However, there are computing barriers that cannot be crossed without the proper resources.

The many ways that High Performance Computing can be delivered for facing Big Data challenges offer a wide spectrum of research opportunities. From FPGAs to cloud computing, technologies and algorithms can be brought to a whole different level and foster incredible insights from massive information repositories.

The papers accepted for publication in this Special Issue cover both fundamental issues and new concepts related to the application of High Performance Computing to the Big Data area.

J.L. Vázquez-Poletti

A multi-dimensional job scheduling

Since march Future Generation Computer Systems has made available (online) our paper entitled “A multi-dimensional job scheduling”. This work is the result of a collaboration with the research group led by Prof. Lucio Grandinetti (University of Calabria, Italy) and it can be accessed here.

Future Generation Computer SystemsWith the advent of new computing technologies, such as cloud computing and contemporary parallel processing systems, the building blocks of computing systems have become multi-dimensional. Traditional scheduling systems based on a single-resource optimization, like processors, fail to provide near optimal solutions. The efficient use of new computing systems depends on the efficient use of several resource dimensions. Thus, the scheduling systems have to fully use all resources. In this paper, we address the problem of multi-resource scheduling via multi-capacity bin-packing. We propose the application of multi-capacity-aware resource scheduling at host selection layer and queuing mechanism layer of a scheduling system. The experimental results demonstrate performance improvements of scheduling in terms of waittime and slowdown metrics.

J.L. Vázquez-Poletti

Special Issue on HPC for Advanced Modeling and Simulation of Materials

At the beginning of 2016 the Journal of Computer Physics Communications (Elsevier, JCR:3.122, Q1) will close a Special Issue in which I’m very honored to serve as Guest Editor. You may be interested in the following Call for Papers.

ComputerPhysicsCommunicationsSupercomputers are rapidly evolving as advances in architecture and semiconductor technology. High performance computing has been applied to accelerate the advanced modeling and simulation of materials. The current trend will provide challenges in parallelism because of increased processing units, accelerators, complex hierarchical memory systems, interconnection networks, storage and uncertainties in programming models. The interdisciplinary collaboration is becoming more and more important in high performance computation. Realistic material modeling and simulation need to combine material modeling methods, mathematical models, parallel algorithms and tools for exploiting supercomputers effectively.



Topics include but are not limited to:

  • Numerical methods and parallel algorithms for the advanced modeling and simulation of materials
  • Use of hardware accelerators (MIC, GPUs, FPGA) and heterogeneous hardware in computational material science
  • Mathematical modeling and high performance computing tools in large-scale material simulation
  • Programming model for material algorithm scalability and resilience
  • Visualization on material data
  • Multi-scale modeling and simulation in materials science
  • Performance modeling and auto-tuning methods in material simulation
  • Big data of materials science
  • Accelerate dissipative particle dynamics by hardware accelerators
  • Large-scale material modeling based on the new features of message passing programming model

Submission Format and Guideline

All submitted papers must be clearly written in excellent English and contain only original work, which has not been published by or is currently under review for any other journal or conference. Papers must not exceed 25 pages (one-column, at least 11pt fonts) including figures, tables, and references. A detailed submission guideline is available as “Guide to Authors” at:

All manuscripts and any supplementary material should be submitted through Elsevier Editorial System (EES). The authors must select as “SI: CPC_HPCME 2015” when they reach the “Article Type” step in the submission process. The EES website is located at:

All papers will be peer-reviewed by three independent reviewers. Requests for additional information should be addressed to the guest editors.

Editors in Chief

N. Stanley. Scott

Guest Editors

Fei Gao
Jose Luis Vazquez-Poletti
Jianjiang Li
Jue Wang

Important dates

Submission deadline: 2016.1.20
Acceptance deadline: 2016.4.20
Publication: 2016.6.20

J.L. Vázquez-Poletti

Interference-Aware Component Scheduling for Reducing Tail Latency in Cloud Interactive Services

At the end of June one of our recent works in collaboration with the Institute of Computing Technology from the Chinese Academy of Sciences was presented at the 35th IEEE International Conference on Distributed Computing systems (ICDS 2015), which took place at Columbus (Ohio, USA). The paper can be accessed here.

ICDCS2015Large-scale interactive services usually divide requests into multiple sub-requests and distribute them to a large number of server components for parallel execution. Hence the tail latency (i.e. the slowest component’s latency) of these components determines the overall service latency. On a cloud platform, each component shares and competes node resources such as caches and I/O bandwidths with its co-located jobs, hence inevitably suffering from their performance interference.

In this paper, we study the short-running jobs in a 12k-node Google cluster to illustrate the dynamic resource demands of these jobs, resulting in both individual components’ latency variability over time and across different nodes and hence posing a major challenge to maintain low tail latency. Given this motivation, this paper introduces a dynamic and interference-aware scheduler for large-scale, parallel cloud services. At each scheduling interval, it collects workload and resource contention information of a running service, and predicts both the component latency on different nodes and the overall service performance. Based on the predicted performance, the scheduler identifies straggling components and conducts near-optimal component-node allocations to adapt to the changing workloads and performance interferences. We demonstrate that, using realistic workloads, the proposed approach achieves significant reductions in tail latency compared to the basic approach without scheduling.

J.L. Vázquez-Poletti

Double date at GridKa School 2015

Next week I’ll be in Karlsruhe (Germany) for this year’s GridKa School edition. This event takes place in Karlsruhe Institute of Technology since 2003.

gridka_logoIf two years ago I gave a talk, this time my participation will be double.

Talk: From Mars to Earth through Cloud Computing

Our society has benefited from Space exploration in many ways. Many of the inventions we use nowadays have their origin in or have been improved by Space research. Computer Science is not an exception.


This talk will introduce the application of Cloud Computing done by me in the context of different Mars missions: Mars MetNet (Spain-Russia-Finland), MSL Curiosity (NASA) and ExoMars2016 (ESA). The achieved know-how allowed the optimization of other areas on Planet Earth, such as weather forecast and agricultural wireless sensor networks processing.

Tutorial: HPCCloud 101, HPC on Cloud Computing for newcomers

Never been into Cloud Computing before? Do you think that an extra computing power is crucial for your research? Do you have some neat parallel codes that your institution doesn’t allow you to execute because the cluster is full? Maybe this tutorial is for you!

HPC Cloud 101The tutorial will cover the following topics:

OpenNebulaAs Virtual Clusters deployed by StarCluster have Sun Grid Engine and OpenMPI installed you are more than welcome to bring your own codes and give them a try!

J.L. Vázquez-Poletti