
\documentclass{acm_proc_article-sp}

%\usepackage{usenix}
%\documentclass[letterpaper,twocolumn,10pt]{article}
\usepackage{amsmath}
\usepackage[usenames,dvipsnames]{color}
\usepackage{latexsym}
\usepackage{epsfig}
\usepackage{xspace}
\usepackage{ifthen}
\usepackage{verbatim}
\usepackage{amssymb}
\usepackage{flushend}
\usepackage{graphicx} 
\usepackage{url}
\usepackage{subfigure}
\usepackage{cite}
\usepackage{endnotes}
\usepackage{attrib}
% \usepackage[small,compact]{titlesec}
\usepackage[normalmargins,normalindent,normalfloats,normalleading]{savetrees}
\usepackage[bookmarks,colorlinks=true, 
  pdftitle={Rise of RaaS: the Resource-as-a-Service Cloud},
  pdfauthor={Agmon Ben-Yehuda, Orna and Ben-Yehuda, Muli and 
Schuster, Assaf and Tsafrir, Dan},
  pdfsubject={CACM},
  urlcolor=blue,
  citecolor=blue]{hyperref}
\input{microtype}

\let\oldfootnotesize\footnotesize
\let\footnotesize\scriptsize

\let\oldbibliography\thebibliography
\renewcommand{\thebibliography}[1]{%
  \oldbibliography{#1}%
  \setlength{\itemsep}{0pt}%
}
\newcommand{\csemail}[1] {\small{\textrm{{#1}@cs.technion.ac.il}}}

\begin{document}

\newcommand{\nom}{\texttt{nom}\xspace}

\setlength{\marginparwidth}{0.8in}
\newcommand{\note}[1]{%
  \marginpar{\colorbox{yellow}{\parbox{1.5cm}{%
  \textcolor{red}{\tiny{#1}}}}}}


\date{}
\title{\Large \bf Rise of RaaS: the Resource-as-a-Service Cloud}


\numberofauthors{1} %  in this sample file, there are a *total*
% of EIGHT authors. SIX appear on the 'first-page' (for formatting
% reasons) and the remaining two appear in the \additionalauthors section.

%for single author (just remove % characters)
\author{
Orna Agmon Ben-Yehuda\\
\affaddr{Technion}\\% -- Israel Institute of Technology\\
\csemail{ladypine}
\and
Muli Ben-Yehuda\\
\affaddr{Technion \& IBM Research}\\
\csemail{muli}
\and
Assaf Schuster\\
\affaddr{Technion}\\% -- Israel Institute of Technology\\
\csemail{assaf}
% copy the following lines to add more authors
% \and
% {\rm Name}\\
%Name Institution
\and
Dan Tsafrir\\
\affaddr{Technion}\\% -- Israel Institute of Technology\\
\csemail{dan}
} % end author

%\author{
%\begin{tabular}{cccc}
%{\rm Orna Agmon Ben-Yehuda} & {\rm Muli Ben-Yehuda} & {\rm Assaf
%Schuster} & {\rm Dan Tsafrir}\\
%\multicolumn{4}{c}{\vspace{\parskip}\it Technion---Israel Institute of Technology}\\
%\multicolumn{4}{c}{\vspace{\parskip}\it \{ladypine,muli,assaf,dan\}@cs.technion.ac.il}
%\end{tabular}
%}

\maketitle
%\thispagestyle{empty}

%\subsection*{Abstract}
\begin{abstract}
Over the next few years, a new model of buying and selling cloud
computing resources will evolve. Instead of providers exclusively
selling server-equivalent virtual machines for relatively long periods
of time (as done in today's IaaS clouds), they will increasingly sell
clients individual resources (such as CPU, memory, and I/O resources)
for a few seconds at a time. Instead of allocating resources on a
first-come first-served basis, they will do so in accordance with
market-driven supply-and-demand conditions. We term this nascent
economic model of cloud computing the Resource-as-a-Service (RaaS)
cloud, and we argue that its rise is the likely culmination of recent
trends in the construction of Infrastructure-as-a-Service (IaaS)
clouds and of the economic forces operating on both providers and
clients.
\end{abstract}
%
%Over the next few years, a new model of buying and selling cloud
%computing resources will evolve. Instead of providers selling server
%equivalent virtual machines, as done in today's IaaS clouds,  
%providers will continuously sell clients 
%individual resources (such as 
%
%Over the next few years, a transition will occur in how cloud
%computing resources are bought and sold. From providers selling server
%equivalent virtual machines, as done in today's IaaS clouds, to
%providers continuously selling clients individual resources (such as
%
%CPU, memory, and I/O resources) for a few seconds at a time. We term
%this nascent economic model of cloud computing the
%Resource-as-a-Service (RaaS) cloud. We argue that its rise is the
%likely culmination of recent trends in the construction of IaaS clouds
%and of the economic forces operating on both providers and clients.

%\begin{quote} 
%\textit{The single most important proposition in economic theory,
%first stated by Adam Smith, is that competitive markets do a good job
%allocating resources.} (Stephen LeRoy)
%\end{quote}

\begin{quote}
When the quantity of any commodity which is brought to market
falls short of the effectual demand, [$\ldots$] some [$\ldots$] will
be willing to give more. \attrib{Adam Smith, \textit{An Inquiry into
    the Nature and Causes of the Wealth of Nations}, \S1.7.9}
\end{quote}

\category{K.4.4}{Computing Milieux}{Computers and Society}[Electronic Commerce]
\terms{Economics, Human Factors, Design}
\keywords{cloud, trends} % NOT required for Proceedings

\section{Introduction}
%cloud benefits? How much does it cost? what is the market?

Cloud computing is taking the computer world by storm. Today,
Infrastructure-as-a-Service (IaaS) clouds, such as Amazon EC2, allow anyone
with a credit card to tap into a 
seemingly unlimited fountain of
computing resources by renting virtual machines for several cents or
dollars per hour. According to a recent report by Forrester
Research~\cite{ried11}, the
yearly cloud computing market is expected to top \$241 billion in 2020,
compared to \$40.7 billion in 2010, a sixfold increase.

What will these 2020 clouds look like? Given the current pace of
innovation in cloud computing, substantial shifts are bound to occur
in how providers design, operate and sell cloud computing resources,
and in how clients purchase and use those resources.

A transition is beginning in Infrastructure-as-a-Service clouds, from
providers selling bundles of resources packaged as server-equivalent
virtual machines 
%(e.g., Amazon EC2 selling a ``virtual machine roughly
%equivalent to a server with 2-CPU Xeon processors and 2 GBs of
%memory'') 
to providers continuously selling their clients individual
computing, memory, and I/O resources for a few seconds at a time. 
We
call this model of cloud computing the \emph{Resource-as-a-Service
  (RaaS)} cloud model. In a RaaS cloud, 
provider-governed economic mechanisms control clients'
access to resources. Hence, clients deploy economic agents that 
continuously buy and sell computing
resources in accordance with the provider's current supplies and the clients'
current demands.

We begin with an overview of three existing trends in the
construction, operation, and use of IaaS cloud computing platforms
that underlie this transition: the shrinking duration of rental
periods (Section~\ref{sec:duration-of-rent}), the increasingly
fine-grained resources offered for sale
(Section~\ref{sec:resource-granularity}), the increasingly meaningful
pricing of resources (Section~\ref{sec:pricing}), and the provisioning of
useful service level agreements (SLAs)
(Section~\ref{sec:slas}). Following each trend to its culmination, and
taking into account the economic forces operating on both clients and
providers (Section~\ref{sec:economy}), we describe the RaaS cloud
(Section~\ref{sec:raas}). We conclude by outlining the challenges and
opportunities that the RaaS cloud presents
(Section~\ref{sec:conclude}).

\section{Recent IaaS Trends}
\subsection{Duration of Rent}
\label{sec:duration-of-rent}

Before cloud computing, the average useful lifetime of a purchased
server was approximately three years. With the advent of Web hosting,
clients could rent a server on a monthly basis. With the introduction
of on-demand EC2 instances, Amazon radically changed the time
granularity of server rental, making it possible to rent a server
equivalent for as little as one hour. This 
trend---of renting server-equivalents for increasingly shorter time durations---is driven by
economic forces that keep pushing clients to improve efficiency and
minimize waste: if a partial instance-hour is billed as a full hour, 
%wording from EC2
you might waste up to an hour over the lifetime of every virtual
machine. If a partial instance-second is billed as a full second, then you
are only liable to waste up to a second over the lifetime of every virtual
machine. Thus, shorter durations of rent open the cloud for business
for shorter workloads. Notably, they encourage \emph{horizontal elasticity}:
clients who need a burst of compute power (many machines for a short
time) may settle for using fewer machines for longer times, or even
using only self-owned hardware, to
minimize the per-machine penalty. Such clients are more likely to use
the cloud when the penalty per machine is made
smaller. 

This trend is rapidly gaining ground, as some providers are already selling
virtual machines for less than an hour: Amazon spot-instance prices
change as often as every five minutes~\cite{agmonben-yehuda11}.
CloudSigma\endnote{\url{http://www.cloudsigma.com}} announces
new prices once every five minutes. New providers use even finer time
granularity: 
Gridspot\endnote{\url{http://gridspot.com}} charges by three-minute chunks,
and ProfitBricks\endnote{\url{http://www.profitbricks.com}} charges by the minute.

We expect this trend to continue; eventually, cloud providers
will sell computing
resources for seconds at a time. Such durations are consistent with
peak demands that can change over seconds when a site is
``slashdotted'' (linked from a high-profile Web site).\endnote{``Fifty
percent of the time the
  site is down in seconds---even when we've contacted site owners and
  they've told us everything will be fine. It's often an unprecedented
  amount of traffic, and they don't have the required
  capacity.''--Stephen Fry,
  \url{http://tinyurl.com/StephenFrySeconds}.}

We draw an analogy between cloud providers and phone companies, which have
progressed over the years from charging landlines per several minutes
to charging cell-phones by the minute, and then, due to customer
pressure, to charging per several seconds and even per
second. Similarly, car rental (by the day) is also giving way to car sharing (by the hour).

%Such durations are short enough that
%waste is minimized but long enough so that the resulting overhead from
%the provider's metering, accounting and billing for different
%resources, and the clients' overhead of adapting to changing resource
%conditions, is also minimal.\note{minimal is a strong unbacked claim}

\subsection{Resource Granularity}
\label{sec:resource-granularity}

In most IaaS clouds, clients rent resources as a fixed bundle of
compute, memory, and I/O resources. Amazon and
Rackspace\endnote{\url{http://www.rackspace.com/cloud/public/servers/techdetails/}}
call these bundles
``instance types'' and GoGrid\endnote{\url{http://www.gogrid.com}}
calls them ``server sizes.'' Selling resources this way provides
clients with a familiar abstraction of a server-equivalent.  This
abstraction, however, is starting to unravel. Amazon already allows
clients to add and remove different ``network instances'' and ``block
instances'' from running virtual machines, thereby dynamically
increasing or decreasing the I/O resources available to a virtual
machine. Amazon also allows clients to provision the desired rate of
I/O operations per second on a per-block-instance basis.\endnote{\url{
%http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/AmazonEBS.html#
%EBSVolumeTypes}}
http://aws.amazon.com/ebs/}}
CloudSigma, Gridspot, and ProfitBricks offer clients the
ability to compose a flexible
bundle from varying amounts of resources, similar to building a
custom-made server out of different mixtures of resources such as
CPUs, memory, and I/O
devices. 
%In the PaaS model,
%Dotcloud\endnote{\url{https://www.dotcloud.com/pricing.html}} 
%offers flexible bundles with memory granularity of 32Mb.

Renting a fixed combination of cloud resources cannot and
does not reflect the interests of clients.  First, as server size is
likely to continue to increase---hundreds of cores and
hundreds of gigabytes of memory per server in a few years---an entire
server-equivalent may be too large for some customer needs. Second,
selling a fixed combination of resources is only efficient when the
load customers need to handle is both known in advance and remains
strictly constant. As neither condition is likely, the ability to
dynamically mix-and-match different amounts of compute, memory, and
I/O resources would probably be highly valued by clients.

By extrapolation, compute, memory, and I/O resources will be rented
and charged for in dynamically changing
%\note{what is the difference between flexible and constantly
%changing?}
amounts and not in fixed bundles. Clients will buy seed virtual
machines with some initial amount of resources, and will then deploy
an economic agent (described in Section~\ref{sec:raas}) to buy or sell
additional resources. The economic agent will make decisions based on
the current prices of those resources, the changing load the machine
should handle, and the client's subjective valuation of those
different resources at different points in time.

%% Another technological trend is accounting. Traditionally (reference to
%% IBM big machines from the 60s???), CPU-time accounting is performed a
%% lot for time sharing, even within an operating
%% system. Cgroups(reference) allows capping different resources, and
%% when it is used inside KVM~\cite{kivity07}, it can cap resource use of
%% virtual machines. With energy awareness, physical meters are
%% introduced to hardware (references...). We extrapolate that accounting
%% of a guest VM's usage of physical resources (such as memory, CPU, IO)
%% will be available within 10 years.

%\subsection{Service-level Agreements}
\subsection{Meaningful Resource Pricing}
%\subsection{Selling Resources, Not Performance}
\label{sec:pricing}


 %SaaS: They further guarantee the performance of their load
%balancing service in terms of throughput, latency, concurrent
%connections and transaction rate.
%Some
%providers place the burden of proof on the client by demanding that
%the client prove it suffered any downtime; not all of them supply the
%means to do this. 

The usual SLAs in use today state that the provider provides the client
virtual machines with resources (e.g., RAM, storage, CPU, I/O) 
equivalent to servers of certain
sizes. Virtualization, however, is beneficial when at least some
resources are shared (e.g., heat sink, bus, last-level cache), and
others are time shared (e.g., when a fraction of a CPU is rented, or
physical memory is overcommited).
The performance of a given 
virtual machine can vary wildly at different times due to 
interference caused by other virtual
machines that share resources whose use is not measured and 
allocated~\cite{verma08,nathuji10,gupta08}. Moreover, this interference depends on the
activity of all the virtual machines involved, and is not easily
quantified in a live environment. Thus, there is a discrepancy
between what providers 
provide and what clients would actually like: in practice, what
clients care about is their virtual machines' subjective performance.

To bridge this discrepancy, researchers proposed to base the SLA on client
performance instead of consumed
resources~\cite{nathuji10,heo09,padala09,baset12}. This approach is only
applicable where performance is well defined, and where the
 client applications are fully visible to the provider (as is the case in 
Software-as-a-Service (SaaS) and Platform-as-a-Service (PaaS) clouds),
or the client virtual machines fully cooperate with the provider, 
as may be the case in private IaaS clouds. 
However, the IaaS cloud providers and clients are separate economic
entities. In general, they do not trust each other, and do not
cooperate without a good reason. 
Hence, guaranteeing client performance levels is not applicable
to a public IaaS cloud, where the impact of allocated
resources on performance is individual, where the very definition
of performance is individual, where client virtual
machines are opaque, and where the provider cannot rely on clients
to tell the truth with regard to their desired and achieved
performance. If the provider guarantees a certain performance level, it
is in the client's interest to claim the performance is still too low,
so that the provider will add resources.

Therefore, public clouds will have to forsake the approach of
charging users a predefined sum for resource bundles of
unknown performance. They will either have to precisely measure
all the system's resources to
quantify the interference between virtual machines, 
or switch to a market-driven model. 
%
A market-driven model is based on how clients value the few monitored
resources. Clients would need to develop a model for their value
of a smaller number of monitored resources, already
factoring the impact of interference with other non-monitored resources.
%This will include subjective factors, such as
%the interference experienced by the virtual machine due to
%non-monitored resources and the monetary tag that the client attaches
%to the virtual machine's performance.
The clients would be willing to pay for each of the few
monitored resources what they believe the resources are worth,  
%according to their valuations,
thus affecting prices. 
Unlike previously proposed models, this economic model can
accommodate real-world, selfish, rational clients.

\subsection{Tiered Service Levels}
\label{sec:slas}

Tiered service, where different clients get different levels of
service, can be found in certain scientific grids. Jobs of clients
with low privileges may be preempted (aborted or suspended) by jobs of
clients with higher privileges.  Although the first clouds did not
offer such prioritized service but rather supplied service at only one
fixed level, Amazon has since introduced different priority levels
within EC2. Reserved and on-demand instances provide the two highest
priority levels. Spot instances provide a continuum of lower service
levels, since Amazon prioritizes the client's spot instances according to
the bid price stated by each client. As in grids, these priorities are
relative, so it is hard to explicitly define their meaning in terms of
absolute availability. For example, the availability of on-demand
instances depends on the demand for reserved instances. The PaaS
provider Dotcloud\endnote{\url{https://www.dotcloud.com/pricing.html}}
also offers two SLA levels, one for paying customers and the other for
customers experimenting with the service for free.

Having clients with different priorities is useful to the provider,
who can provide high-priority clients with elasticity and availability
at the expense of lower-priority clients, while simultaneously renting
out currently-spare resources to low-priority clients when
high-priority clients do not need them. Likewise, different priorities
allow budget-constrained cloud clients cheap access to computing
resources with poorer availability.  Mixing clients of different
relative priorities
will allow the providers to simultaneously achieve high resource
utilization and maintain adequate spare capacity for handling sudden
loads.

Extrapolating from the progression of SLA terms we have seen to-date,
we expect that in the RaaS cloud clients will be able to define their
own priority level
%---their own SLA---
individually, choosing from a
continuum of availability levels priced relatively to other levels. As
client needs and resource consumption change over time, clients will
also want to change their priority level over time, just as EC2 spot
instance clients can now change their bid over the lifetime of the instance.

Service levels may be expressed in relative or absolute
terms. The existing examples of tiered SLAs, as provided by GridSpot
and Amazon EC2 spot instances, take the relative approach, providing
tiered service levels via relative prioritization between
clients. Nonetheless, the prevalent SLA language---''unavailability of
a minimal period $X$, which is at least a fraction $Y$ of a service
period $Z$''---could be extended to express different absolute levels
by controlling the parameters $X$,$Y$,$Z$~\cite{baset12}.

% For example, a private Web site may settle for availability of 97\%
% of the time instead of the 99.9\% that a commercial site requires.
% Alternatively, the private site will allow for 3\% of downtime each
% month, agreeing to be out of service for a full day.  The commercial
% site will only allow for 3\% of downtime out of each hour, suffering
% only two minutes of continuous downtime.

\section{Economic Forces}
\label{sec:economy}
In the previous section, we surveyed several ongoing trends and tried
to surmise where they will lead us next. We now survey the economic
forces already operating on both clients and providers. These forces
not only explain the phenomena discussed in the previous section but
are likely to cause them to continue well into the foreseeable
future, culminating in the emergence of the RaaS cloud.

\subsection{Forces Acting on Clients}

As clients move more of their computing capacity to the cloud, they
seek to reduce their costs. Since load and desired performance
constantly change, clients seek to pay only for the resources
they need, and only when they need them. The more flexible the
provider offerings, the better control clients have over their costs
and the resulting performance. 
%Being able to control costs makes it
%possible for clients to act according to their economic incentives;
%these economic incentives drive clients to develop methods for
%efficient resource usage. 
As providers offer increasingly fine-grained
resources and service levels, clients are able to develop methods for
efficient resource usage. As the time scales involved shorten, such
clients grow to rely on computerized agents to negotiate
resources and prices on their behalf.

\subsection{Forces Acting on Providers}

% force 1: commoditization => competition => resource pressure
As cloud computing continues on the road to commoditization and hungry
new providers join the fray, competition increases. As
migration between cloud providers becomes easier,
%~\cite{krieger10},
%no point in citing Oran on his vmware work when VMWARE just announced openstack.
especially in IaaS clouds, providers
are forced to economize, lowering prices while simultaneously
attempting to improve shrinking profit margins. They seek to
maintain the least amount of supply (computing resources) that can
profitably be sold for handling the expected demand (client requests
for resources). When the demand exceeds the provider's expectations,
providers take advantage of the different client priorities to
deprive low-paying clients and sell the limited resources to clients
willing to pay more.

Commoditization (e.g., due to the adoption of OpenStack\endnote{\url{http://openstack.org}} by leading companies such as Rackspace, RedHat, and VMware)
 drives the need to differentiate offers. 
Providers who offer clients the ability to rent resources when they need them, as
they need them, at the right price, 
differentiate their offerings from other providers who only offer
fixed bundles of server-equivalent resources.

Increased competition between providers forces them to become more
efficient. Constant load changes render static resource allocation
inefficient, because as load changes, resources go unused. Providers
therefore benefit from selling their clients the available cycles,
memory, and I/O-bandwidth that the clients need, when they need it, at
market-driven prices, so that no profitable resource goes unused.

Providers who keep guaranteeing absolute QoS levels while clients
change their resource consumption on-the-fly, have to
reserve a conservative amount of headroom for each resource on each
physical server. This headroom---spare resources---is required just in
case all clients simultaneously require all the resources promised
them. Clients who change their resource consumption on-the-fly
do not pay for this headroom unless and until they
need it, so keeping it around all the time is wasteful. Extending 
the current absolute SLA language to several tiers 
only reduces some of 
the headroom. To get rid of the headroom completely, 
providers must resort to prioritization,
which only guarantees clients relative QoS.

% %Best-effort systems usually do not provide QoS guarantees. By
% %contrast, cloud providers do try to sell QoS guarantees. 
% As the
% provider segments its market and offers increasingly fine-grained
% payment levels for increasingly finer-grained bundles (e.g., by
% varying the CPU power or the bandwidth between resource bundles),
% the amount of resources per instance can also be enlarged on-the-fly. 
% Clients expect
% assurance that by paying more they will
% actually get more. 
% %However, client machines that share the same
% %hardware interfere with one another by sharing non-monitored
% %resources. For example, two VMs may share exactly one CPU and get half
% %the cycles each, but the way they use their cache will determine the
% %performance degradation they experience~\cite{verma08}. 
% However, on-the-fly resource changes introduce explicit interference (over the
% implicit interference of unmonitored resources): 
% clients may explicitly expand their monitored
% resources at such a time that causes congestion and interferes with
% other clients' expansion.

%% The current solution in IaaS nowadays is overcommitment of resources
%% while trying to preserve the same performance. The overcommitment
%% relies on the fact that client only partially utilize their
%% resources. When overcommitting resources in large systems with many
%% uncorrelated guests, resource pressure (guests requiring more
%% resources than actually available at the time) is statistically less
%% likely to happen. However, to increase revenue and utilize all
%% resources, the provider needs to squeeze even more guests on the same
%% resources, making the resource pressure problem harder again.

%The inaccuracy of this approach leads to the definition of roughly
%equivalent servers. To squeeze more flexibility out of the system,
%performance degradation must be allowed at times. When actual
%resources are sold for short durations, 
%instead of average performance-equivalent, the provider
%can truthfully sell the resources it actually allocates the
%guest. Thus, resource pressure

%\note{Muli - I know you like the poetic paragraph here, but I am not sure
%it serves a purpose}
%They will seek to extract every
%useful CPU cycle, to keep every DRAM page used, to effectively utilize
%every byte of I/O bandwidth to network, disks and SSDs, and every Watt
%of power. 

%% Hence, they will increase both the time granularity and the
%% resource granularity by which they sell virtual
%% machines. Interference between neighboring guest VMs will increase,
%% and with it the need to charge exactly for what is supplied, and
%% not for a promise which may or may not be fulfilled.

%increase 1 to move start of figure to the right.
%increase 2 to lower figure
%increase 4 to move bottom of figure down
\begin{figure}[t!]
  \includegraphics[scale=0.56]{reasoning-4trends.pdf}
  \caption{Forces behind the transition from the IaaS model, with its inefficient
    use of resources, to the RaaS model, where all
    resources are used efficiently. The forces are the trends (yellow)
    and game theoretic consideration (white). The elements of RaaS are
    shown in blue, and the efficiency results are shown in green.}
  \label{fig:reasoning}
\end{figure}

\section{The RaaS  Cloud} 
\label{sec:raas}

%increase 1 to move start of figure to the right.
%increase 2 to lower figure
%increase 4 to move bottom of figure down
\begin{figure*}[t!]
\centering
%  \includegraphics[viewport=70 120 370 740, scale=0.55]{reasoning-for-poster.pdf}
%  \includegraphics[viewport=-0 020 070 540, scale=0.45]{reasoning-for-poster.pdf}
%  \includegraphics[viewport=-0 020 070 540, scale=0.5]{reasoning-for-poster.eps}
  \includegraphics[scale=0.6]{ginseng-for-raas.pdf}
%  \includegraphics[scale=0.6]{reasoning-for-poster.eps}
  \caption{A RaaS machine, comprising a host and
    several guest virtual machines with their economic entities
    (coordinator and agents, respectively) and communication channels}
  \label{fig:ginseng}
\end{figure*}

In this section we describe how the trends %(fine granularity of
%resources, short rental durations, and actual-resource SLAs with
%varied priorities) 
and economic forces outlined in the previous
sections are transforming today's IaaS clouds into what we call
\emph{Resource-as-a-Service (RaaS) clouds}. The forces
behind this transition are depicted in Figure~\ref{fig:reasoning}.



\subsection{Trading in Fine-Grained Resources}
%todo - the agents are rational, act exactly according to what is
%coded in them, so a market might work, as opposed to dan arielli
%saying that markets do not find the market price cause people are
%irrational, e.g. stamping?
\paragraph{Seed virtual machine}
In RaaS clouds, the client purchases upon admittance a seed virtual
machine. The seed virtual machine has only a minimal initial amount of
dedicated resources. All other resources needed for the efficient
intended operation of the virtual machine are continuously rented.
This combination of resource
rental schemes---prepurchasing and multiple on-demand levels---benefits
the clients with the flexibility of choice. As experiments show, clients who
are presented with such resource rental options tend to make use of the
full range of choices~\cite{altmann01}.

%develop - the first buy then sublet
\paragraph{Fine-grained resources}
The resources available for rent include CPU, RAM, and I/O resources, as
well as emerging resources such as computational accelerators (e.g.,
GPGPUs and FPGAs) and Flash devices.
CPU
capacity is sold on a hardware-thread basis, or even as number of
cycles per unit of time; RAM is sold on the basis of memory frames;
I/O is sold on the basis of subsets of I/O devices with associated I/O bandwidth
and latency guarantees. Such devices include network interfaces and block
interfaces. Accelerators are sold both as I/O devices and as CPUs.
A subset of an I/O device may be presented to the virtual
machine as a direct-assigned SR-IOV Virtual Function(VF)~\cite{gordon12} or as an
emulated~\cite{amit11} or para-virtual
device%~\cite{barham03}. 
Every resource has a dynamically changing
price tag attached to it.
%clients can be price takers or affect the price - they will affect it
%less as the system grows - mention kelly?
%discuss cloudsigma - clients actually affect the price probably, but
%not in a transparent way

%host agent
\paragraph{Host economic coordinator}
To facilitate continuous trading, the provider's cloud software
includes an economic coordinator representing the provider's interests.
This coordinator operates an economic mechanism which defines the resource
allocation and billing mechanism: which
client gets which resources and at what price. Several auctions were
proposed to such ends, e.g., by Kelly~\cite{kelly97}, Chun and Culler~\cite{chun00},
Lazar and Semret~\cite{lazar99}, Waldspurger et
al.~\cite{waldspurger92}, and Lubin et al.~\cite{lubin09}. 
In addition, the coordinator may act as a clearing house and 
support a secondary market of computing
resources inside the physical machine, as
SpotCloud\endnote{\url{http://spotcloud.com}} offers to do for
fixed-bundle virtual machines and as Kash et al.~\cite{kash11} propose
to do for the wireless spectrum.

%Chun and culler~\cite{chun00} used Waldspurger's proportional-share
%resource management~\cite{waldspurger94} to implement a micro-economic
%CPU scheduler. Kelly~\cite{kelly97} proposed and analyzed a
%proportional-share bandwidth allocation. The common preference behind
%these schedulers is simplicity: the assumption that clients can be
%viewed as price takers. Clients do not try or cannot affect the price
%they pay.
%
%Another body of work considers strategic (selfish and rational) 
%clients. Lazar and Semret~\cite{lazar99} propose using a progressive second
%price auction to allocate bandwidth. Waldspurger et
%al.~\cite{waldspurger92} use a second price auction to allocate CPU
%time in the SPAWN system.

%guest agent
\paragraph{Guest economic agent}
To take part in the auction and trade, clients' virtual machines must include
an economic agent. This agent represents the client's business
needs. It rents the
necessary resources---given current requirements, load and costs---at
the best possible prices, from either the provider or its \emph{neighbors}---virtual
machines collocated on the same physical machine. 
When demand outstrips supply, the agent changes its bidding strategy (to the
provider) or negotiates with neighbors' agents,
mediating between the client's requirements and the resources
available in the system, ultimately deciding how much to offer to pay
for each resource at any given time. The layout of a RaaS machine is
illustrated in Figure~\ref{fig:ginseng}.

%subletting, iaas futures, nested
\paragraph{Subletting}
Clients can secure resources early and sublet them later if they no longer
need them. They can sublet to other clients on
the same physical machine using infrastructure provided by the
host's coordinator: the clients agree to redivide resources between them,
inform the coordinator, and the coordinator transfers the local
resources from one guest to another (as Hu et al.~\cite{hu12} do for
bandwidth resources). Resource subletting also lays the ground for
resource futures markets.
In addition, they can create full virtual
machines using nested virtualization~\cite{ben-yehuda10}, on the basis
of excess resources, and sublet them to anyone.
%Such futures markets were contemplated for IaaS
%bundles~\cite{altmann08,vanmechelen11}. 

%legacy clients
\paragraph{Legacy clients}
Legacy clients, without an economic agent, can still function in the
RaaS cloud just as they function in a IaaS cloud. 
For backward compatibility, legacy clients simply rent large seed
machines, which serve as IaaS instances.
%: fixed resource bundles, which are purchased
%upfront at the futures-market~\cite{altmann08,vanmechelen11} price.
%This means legacy clients can still act as if they are in the IaaS cloud. 
These clients can easily verify that they
meet budget constraints, but they are unable to modify their
consumption and thus suffer from either wasted resources or
insufficient resources, just as IaaS clients do.
Therefore, IaaS providers who introduce RaaS capabilities can do so
gradually, offering their clientele new options, and not forcing them
to change their business logic.

%private cloud
\paragraph{Private clouds}
Should the provider and clients all belong to the same economic entity
(e.g., as might happen in a company's private cloud), then the
economic mechanism is not used for actual payments, but still reflects
the relative importance of the different clients and the subjective
costs of resources (electricity, for example). %comment that it would
%be like ginkgo in this case?

\subsection{Prioritized Service Levels}

%priorities for headroom only
\paragraph{Priorities for headroom only}
In the RaaS cloud, the client gets an absolute guarantee (for receiving the
resources and for the price paid) only for its 
minimal consumption, which is constant. Additional resources are
provided on a priority basis in market prices. A risk-averse client
can prepay for a larger amount of constant resources, trading low
costs for peace of mind. From the provider's point of view, the
aggregate constant consumption provides a steady income source. Only 
resources which may go unused (the headroom) are allocated on the basis of a market competition.

%anti robin hood
\paragraph{Vertical elasticity: like Robin Hood, only in reverse}
RaaS clients are offered on-the-fly, fine-grained, fine-timed 
\emph{vertical elasticity} for each
instance: the ability to expand and shrink the resource
consumption of each virtual machine. 
The resources required for this vertical elasticity are limited by
the physical resources contained in a single machine, because 
migrating running virtual machines from
one physical machine to another will likely remain less efficient than
dynamically balancing the available resources between virtual machines
co-existing on the same physical machines. Hence, during peak demand
times,
to enable one client to vertically upscale a virtual machine, 
the additional resources must be taken from a \emph{neighbor}:
another virtual machine on the same physical machine (which might
belong to another client). Instead
of static priorities, in the RaaS cloud providers use the willingness
of clients to pay a certain price for resources at a given moment in
time (a bid) to decide which client gets which resource. Thus market-forces
dictate both the constantly changing prices of resources and their
allocation. In effect, the RaaS cloud provider does the opposite of
Robin Hood: it takes from the poor and gives to the rich.

%the importance of neighbors
%\paragraph{The right mixture of neighbors}
\paragraph{A few good neighbors}
The RaaS virtual machine's vertical elasticity is determined, via a market
mechanism, by its neighbors'  willingness to pay. The neighbors also
determine the cost of the elastic expansion. 
Due to the inherent inefficiencies of live virtual machine migration,
RaaS clouds must include an algorithm for placing client virtual
machines on physical machines. The algorithm should achieve the right
mixture of clients with different SLAs on each physical machine in the
cloud, such that high-priority clients always have low-priority
clients beside them, to provide them with more capacity
when their demands peak. The low-paying clients can use the
high-paying clients' leftover resources when they do not need them,
keeping the provider's machines constantly utilized. Another objective
of the allocation algorithm is to allow the low-priority clients
enough aggregate resources for their needs. A low priority client can
be expected to tolerate a temporary loss of service every so often,
but if the physical resources are strictly smaller than the mean
demand, such a client will never get enough resources to make
meaningful progress. Therefore, to retain the low-priority clients,
the placement algorithm must provide them with enough resources to
make (some) progress.

\paragraph{Far from the Madding crowd}
The RaaS provider also influences the quality of service that the RaaS
client experiences by limiting the \emph{maximal possible aggregate demand} for
physical resources on the machine. Demand can be limited by
controlling the number of virtual machines per physical machine and the maximal
vertical elasticity to which each virtual machine is entitled. When the maximal
  possible aggregate demand is lower than the supply, resources are
wasted, but all virtual machines
can freely expand. As the maximal possible aggregate demand
exceeds the supply, clients will be less likely to succeed in 
vertical expansion when they need it, 
or might be forced to pay more for the same expansion. 
Hence, RaaS clients are willing to pay more to
be hosted in a physical machine with lower maximal possible aggregate
demand. This encourages RaaS providers to expose information about the
aggregate demand and supply on the physical machine to its clients. 
%By
%contrast, under the prevalent SLA language, clients are not exposed to
%such information, and must constantly monitor and benchmark their instances to
%verify their resources are adequate.


\section{Implications, Challenges, Opportunities}
\label{sec:conclude}

The RaaS cloud gives rise to a number of implications, challenges, and
opportunities for both providers and clients, which did not exist in
markets of entire virtual machines~\cite{shneidman05,ortuno10,altmann08,vanmechelen11,rahman11,zaman11}.
Broadly speaking, the new research areas
can be divided into two categories: technical mechanisms and policies.

The RaaS cloud requires new mechanisms for allocating, metering,
charging for, reclaiming, and redistributing CPU, memory and I/O
resources between untrusted, not-necessarily-cooperative clients every
few seconds. These mechanisms must be efficient and
reliable. In particular, they must be resistant to side-channel attacks from
malicious clients~\cite{ristenpart09}. Hardware mechanisms are
especially needed for fine-grained resource metering in the RaaS cloud.

The RaaS cloud requires new system software and new
applications. Usually, current operating systems and applications are
written under the assumptions that their underlying resources are
fixed and always available. In the RaaS cloud, virtual machines never
know the precise amount of resources that will be available to them at
any given second. Thus, the software running in those virtual
machines must adapt to changing resource availability and exploit
whatever resources the software has, when it has them. Assume a client
application that just got an extra 2Gbps of networking bandwidth at a
steal of a price, but only for one second. To use it effectively while
it is available, all the software layers, including the operating system, run-time layer, and application
must be aware of it.

The RaaS cloud requires efficient methods of balancing resources
within a single physical machine, while taking into consideration the
different guaranteed service levels. Bottleneck-resource
allocation~\cite{gutman12,ghodsi11,dolev11} is a step towards
allocation of resource bundles, but it still requires an algorithm for
setting the system share to which each client is entitled.
%Gutman and Nisan~\cite{gutman12}, Dolev et al.~\cite{dolev11} and
%Ghodsi et al.~\cite{ghodsi11}  solve the problem of multi resource
%fair allocation, according to claims to some share of the system.

To allow the resource balancer
different service levels to work with, workload balancers also require
efficient methods of balancing resources across entire cloud
data centers. These balancers
will need to consider the virtual machines' flexibility and SLA in addition to
current considerations (static resource requests only). 

Under dynamic
conditions, the 
intra-machine RaaS mechanisms will quickly respond to flexibility
needs, holding the fort until the slower live migration can take
place. However, live migration must take place to mitigate the resource
pressure on the effectively most stressed machines, 
and allow clients to change their flexibility bounds.
Large IaaS providers apparently manage without live 
migration~\cite{ristenpart09}: the high rate of initialization and
shutdown of virtual machines makes the initial balancer 
effective enough. 
However, the fine time granularity of the changes in the RaaS cloud means live
migration is going to be required more often. Hence, the RaaS cloud
will require efficient methods for live migration of virtual machines 
and for network virtualization. 

On the policy side, the RaaS cloud requires new economic models for
deciding what to allocate, when to allocate it, and at what
prices~\cite{danak10}. 
Ideally, they should optimize the provider's revenue or a social
welfare function: a function of the benefit of all the clients. The
clients may measure their benefit in terms of starvation, latency, or
throughput, but the mechanisms should optimize the impact of those
performance metrics on the welfare of the clients, for example by
maximizing the sum of client benefits or by minimizing the unhappiness
of the most unsatisfied client. 
In game-theoretical terms, these mechanisms
should be incentive compatible (like VCG~\cite{vickrey61,clarke71,groves73}): 
truth telling regarding private
information should be a good course of action for the clients, so that
the provider can easily optimize the resource allocations. 
The
mechanisms should be collusion-resistant: a virtual machine should not
suffer if several of the virtual machines it is co-located with happen
to belong to the same client. Like 
approximation algorithms for multi-unit auctions~\cite{dobzinski10,voecking12}, they should be
computationally efficient at large scale, so that
solving the resource allocation problem does not become prohibitive.
Nisan and Ronen~\cite{nisan07} propose a family of appeal-based
mechanisms which alleviate this difficulty. 
The mechanisms should
preserve the clients' privacy, like McSherry and
Talwar~\cite{mcsherry07}. 
Lazar and Semret~\cite{lazar99} and Maill\'e and Tuffin~\cite{maille04} proposed
feasible mechanisms 
for a single divisible resource that do not preserve any privacy to
reach equilibrium, and thus expose client information to other clients.
%the class of mechanisms proposed by Nissim et al.~\cite{nissim10} do
%preserve it. 
The mechanisms should also
minimize the price-of-anarchy~\cite{koutsoupias99}: the waste incurred
by using a distributed mechanism.

In addition, to work in the real world, the economic mechanisms must
be realistic. 
%realistic function
Realistic clients' willingness to pay for resources is the result of
their measurements of performance, as illustrated in
Figure~\ref{fig:ginseng}. 
The mechanism must support such measured functions, which are not
necessarily mathematically nice and regular (e.g., contain
steps~\cite{parkes12}).
%simplicity
Another real-world
demand is the simplicity of the mechanism. If researchers combined 
some of the works mentioned above to create a cumbersome mechanism
with satisfactory theoretical qualities, that still would not guarantee
 its acceptance by the market: the providers and the
clients.
%\note{Orna - trying to get a  reference from Tenenholtz for here, about adwords}
  
%\section{Related Work} 
%Markets, future markets and auction based allocation of
%complete IaaS instances (denoted ``resources'' in this literature)
%were extensively researched.
%~\cite{agmonben-yehuda11}~\cite{nathuji10}
%~\cite{nathuji10,heo09,padala09}~\cite{harding68}.r~\cite{krieger10} ~\cite{altmann01}.in the developing IaaS
%market~\cite{altmann08,vanmechelen11}.
%
%
%
%\section{Conclusion}
In conclusion, making the RaaS cloud a reality will require solving
hard problems spanning the entire gamut from game theory and economic
models to system software and architecture. The onus is now on us, the
cloud computing research community, to lead the way and build the
mechanisms and policies that will make the RaaS cloud a reality.

{\footnotesize
\bibliographystyle{acm}
\bibliography{raas}}

\theendnotes

\end{document}


% LocalWords:  RaaS LeRoy LeRoy's IaaS roadmap CACM CloudSigma Gridspot nom dan
% LocalWords:  ProfitBricks slashdotted Orna Agmon Yehuda Muli Assaf Schuster
% LocalWords:  Tsafrir Technion ladypine muli assaf Rackspace GoGrid CPUs SLAs
% LocalWords:  Softlayer SLA Virtualization overcommited SaaS PaaS Dotcloud QoS
% LocalWords:  GridSpot commoditization OpenStack RedHat VMware prepurchasing
% LocalWords:  GPGPUs FPGAs IOV Chun Culler Semret Waldspurger et al Lubin Kash
% LocalWords:  SpotCloud Hu virtualization Gbps balancer balancers intra VCG
% LocalWords:  multi Ronen McSherry Talwar Maill Tuffin raas
