Sadjad Fouladi, Francisco Romero, Dan Iter, and Qian Li, Stanford University; Shuvo Chatterjee, unaffiliated; Christos Kozyrakis, Matei Zaharia, and Keith Winstein, Stanford University
gg, a framework and a set of command-line tools that
helps people execute everyday applications—e.g., software
compilation, unit tests, video encoding, or object recognition—using
thousands of parallel threads on a cloud-functions service to achieve
near-interactive completion time. In the future, instead of
running these tasks on a laptop, or keeping a warm cluster running in
the cloud, users might push a button that spawns 10,000 parallel cloud
functions to execute a large job in a few seconds from start.
designed to make this practical and easy.
gg, applications express a job as a composition of lightweight
OS containers that are individually transient (lifetimes of 1–60
seconds) and functional (each container is hermetically sealed and
gg takes care of instantiating these containers on
cloud functions, loading dependencies, minimizing data movement,
moving data between containers, and dealing with failure and
We ported several latency-sensitive applications to run on
evaluated its performance. In the best case, a distributed compiler
gg outperformed a conventional tool (
2–5×, without requiring a warm cluster running continuously.
In the worst case,
gg was within 20% of the hand-tuned performance
of an existing tool for video encoding (ExCamera).
Open Access Media
USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.