« Back Record Document

Camel: collective-aware message logging




The continuous progress in the performance of supercomputers has made possible the understanding of many fundamental problems in science. Simulation, the third scientific pillar, constantly demands more powerful machines to use algorithms that would otherwise be unviable. That will inevitably lead to the deployment of an exascale machine during the next decade. However, fault tolerance is a major challenge that has to be overcome to make such a machine usable. With an unprecedented number of parts, machines at extreme scale will have a small mean-time-between-failures. The popular checkpoint/restart mechanism used in today’s machines may not be effective at that scale. One promising way to revamp checkpoint/restart is to use message-logging techniques. By storing messages during execution and replaying them in case of a failure, message logging is able to shorten recovery time and save a substantial amount of energy. The downside of message logging is that memory footprint may grow to unsustainable levels. This paper presents a technique that decreases the memory pressure in message-logging protocols by only storing the necessary messages in collective-communication operations. We introduce Camel, a protocol that has a low memory overhead for multicast and reduction operations. Our results show that Camel can reduce memory footprint in a molecular dynamics benchmark for more than 95 % on 16,384 cores. © 2015, Springer Science+Business Media New York.

Instituto Tecnológico de Costa Rica

Lidia Gómez

Cartago - 300m Este del Estadio Fello Meza. Apartado 159-7050.

2550-2263, 2550-2365

Address: Av. Mariscal Antonio José de Sucre N58-63 y Fernández Salvador Edif. Olade - San Carlos, Quito - Ecuador.

Web: www.olade.org

Phone: (593 2) 259 8122 / 2598 280

Correo: realc@olade.org

Desarrollado por: Aikyu-Systems