Deep reinforcement learning (DRL) and evolution strategies (ESs) have surpassed human-level control in many sequential decision-making problems, yet many open challenges still exist. To get insights into the strengths and weaknesses of DRL versus ESs, an analysis of their respective capabilities and limitations is provided. After presenting their fundamental concepts and algorithms, a comparison is provided on key aspects, such as scalability, exploration, adaptation to dynamic environments, and multiagent learning. Current research challenges are also discussed, including sample efficiency, exploration versus exploitation, dealing with sparse rewards, and learning to plan. Then, the benefits of hybrid algorithms that combine DRL and ESs are highlighted.@en

Tiny energy harvesting battery-free devices promise maintenance free operation for decades, providing swarm scale intelligence in applications from healthcare to building monitoring. These devices operate intermittently because of unpredictable, dynamic energy harvesting environments, failing when energy is scarce. Despite this dynamic operation, current programming models are static; they ignore the event-driven and time-sensitive nature of sensing applications, focusing only on preserving forward progress while maintaining performance. This paper proposes InK; the first reactive kernel that provides a novel way to program these tiny energy harvesting devices that focuses on their main application of event-driven sensing. InK brings an event-driven paradigm shift for batteryless applications, introducing building blocks and abstractions that enable reacting to changes in available energy and variations in sensing data, alongside task scheduling, while maintaining a consistent memory and sense of time. We implemented several event-driven applications for InK, conducted a user study, and benchmarked InK against the state-of-the-art; InK provides up to 14 times more responsiveness and was easier to use. We show that InK enables never before seen batteryless applications, and facilitates more sophisticated batteryless programs.@en

This poster presents a hardware/software solution, called On-The-Go Switch (OTGS), enabling a smartphone to control the connection state of a USB-attached device. Through an example, we show how OTGS can reduce the energy consumption of portable spectrum sensing platform utilizing lowcost Software Defined Radio. We show that OTGS can reduce the energy consumption by 47%, compared to a baseline, which would be difficult with a software alone, e.g. kernel modification.@en

Energy-neutral Internet of Things requires freeing embedded devices from batteries and powering them from ambient energy. Ambient energy is, however, unpredictable and can only power a device intermittently. Therefore, the paradigm of intermittent execution is to save the program state into non-volatile memory frequently to preserve the execution progress. In task-based intermittent programming, the state is saved at task transition. Tasks are fixed at compile time and agnostic to energy conditions. Thus, the state may be saved either more often than necessary or not often enough for the program to progress and terminate. To address these challenges, we propose Coala, an adaptive and efficient task-based execution model. Coala progresses on a multi-task scale when energy permits and preserves the computation progress on a sub-task scale if necessary. Coala's specialized memory virtualization mechanism ensures that power failures do not leave the program state in non-volatile memory inconsistent. Our evaluation on a real energy-harvesting platform not only shows that Coala reduces runtime by up to 54% as compared to a state-of-the-art system, but also it is able to progress where static systems fail.@en

Battery-free energy-harvesting devices have the potential to operate for decades, since they draw power fromvirtually unlimited energy sources, such as sunlight. However, ambient energy sources are volatile, and tiny harvesters can extract only weak power from them. Thus, small energy-harvesting devices operate intermittently: first, they charge their buffers then start operating, which depletes the buffered energy and causes the devices to power down, letting the harvesters to refill the energy buffers for the next operational round. Classical programming architectures assume continuous power. Therefore, frequent power failures render them useless; power failures reset the computational progress and delete volatile data. Thus, the intermittent programming and execution paradigm has emerged. Generally, there are two strategies being employed to support intermittent execution: checkpoint-based and task-based. Prior checkpoint- and task-based systems tackled mainly challenges related to enabling efficient computing on intermittent power. However, they have ignored the challenges associated with sensing, which is the primary application for intermittent systems. Therefore, from a sensing standpoint, these systems have several drawbacks. @en

The main obstacles to achieve truly ubiquitous sensing are (i) the limitations of battery technology - batteries are short-lived, hazardous, bulky, and costly - and (ii) the unpredictability of ambient power. The latter causes sensors to operate intermittently, violating the availability requirements of many real-world applications. In this paper, we present the Coalesced Intermittent Sensor (CIS), an intermittently-powered sensor that senses continuously! Although a single node will frequently be off charging, a group of nodes can -in principle- sense 24/7 provided that their awake times are spread apart. As communication is too expensive, we rely on inherent component variations that induce small differences in power cycles. This basic assumption has been verified through measurements of different nodes and power sources. However, desynchronizing nodes is not enough. An important finding is that a CIS designed for certain (minimal) energy conditions will become synchronized when the available energy exceeds the design point. Nodes employing a sleep mode (to extend their availability) do wake up collectively at some event, process it, and return to charging as the remaining energy is typically too low to handle another event. This results in multiple responses (bad) and missing subsequent events (worse) due to the synchronized charging. To counter this undesired behavior we designed an algorithm to estimate the number of active neighbors and respond proportionally to an event. We show that when intermittent nodes randomize their responses to events, in favorable energy conditions, the CIS reduces the duplicated captured events by 50% and increases the percentage of capturing entire bursts above 85%.@en

Energy harvesting and battery-free sensing devices show great promise for revolutionizing computing in the home, in the wild, and on the body. The promise of cheap, dense, and ubiquitous sensing technology brings new applications for the Internet of Things. However, the future programming model is blurry and complex. With a potential for trillions of devices, and thousands of devices per person on earth, programming languages and associated operating systems must be usable, flexible, and resource efficient. Because of the thousands of applications and fine grained differences in requirements, multi-tenancy may be a part of the solution to solving this programming model crisis. This paper explores the energy and resources costs, feasibility, and motivation for multi-tenancy on these tiniest of computing devices-namely the difficulties in scheduling tasks fairly, efficiently, and simply. Because of intermittent power, resources and energy must be mostly devoted towards user tasks, we implement a rudimentary operating system with low overhead to conduct experiments and test time-sharing and scheduling protocols. We close with a discussion on challenges to implementing a multi-tenant run-time on battery-free tags, and proposals for future work.@en

—Battery-free computational RFID platforms, such as WISP (Wireless Identification and Sensing Platform), are intermittently-powered devices designed for replacing existing sensor networks. Accordingly, synchronization appears as one of the crucial building blocks for collaborative and coordinated actions in these platforms. However, intermittent power leads to frequent loss of computational state and short-term clock frequency instability that makes synchronization challenging. In this article, we introduce the WISP-Sync protocol that provides synchronization among WISP tags in the communication range of an RFID reader. WISP-Sync overcomes the aforementioned challenges by employing a Proportional-Integral (PI) controller-inspired algorithm which (i) is adaptive—reactive to short-term clock instabilities; (ii) requires only a few computation steps—suitable for limited harvested energy; and (iii) keeps a few variables to hold the synchronization state—minimum overhead to recover from power interrupts. Evaluations in our testbed showed that WISP-Sync ensured an average synchronization error of approximately 1 ms among the tags with an average energy overhead of 1.85 mJ per synchronization round.@en

We present Stork - an extension of the EPC C1G2 protocol allowing streaming of data to multiple Computational Radio Frequency IDentification tags (CRFIDs) simultaneously at up to 20 times faster than the prior state of the art. Stork introduces downstream attributes never before seen in (C)RFIDs: (i) fast feedback for CRFID downstream verification based on the internal EPC C1G2 memory check command - which we analytically and experimentally show to be the best possible downstream verification process based on EPC C1G2; (ii) ability to perform multi-CRFID transfer - which in our experiments speeds up downstream by more than two times compared to sequential transmission; and (iii) the use of compressed data streams - which improves firmware reprogramming times by up to 10% at large reader-to-CRFID distances.@en

The ability to localize a sound source is essential for safe integration of robots in our society. However, its high computational demand makes it challenging to equip small swarm robots with such capability. Therefore, this paper investigates potential means for developing a lightweight sound source localization method. It introduces the Cutting The Plane (CTP) algorithm which leverages recent advancements in embedded microphone technology to reduce the computation complexity of audio source localization. We analyzed its performance and compared it to a software-based approach using simulation. Finally, we performed tests on real hardware to show the feasibility of using our system for relative localization between swarm robots.@en

Backscatter has emerged as the dominant paradigm for battery-free networking among the (potentially) trillions of devices in the future Internet of Things, partly because of the order of magnitude smaller energy consumption, but at the cost of collisions, low data rates, and short distances. This position paper explores the alternative approach: using lowpower, yet active radios to communicate among the battery-less swarm. We describe the challenges of using active radios in this context, including lack of tight time guarantees, high listening costs, and intermittent operation. While backscatter is promising, this paper hopes to broaden the conversation around alternative methods for networking the future IoT.@en