The generalized version of policy improvement and policy evaluation allows one to leverage the solution of some tasks to speed up the solution of others. If the reward function of a task can be well approximated as a linear combination of the reward functions of tasks previously solved, we can reduce a reinforcement-learning problem to a simpler linear regression. When this is not the case, the agent can still exploit the task solutions by using them to interact with and learn about the environment. Both strategies considerably reduce the amount of data needed to solve a reinforcement-learning problem.
The combination of reinforcement learning with deep learning
is a promising approach to tackle important sequential decisionmaking
problems that are currently intractable. One obstacle to
overcome is the amount of data needed by learning systems of
this type. This article proposes to address this issue through
a divide-and-conquer approach. The authors argue that complex decision
problems can be naturally decomposed into multiple tasks that
unfold in sequence or in parallel. By associating each task with
a reward function, this problem decomposition can be seamlessly
accommodated through a generalization
of two fundamental operations in reinforcement learning:
policy improvement and policy evaluation.
In this paper we survey a number of interesting applications of blockchain technology not related to cryptocurrencies. As a matter of fact, after an initial period of application to cryptocurrencies and to the financial world, blockchain technology has been successfully exploited in many other different scenarios, where its unique features allowed the definition of innovative and sometimes disruptive solutions. In particular, this paper takes into account the following application scenarios: end-to-end verifiable electronic voting, healthcare records management, identity management systems, access control systems, decentralized notary (with a focus on intellectual property protection) and supply chain management. For each of these, we firstly analyse the problem, the related requirements and the advantages the adoption of blockchain technology might bring. Then, we present a number of relevant solutions proposed in the literature both by academia and companies.
Emerging class of context-aware mobile applications, such as Google Now and Foursquare require continuous location sensing to deliver different location-aware services. Existing research, in finding location at higher abstraction, use GPS and WiFi location interfaces to discover places, which result in high power consumption. These interfaces are also not available on all feature phones that are in majority in developing countries. In this paper, we present a framework PlaceMap that discovers different places and routes, solely using GSM information, i.e., Cell ID. PlaceMap stores and manages all the discovered places and routes, which are used to build spatio-temporal mobility profiles for the users. PlaceMap provides algorithms that can complement GSM-based place discovery with an initial WiFi-based training to increase accuracy. We performed a comprehensive offline evaluation of PlaceMap algorithms on two large real-world diverse datasets, self-collected dataset of 62 participants for 4 weeks in India and MDC dataset of 38 participants for 45 weeks in Switzerland. We found that PlaceMap is able to discover up to 81% of the places correctly as compared to GPS. To corroborate the potential of PlaceMap in real-world, we deployed a life-logging application for a small set of 18 participants and observed similar place discovery accuracy.
Kuldeep Yadav, Abhishek Kumar, Prateek Jassal, Vinayak S. Naik
Published: Jan 2014
Most of the contextual dependent applications require high level location attributes in terms of places and routes, more than just fine-grained latitude and longitude. Currently, these applications perform place discovery and recognition in an isolated manner with very little coordination and collaboration with each other. This approach is inhibiting for the application developer as well as the endmobile user, as former has to write redundant and undesirable code for place discovery and the latter's device is subjected to redundant sensing, processing, storage, and higher energy consumption.
In this paper we present a novel framework PMWare, which is a middleware that caters to place and route sensing needs of 3rd party applications in a unified and integrated manner. It provides an end-to-end service from sensing user's location to discovering high-level location attributes while providing interfaces for managing and storing large-scale human mobility patterns. PMWare handles the energy-accuracy tradeoff and uses triggered-sensing approach to reduce battery consumption. To demonstrate the end-to-end working of our middleware, we developed an application PlaceADs that uses place visiting history of a person to push relevant advertisements. We deployed this application among 16 participants in real-world to check the effectiveness of PMWare.
Large proliferation of mobile phone applications result in extensive use of data intensive services such as multimedia download and social network communication. With limited penetration of 3G/4G networks in developing countries, it is common to use low bandwidth 2G services for data communication, resulting in larger download time and correspondingly high power consumption. In this paper, we present a system architecture, Unity, that enables collaborative downloading across co-located peers. Unity uses short range radio interfaces such as Bluetooth/WiFi for local coordination, while the actual content is downloaded using a cellular connection. Unity is designed to support mobile phones with diverse capabilities. End-to-end implementation and evaluation of Unity on Android based phones, with varying workload sizes and number of peers, show that Unity can result in multifold increase in download rate for the co-located peers. We also describe architecture of cloud-based Unity which uses principles of mobility prediction, social interactions, and opportunistic networking to make collaboration more pervasive and useful.
Mishari Al Mishari, Emiliano De Cristofaro, Karim El Defrawy, Gene Tsudik
Published: Sep 2009
Web-fraud is one of the most unpleasant features of today's Internet. Twowell-known examples of fraudulent activities on the web are phishing andtyposquatting. Their effects range from relatively benign (such as unwantedads) to downright sinister (especially, when typosquatting is combined withphishing). This paper presents a novel technique to detect web-fraud domainsthat utilize HTTPS. To this end, we conduct the first comprehensive study ofSSL certificates. We analyze certificates of legitimate and popular domains andthose used by fraudulent ones. Drawing from extensive measurements, we build aclassifier that detects such malicious domains with high accuracy.
Large amounts of data emerging from experiments in molecular medicine are leading to the identification of molecular signatures associated with disease subtypes. The contextualization of these patterns is important for obtaining mechanistic insight into the aberrant processes associated with a disease, and this typically involves the integration of multiple heterogeneous types of data. In this review, we discuss knowledge representations that can be useful to explore the biological context of molecular signatures, in particular three main approaches, namely, pathway mapping approaches, molecular network centric approaches and approaches that represent biological statements as knowledge graphs. We discuss the utility of each of these paradigms, illustrate how they can be leveraged with selected practical examples and identify ongoing challenges for this field of research.
A scholarly communication system needs to register, distribute, certify, archive, and incentivize knowledge production. The current article-based system technically fulfills these functions, but suboptimally. I propose a module-based communication infrastructure that attempts to take a wider view of these functions and optimize the fulfillment of the five functions of scholarly communication. Scholarly modules are conceptualized as the constituent parts of a research process as determined by a researcher. These can be text, but also code, data, and any other relevant pieces of information that are produced in the research process. The chronology of these modules is registered by iteratively linking to each other, creating a provenance record of parent and child modules (and a network of modules). These scholarly modules are linked to scholarly profiles, creating a network of profiles, and a network of how profiles relate to their constituent modules. All these scholarly modules would be communicated on the new peer-to-peer Web protocol Dat, which provides a decentralized register that is immutable, facilitates greater content integrity than the current system through verification, and is open-by-design. Open-by-design would also allow diversity in the way content is consumed, discovered, and evaluated to arise. This initial proposal needs to be refined and developed further based on the technical developments of the Dat protocol, its implementations, and discussions within the scholarly community to evaluate the qualities claimed here. Nonetheless, a minimal prototype is available today, and this is technically feasible.