gwern's posts
Post has attachment
Public
"Sim-to-Real Robot Learning from Pixels with Progressive Nets", Rusu et al 2016:
"Applying end-to-end learning to solve complex, interactive, pixel-driven control tasks on a robot is an unsolved problem. Deep Reinforcement Learning algorithms are too slow to achieve performance on a real robot, but their potential has been demonstrated in simulated environments. We propose using progressive networks to bridge the reality gap and transfer learned policies from simulation to the real world. The progressive net approach is a general framework that enables reuse of everything from low-level visual features to high-level policies for transfer to new tasks, enabling a compositional, yet simple, approach to building complex skills. We present an early demonstration of this approach with a number of experiments in the domain of robot manipulation that focus on bridging the reality gap. Unlike other proposed approaches, our real-world experiments demonstrate successful task learning from raw visual input on a fully actuated robot manipulator. Moreover, rather than relying on model-based trajectory optimisation, the task learning is accomplished using only deep reinforcement learning and sparse rewards."
A nice use of progressive networks; see also the Schmidhuber paper.
"Applying end-to-end learning to solve complex, interactive, pixel-driven control tasks on a robot is an unsolved problem. Deep Reinforcement Learning algorithms are too slow to achieve performance on a real robot, but their potential has been demonstrated in simulated environments. We propose using progressive networks to bridge the reality gap and transfer learned policies from simulation to the real world. The progressive net approach is a general framework that enables reuse of everything from low-level visual features to high-level policies for transfer to new tasks, enabling a compositional, yet simple, approach to building complex skills. We present an early demonstration of this approach with a number of experiments in the domain of robot manipulation that focus on bridging the reality gap. Unlike other proposed approaches, our real-world experiments demonstrate successful task learning from raw visual input on a fully actuated robot manipulator. Moreover, rather than relying on model-based trajectory optimisation, the task learning is accomplished using only deep reinforcement learning and sparse rewards."
A nice use of progressive networks; see also the Schmidhuber paper.
Post has attachment
Public
"Safe landing strategies during a fall: Systemic review and meta-analysis", Moon & Sosnoff 2016:
"*Objective*: To systematically synthesize information on safe landing strategies for a fall and quantitatively examine the effects of the strategies to reduce risk of injury from a fall.
Data Sources: PubMed, Web of science, Cumulative Index to Nursing and Allied Health Literature, and Cochrane Library
Study Selection: Databases were searched using the combinations of keywords of “falls”, “strategy”, “impact” and “load”. Randomized control trials, cohort studies, pre-post studies, or cross-sectional studies were included.
Data Extraction: The fall strategies were extracted and categorized by falling direction. Measurements of impact loads that reflect the risk of injuries were extracted (e.g. impact velocity, impact force, fall duration, and impact angle). Hedges g was used as effect size to quantify effect of a protective landing strategy to reduce the impact load.
Data Synthesis: A total of seven landing strategies (squatting, elbow flexion, forward rotation, martial arts rolling, martial arts slapping, relaxed muscle, and stepping) in 13 studies were examined. In general, all strategies, except for the martial arts slapping technique, significantly reduced impact load (g’s=0.73 to 2.70). Squatting was an efficient strategy to reduce impact in backward falling (g=1.77) while elbow flexion with outstretched arms was effective in forward falling (g=0.82). Also, in sideways falling strategies, martial arts rolling (g=2.70) and forward rotation (g=0.82) were the most efficient strategies to reduce impact load.
Conclusions: The result showed that landing strategies have significant effect on reducing impact load during a fall and might be effective to reduce impact load of falling. The current study also highlighted limitations of the previous studies which focused on a young population and self-initiated falls. Further investigation with elderly individuals and unexpected falls is necessary to verify effectiveness and suitableness of the strategies to at-risk population in real-life falls.
A fall is an unexpected event in which an individual comes to rest on the ground floor or lower level 1 . They are one of the leading causes of injury and death among the elderly 2 . An estimated 40% of community-dwelling people aged over 65 years fall at least once a year, and nearly 15% fall twice or more per year 3 . Falls result in 62.5% (2.5 million) of non-fatal injuries of older adults in the United States that require treatment in emergency departments and hospitalization 4 . The direct medical cost for fall related injuries reaches $19 billion annually in the U.S. alone 5 . In addition, as the population ages, the number of annual fall related injuries in the United States is expected to increase to 5.7 million by the year 2030 6 ."
Useful advice for anyone planning on living long enough to become elderly.
"*Objective*: To systematically synthesize information on safe landing strategies for a fall and quantitatively examine the effects of the strategies to reduce risk of injury from a fall.
Data Sources: PubMed, Web of science, Cumulative Index to Nursing and Allied Health Literature, and Cochrane Library
Study Selection: Databases were searched using the combinations of keywords of “falls”, “strategy”, “impact” and “load”. Randomized control trials, cohort studies, pre-post studies, or cross-sectional studies were included.
Data Extraction: The fall strategies were extracted and categorized by falling direction. Measurements of impact loads that reflect the risk of injuries were extracted (e.g. impact velocity, impact force, fall duration, and impact angle). Hedges g was used as effect size to quantify effect of a protective landing strategy to reduce the impact load.
Data Synthesis: A total of seven landing strategies (squatting, elbow flexion, forward rotation, martial arts rolling, martial arts slapping, relaxed muscle, and stepping) in 13 studies were examined. In general, all strategies, except for the martial arts slapping technique, significantly reduced impact load (g’s=0.73 to 2.70). Squatting was an efficient strategy to reduce impact in backward falling (g=1.77) while elbow flexion with outstretched arms was effective in forward falling (g=0.82). Also, in sideways falling strategies, martial arts rolling (g=2.70) and forward rotation (g=0.82) were the most efficient strategies to reduce impact load.
Conclusions: The result showed that landing strategies have significant effect on reducing impact load during a fall and might be effective to reduce impact load of falling. The current study also highlighted limitations of the previous studies which focused on a young population and self-initiated falls. Further investigation with elderly individuals and unexpected falls is necessary to verify effectiveness and suitableness of the strategies to at-risk population in real-life falls.
A fall is an unexpected event in which an individual comes to rest on the ground floor or lower level 1 . They are one of the leading causes of injury and death among the elderly 2 . An estimated 40% of community-dwelling people aged over 65 years fall at least once a year, and nearly 15% fall twice or more per year 3 . Falls result in 62.5% (2.5 million) of non-fatal injuries of older adults in the United States that require treatment in emergency departments and hospitalization 4 . The direct medical cost for fall related injuries reaches $19 billion annually in the U.S. alone 5 . In addition, as the population ages, the number of annual fall related injuries in the United States is expected to increase to 5.7 million by the year 2030 6 ."
Useful advice for anyone planning on living long enough to become elderly.
Post has attachment
Public
Use and abuse of 'arbitration' clauses in international trade treaties.
Post has attachment
Public
Fun MRI applications, selecting guide dogs: "Functional MRI in awake dogs predicts suitability for assistance work", Berns et al 2016:
"The overall goal of this work was to measure the efficacy of fMRI for predicting whether a dog would be a successful service dog. The training and imaging were performed in 50 dogs entering advanced training at 17-21 months of age. FMRI responses were measured while each dog observed hand signals indicating either reward or no reward and given by both a familiar handler and a stranger. 49 dogs successfully completed fMRI training and scanning. Of these, 33 eventually completed service training and were matched with a person, while 10 were released for behavioral reasons. Using anatomically defined regions-of-interest in the ventral caudate, amygdala, and visual cortex, we developed a classifier based on the dogs' outcomes. We found that responses in the stranger condition were sufficient to develop an accurate brain-based classifier. On all data, the classifier had a positive predictive value of 96% with 10% false positives. The area under the receiver operating characteristic curve was 0.90 (0.79 with 4-fold cross-validation, P=0.02), indicating a significant diagnostic capability. Within the stranger condition, the differential response to [reward - no reward] in ventral caudate was positively correlated with a successful outcome, while the differential response in the amygdala was negatively correlated to outcome. These results show that successful service dogs transfer knowledge to strangers as indexed by ventral caudate activity without excessive arousal as measured in the amygdala...Most dogs are not destined to be service dogs. Even with well-managed breeding programs, the success rate in training is typically 3040%. By many estimates, the cost of training a service dog is $20,000 to $50,000. If dogs that are destined to fail training could be identified earlier, the average cost would decline. Thus, there is both a need to increase the number of service dogs and decrease the average cost by early identification"
"The overall goal of this work was to measure the efficacy of fMRI for predicting whether a dog would be a successful service dog. The training and imaging were performed in 50 dogs entering advanced training at 17-21 months of age. FMRI responses were measured while each dog observed hand signals indicating either reward or no reward and given by both a familiar handler and a stranger. 49 dogs successfully completed fMRI training and scanning. Of these, 33 eventually completed service training and were matched with a person, while 10 were released for behavioral reasons. Using anatomically defined regions-of-interest in the ventral caudate, amygdala, and visual cortex, we developed a classifier based on the dogs' outcomes. We found that responses in the stranger condition were sufficient to develop an accurate brain-based classifier. On all data, the classifier had a positive predictive value of 96% with 10% false positives. The area under the receiver operating characteristic curve was 0.90 (0.79 with 4-fold cross-validation, P=0.02), indicating a significant diagnostic capability. Within the stranger condition, the differential response to [reward - no reward] in ventral caudate was positively correlated with a successful outcome, while the differential response in the amygdala was negatively correlated to outcome. These results show that successful service dogs transfer knowledge to strangers as indexed by ventral caudate activity without excessive arousal as measured in the amygdala...Most dogs are not destined to be service dogs. Even with well-managed breeding programs, the success rate in training is typically 3040%. By many estimates, the cost of training a service dog is $20,000 to $50,000. If dogs that are destined to fail training could be identified earlier, the average cost would decline. Thus, there is both a need to increase the number of service dogs and decrease the average cost by early identification"
Post has attachment
Post has attachment
Post has attachment
Public
"Do scholars follow Betteridge’s Law? The use of questions in journal article titles", Cook & Plourde 2016 https://www.dropbox.com/s/kootab1g7yr0ecm/2016-cook.pdf
"In journalistic publication, Betteridge’s Law of Headlines stipulates that ‘‘Any headline that ends in a question mark can be answered by the word no.’’ When applied to the titles of academic publication, the assertion is referred to as Hinchcliffe’s Rule and denigrates the use of the question mark in titles as a ‘‘click-bait’’ marketing strategy. We examine the titles of all published articles in the year 2014 from five top-ranked and five mid-range journals in each of six academic fields (n = 7845). We describe the form of questions when they occur, and where a title poses a question that can be answered with a ‘‘yes’’ or ‘‘no’’ we note the article’s substantive answer. We do not find support for the criticism lodged by Betteridge’s Law and Hinchcliffe’s Rule. Although patterns vary by discipline, titles with questions are posed infrequently overall. Further, most titles with questions do not pose yes/no questions. Finally, the few questions that are posed in yes/no terms are actually more often answered with a ‘‘yes’’ than with a ‘‘no.’’ Concerns regarding click-bait questions in academic publications may, therefore, be unwarranted."
"In journalistic publication, Betteridge’s Law of Headlines stipulates that ‘‘Any headline that ends in a question mark can be answered by the word no.’’ When applied to the titles of academic publication, the assertion is referred to as Hinchcliffe’s Rule and denigrates the use of the question mark in titles as a ‘‘click-bait’’ marketing strategy. We examine the titles of all published articles in the year 2014 from five top-ranked and five mid-range journals in each of six academic fields (n = 7845). We describe the form of questions when they occur, and where a title poses a question that can be answered with a ‘‘yes’’ or ‘‘no’’ we note the article’s substantive answer. We do not find support for the criticism lodged by Betteridge’s Law and Hinchcliffe’s Rule. Although patterns vary by discipline, titles with questions are posed infrequently overall. Further, most titles with questions do not pose yes/no questions. Finally, the few questions that are posed in yes/no terms are actually more often answered with a ‘‘yes’’ than with a ‘‘no.’’ Concerns regarding click-bait questions in academic publications may, therefore, be unwarranted."
Post has attachment
Post has attachment
Public
Deep Mind continues tweaking the Neural Turing Machine for better performance: "Hybrid computing using a neural network with dynamic external memory", Graves et al 2016 https://www.dropbox.com/s/0a40xi702grx3dq/2016-graves.pdf https://deepmind.com/blog/differentiable-neural-computers/
"Artificial neural networks are remarkably adept at sensory processing, sequence learning and reinforcement learning, but are limited in their ability to represent variables and data structures and to store data over long timescales, owing to the lack of an external memory. Here we introduce a machine learning model called a differentiable neural computer (DNC), which consists of a neural network that can read from and write to an external memory matrix, analogous to the random-access memory in a conventional computer. Like a conventional computer, it can use its memory to represent and manipulate complex data structures, but, like a neural network, it can learn to do so from data. When trained with supervised learning, we demonstrate that a DNC can successfully answer synthetic questions designed to emulate reasoning and inference problems in natural language. We show that it can learn tasks such as finding the shortest path between specified points and inferring the missing links in randomly generated graphs, and then generalize these tasks to specific graphs such as transport networks and family trees. When trained with reinforcement learning, a DNC can complete a moving blocks puzzle in which changing goals are specified by sequences of symbols. Taken together, our results demonstrate that DNCs have the capacity to solve complex, structured tasks that are inaccessible to neural networks without external read–write memory."
"Artificial neural networks are remarkably adept at sensory processing, sequence learning and reinforcement learning, but are limited in their ability to represent variables and data structures and to store data over long timescales, owing to the lack of an external memory. Here we introduce a machine learning model called a differentiable neural computer (DNC), which consists of a neural network that can read from and write to an external memory matrix, analogous to the random-access memory in a conventional computer. Like a conventional computer, it can use its memory to represent and manipulate complex data structures, but, like a neural network, it can learn to do so from data. When trained with supervised learning, we demonstrate that a DNC can successfully answer synthetic questions designed to emulate reasoning and inference problems in natural language. We show that it can learn tasks such as finding the shortest path between specified points and inferring the missing links in randomly generated graphs, and then generalize these tasks to specific graphs such as transport networks and family trees. When trained with reinforcement learning, a DNC can complete a moving blocks puzzle in which changing goals are specified by sequences of symbols. Taken together, our results demonstrate that DNCs have the capacity to solve complex, structured tasks that are inaccessible to neural networks without external read–write memory."
Post has attachment
Public
Side-channel attacks: eavesdropping on typing by bouncing WiFi signals off moving fingers.
"Keystroke Recognition Using WiFi Signals", Ali et al 2015:
"Keystroke privacy is critical for ensuring the security of computer systems and the privacy of human users as what being typed could be passwords or privacy sensitive information. In this paper, we show for the first time that WiFi signals can also be exploited to recognize keystrokes. The intuition is that while typing a certain key, the hands and fingers of a user move in a unique formation and direction and thus generate a unique pattern in the time-series of Channel State Information (CSI) values, which we call CSI-waveform for that key. In this paper, we propose a WiFi signal based keystroke recognition system called WiKey. WiKey consists of two Commercial Off-The-Shelf (COTS) WiFi devices, a sender (such as a router) and a receiver (such as a laptop). The sender continuously emits signals and the receiver continuously receives signals. When a human subject types on a keyboard, WiKey recognizes the typed keys based on how the CSI values at the WiFi signal receiver end. We implemented the WiKey system using a TP-Link TL-WR1043ND WiFi router and a Lenovo X200 laptop. WiKey achieves more than 97.5% detection rate for detecting the keystroke and 96.4% recognition accuracy for classifying single keys. In real-world experiments, WiKey can recognize keystrokes in a continuously typed sentence with an accuracy of 93.5%."
"Keystroke Recognition Using WiFi Signals", Ali et al 2015:
"Keystroke privacy is critical for ensuring the security of computer systems and the privacy of human users as what being typed could be passwords or privacy sensitive information. In this paper, we show for the first time that WiFi signals can also be exploited to recognize keystrokes. The intuition is that while typing a certain key, the hands and fingers of a user move in a unique formation and direction and thus generate a unique pattern in the time-series of Channel State Information (CSI) values, which we call CSI-waveform for that key. In this paper, we propose a WiFi signal based keystroke recognition system called WiKey. WiKey consists of two Commercial Off-The-Shelf (COTS) WiFi devices, a sender (such as a router) and a receiver (such as a laptop). The sender continuously emits signals and the receiver continuously receives signals. When a human subject types on a keyboard, WiKey recognizes the typed keys based on how the CSI values at the WiFi signal receiver end. We implemented the WiKey system using a TP-Link TL-WR1043ND WiFi router and a Lenovo X200 laptop. WiKey achieves more than 97.5% detection rate for detecting the keystroke and 96.4% recognition accuracy for classifying single keys. In real-world experiments, WiKey can recognize keystrokes in a continuously typed sentence with an accuracy of 93.5%."
Wait while more posts are being loaded
