DreamTeam/Reading: Difference between revisions

From Noisebridge
Jump to navigation Jump to search
 
(210 intermediate revisions by 31 users not shown)
Line 1: Line 1:
(note wiki contains some useful clues re previous neuro research at Noisebridge ... For example, the [[Analog_EEG_Amp]] page describes some project ideas and work done by others here in 2012) 
'''This is essentially the groups meeting notes – a trail of bread crumbs of topics of conversation and projects entertained by the group'''


==Current Discussion==
note: Learn more about previous neuro research at Noisebridge on the wiki... For example, the [[Analog_EEG_Amp]] page describes some project ideas and work done by others here in 2012
(4 September 2013) more on analysis methods:
 
==Websites and events that have piqued our interest==
 
http://cs375.stanford.edu/ -- Dan Yamins Large-Scale Neural Network Models for Neuroscience CS375
 
https://faculty.washington.edu/chudler/facts.html -- brain facts
 
http://onlinehub.stanford.edu/cs224 -- Natural Language Processing with Deep Learning
 
http://neurable.com/
 
https://noisebridge.net/wiki/NBDSM -- noiseBridge Deepnet and Statistical Mechanics -- first meetup @ noisebridge 7/6/17 at 7PM
 
https://metacademy.org/
-- machine learning knowledge graph
 
https://machinelearningguide.libsyn.com/rss -- machine learning guide podcast
 
http://www.thetalkingmachines.com/ -- podcast
 
https://karpathy.github.io/2015/05/21/rnn-effectiveness/
 
http://alexandre.barachant.org/papers/
 
http://ncs.ethz.ch/publications -- neuromorphic cognitive systems
 
https://github.com/crillab/gophersat/blob/master/examples/sat-for-noobs.md -- SAT solvers
 
https://media.ccc.de/v/34c3-8948-low_cost_non-invasive_biomedical_imaging -- Open EIT 34c3 talk https://github.com/OpenEIT
 
http://acrovirt.org/ -- sensors
 
http://www.neuroeducate.com/ -- citizen neuroscience
 
https://www.youtube.com/watch?v=9mZuyUzyN4Q -- "Categories for the Working Hacker"
 
http://radicalsciencenews.org/599-2/ -- "Deep Learning Fuels Nvidia’s Self-Driving Car Technology"
 
https://arxiv.org/abs/1803.03635 -- "The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks" Jonathan Frankle, Michael Carbin
 
== Global Workspace Theory ==
 
http://bernardbaars.pbworks.com/f/BaarsJCS1997.pdf -- "IN THE THEATRE OF CONSCIOUSNESS"
 
 
== Symbolic Mathematics ==
 
https://arxiv.org/pdf/1912.01412.pdf -- "Deep Learning for Symbolic Mathematics"
 
https://www.scottaaronson.com/busybeaver.pdf -- "A Relatively Small Turing Machine Whose Behavior Is Independent of Set Theory"
 
https://www.scottaaronson.com/blog/?p=2725 -- "The 8000th Busy Beaver number eludes ZF set theory: new paper by Adam Yedidia and me"
 
== Vision ==
 
https://webvision.med.utah.edu/
 
== Language Models ==
 
https://blog.scaleway.com/2019/building-a-machine-reading-comprehension-system-using-the-latest-advances-in-deep-learning-for-nlp/ -- "Natural Language Processing: the age of Transformers"
 
https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf -- "Language Models are Unsupervised Multitask Learners"
 
https://arxiv.org/pdf/1706.03762 -- "Attention Is All You Need"
 
https://arxiv.org/pdf/1705.03122.pdf -- "Convolutional Sequence to Sequence Learning"
 
https://papers.nips.cc/paper/5346-sequence-to-sequence-learning-with-neural-networks.pdf -- "Sequence to Sequence Learning with Neural Networks"
 
https://arxiv.org/pdf/1508.07909.pdf -- "Neural Machine Translation of Rare Words with Subword Units"
 
https://github.com/rsennrich/subword-nmt
 
https://github.com/rowanz/grover - grover GPU/TPU based GPT-2 transformer implementation
 
https://arxiv.org/pdf/1905.12616.pdf -- "Defending Against Neural Fake News"
 
https://lilianweng.github.io/lil-log/2018/06/24/attention-attention.html - blog post on attention
 
https://github.com/lilianweng/transformer-tensorflow - sample implementation of "Attention Is All You Need"
 
https://github.com/tensorflow/tensor2tensor/blob/master/tensor2tensor/models/transformer.py - "official" implementation of "Attention Is All You Need"
 
https://jalammar.github.io/illustrated-gpt2/ - The Illustrated GPT-2 (Visualizing Transformer Language Models)
 
== Bioengineering ==
 
https://nips.cc/Conferences/2018/Schedule?showEvent=12487 -- "What Bodies Think About: Bioelectric Computation Outside the Nervous System, Primitive Cognition, and Synthetic Morphology"
 
== Data Visualization ==
 
https://www.csc2.ncsu.edu/faculty/healey/download/tvcg.12b.pdf -- "Interest Driven Navigation in Visualization"
 
== Fractal Dementia ==
 
https://pdfs.semanticscholar.org/0018/7c742e60d35d5034a63251e31e1b8d96c70b.pdf -- "Comparison of Fractal Dimension Algorithms for the Computation of Eeg Biomarkers for Dementia"
 
== Brain Activity Dynamics ==
 
https://arxiv.org/pdf/1802.02523.pdf -- "Plasma Brain Dynamics (PBD): a Mechanism for EEG Waves Under Human Consciousness"
 
https://arxiv.org/pdf/1206.1108.pdf -- "Thermodynamic Model of Criticality in the Cortex Based On EEG/ECOG Data"
 
https://www.bm-science.com/images/bms/publ/art63.pdf -- "Topographic Mapping of Rapid Transitions in EEG Multiple Frequencies"
 
== Silent Speech ==
 
https://dam-prod.media.mit.edu/x/2018/03/23/p43-kapur_BRjFwE6.pdf -- "AlterEgo: A Personalized Wearable Silent Speech Interface"
 
== Image Reconstruction ==
 
https://www.biorxiv.org/content/biorxiv/early/2017/12/28/240317.full.pdf -- "Deep image reconstruction from human brain
activity"
 
== EEG Electrodes ==
 
https://sites.google.com/site/biofeedbackpages/velcro-sensors -- Saline electrodes
 
https://www.commsp.ee.ic.ac.uk/~mandic/Ear_EEG_IEEE_Pulse_2012.pdf -- "The In-the-Ear
Recording Concept"
 
https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8357918 -- "Dry-Contact Electrode Ear-EEG"
 
== Generative Adversarial Networks (GAN) ==
 
https://arxiv.org/pdf/1710.08864 -- "One pixel attack for fooling deep neural networks"
 
== Kolmolgorov Complexity ==
 
ftp://ftp.idsia.ch/pub/juergen/loconet.pdf -- "Discovering Neural Nets with Low Kolmolgorov Complexity and High Generalization Capability"
 
https://papers.nips.cc/paper/394-chaitin-kolmogorov-complexity-and-generalization-in-neural-networks.pdf -- "Chaitin-Kolmogorov Complexity and Generalization in Neural Networks"
 
== OpenCV ==
 
http://arnab.org/blog/so-i-suck-24-automating-card-games-using-opencv-and-python
 
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.185.585&rep=rep1&type=pdf -- "Pyramidal Implementation of the Lucas Kanade Feature Tracker Description of the algorithm"
 
== Category Theory ==
 
https://arxiv.org/pdf/1711.10455 -- "Backprop as Functor: A compositional perspective on supervised learning"
 
http://math.ucr.edu/home/baez/rosetta.pdf -- "Physics, Topology, Logic and Computation: A Rosetta Stone"
 
https://www.youtube.com/watch?v=BF6kHD1DAeU -- "Category theory foundations 1.0 — Steve Awodey"
 
== Proof Searcher ==
 
https://arxiv.org/pdf/cs/0207097 -- "Optimal Ordered Problem Solver"
 
http://people.idsia.ch/~juergen/ultimatecognition.pdf -- "Ultimate Cognition a la Gödel"
 
http://people.idsia.ch/~juergen/selfreflection.pdf -- "Towards an Actual Gödel Machine Implementation"
 
== Capsule Models ==
 
https://arxiv.org/pdf/1710.09829.pdf -- "Dynamic Routing Between Capsules"
 
https://openreview.net/pdf?id=HJWLfGWRb -- "Matrix Capsules with EM Routing"
 
== Multivariate Coherence Training ==
 
https://www.youtube.com/watch?v=qGYjvLki0WY
== Infrared Neuroimaging ==
 
http://www.ecse.rpi.edu/~yazici/bio_book.pdf -- "Near Infrared Imaging and Spectroscopy for Brain Activity Monitoring"
 
http://fangyenlab.seas.upenn.edu/pubs/isr.pdf -- "Intrinsic optical signals in neural tissues:
measurements, mechanisms, and applications"
 
== Geometry ==
 
http://arxiv.org/abs/1710.10784 -- "How deep learning works --The geometry of deep learning"
 
== Affective Computing ==
 
http://affect.media.mit.edu/pdfs/05.ahn-picard-acii.pdf -- "Affective-Cognitive Learning and Decision
Making: A Motivational Reward Framework For Affective Agents"
 
== Explainability ==
 
http://arxiv.org/abs/1708.01785 -- "Interpreting CNN knowledge via an Explanatory Graph"
 
== NLP ==
 
https://arxiv.org/pdf/1605.06640 -- "Programming with a Differentiable Forth Interpreter"
 
https://pdfs.semanticscholar.org/f683/dbe8a22d633ad3a2cff379b055b26684a838.pdf -- "Solving General Arithmetic Word Problems"
 
https://arxiv.org/pdf/1611.04558.pdf -- "Google’s Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation"
 
http://emnlp2014.org/papers/pdf/EMNLP2014162.pdf -- "GloVe: Global Vectors for Word Representation"
 
== RNNs ==
 
https://arxiv.org/pdf/1611.01576.pdf -- "Quasi Recurrent Neural Networks"
 
== Hyper-parameter Optimization ==
 
https://arxiv.org/abs/1603.06560 -- "Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization"
 
== Transfer Learning ==
 
http://arxiv.org/abs/1710.10776v1 -- "Transfer Learning to Learn with Multitask Neural Model Search"
 
== Reinforcement Learning ==
 
http://www2.hawaii.edu/~sstill/StillPrecup2011.pdf -- "An information-theoretic approach to curiosity-driven reinforcement learning"
 
https://arxiv.org/abs/1605.06676 -- "Learning to Communicate with Deep Multi-Agent Reinforcement Learning"
 
== Learning to Learn ==
 
https://arxiv.org/pdf/1703.01041.pdf -- "Large-Scale Evolution of Image Classifiers"
 
https://arxiv.org/pdf/1611.01578 -- "Neural Architecture Search with Reinforcement Learning"
 
== The Utility of "Noise" in ML ==
 
https://www.cs.toronto.edu/~hinton/absps/JMLRdropout.pdf -- "Dropout:  A Simple Way to Prevent Neural Networks from Overfitting"
 
http://yann.lecun.com/exdb/publis/pdf/lecun-90b.pdf -- "Optimal Brain Damage"
 
https://arxiv.org/pdf/1502.01852.pdf -- "Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification"
 
== One-shot learning ==
 
https://arxiv.org/abs/1605%2E06065 -- "One-shot Learning with Memory-Augmented Neural Networks"
 
== Program Synthesis ==
 
https://pdfs.semanticscholar.org/0163/35ce7e0a073623e1deac7138b28913dbf594.pdf -- "Human-level concept learning through probabilistic program induction"
 
https://arxiv.org/pdf/1511.06279.pdf -- "Neural Programmer: Inducing Latent Programs with Gradient Descent"
 
https://arxiv.org/abs/1608.04428 -- "TerpreT: A Probabilistic Programming Language for Program Induction" Gaunt et al 2016
 
== Machine Learning Interaction ==
 
https://teachablemachine.withgoogle.com/#
 
== Game Theory ==
 
https://arxiv.org/abs/1707.01068v1 -  Maintaining cooperation in complex social dilemmas using deep reinforcement learning
 
== Questions of Physics and Free Will ==
 
http://www.scottaaronson.com/papers/giqtm3.pdf - The Ghost in the Quantum Turing Machine
 
== CNN ==
 
https://adeshpande3.github.io/adeshpande3.github.io/A-Beginner's-Guide-To-Understanding-Convolutional-Neural-Networks/ - "A Beginner's Guide To Understanding Convolutional Neural Networks"
 
https://adeshpande3.github.io/A-Beginner's-Guide-To-Understanding-Convolutional-Neural-Networks-Part-2/ - "A Beginner's Guide To Understanding Convolutional Neural Networks Part 2"
 
http://scs.ryerson.ca/~aharley/vis/harley_vis_isvc15.pdf -- "An Interactive Node-Link Visualization
of Convolutional Neural Networks"
 
http://white.stanford.edu/teach/index.php/An_Introduction_to_Convolutional_Neural_Networks
 
http://www.cv-foundation.org/openaccess/content_cvpr_2015/papers/Dosovitskiy_Learning_to_Generate_2015_CVPR_paper.pdf -- "Learning to Generate Chairs With Convolutional Neural Networks"
 
http://www.pamitc.org/cvpr15/files/lecun-20150610-cvpr-keynote.pdf
-- "What's Wrong With Deep Learning?"
 
== Mind-Body Relations ==
 
http://www.pnas.org/content/111/20/7379.full.pdf -- "Voluntary activation of the sympathetic nervous system and attenuation of the innate immune response in humans"
 
== Math ==
 
https://arxiv.org/pdf/1311.1090.pdf -- "Polyhedrons and Perceptrons Are Functionally Equivalent"
 
Example code and training data using polyhedrons developed by author of above paper:  https://www.noisebridge.net/wiki/DreamTeam#Code
 
== Bayesian Inference ==
 
https://noisebridge.net/images/e/ef/Perception_is_in_the_Details12.pdf --
"Perception is in the Details: A Predictive Coding Account of the Psychedelic Phenomenon"
 
http://rsif.royalsocietypublishing.org/content/10/86/20130475 --
"Life as we know it"
 
http://jmlr.csail.mit.edu/proceedings/papers/v31/wang13b.pdf --
"Collapsed Variational Bayesian Inference for Hidden Markov Models"
 
http://www.datalab.uci.edu/papers/nips06_cvb.pdf --
"A Collapsed Variational Bayesian Inference Algorithm for Latent Dirichlet Allocation"
 
http://www.cnbc.cmu.edu/~tai/papers/lee_mumford_josa.pdf --
"Hierarchical Bayesian inference in the visual cortex"
 
https://www.researchgate.net/profile/Til_Bergmann/publication/262423308_Temporal_coding_organized_by_coupled_alpha_and_gamma_oscillations_prioritize_visual_processing/links/0deec537d1bfda474c000000/Temporal-coding-organized-by-coupled-alpha-and-gamma-oscillations-prioritize-visual-processing.pdf --
"Temporal coding organized by coupled alpha and gamma oscillations prioritize visual processing"
 
http://www.cell.com/neuron/pdf/S0896-6273(15)00823-5.pdf --
"Rhythms for Cognition: Communication through Coherence"
 
http://www.biorxiv.org/content/biorxiv/early/2014/05/06/004804.full.pdf --
"Visual Areas Exert Feedforward and Feedback Influences through Distinct Frequency Channels"
 
== Speech Recognition ==
 
https://arxiv.org/pdf/1612.00694v1 -- "ESE: Efficient Speech Recognition Engine with Compressed LSTM on FPGA"
 
== Sound Classification ==
 
https://arxiv.org/pdf/1608.04363v2 -- "Deep Convolutional Neural Networks and Data Augmentation for Environmental Sound Classification"
 
https://arxiv.org/pdf/1605.09507 "Deep convolutional neural networks for predominant instrument recognition in polyphonic music"
 
== Hardware Implementations - FPGA, GPU, etc ==
 
https://www.cse.iitk.ac.in/users/isaha/Publications/Journals/NC10.pdf --
"Artificial neural networks in hardware: A survey of two decades of progress"
 
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.133.9185&rep=rep1&type=pdf
"A Self-Repairing Multiplexer-Based FPGA Inspired by Biological Processes"
 
http://www.genetic-programming.com/jkpdf/fpga1998.pdf -- "Evolving Computer Programs using Rapidly Reconfigurable Field-Programmable Gate Arrays and Genetic Programming"
 
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.114.2588&rep=rep1&type=pdf -- "Flexible Implementation of Genetic Algorithms on FPGAs"
 
http://www.users.muohio.edu/jamiespa/html_papers/gem_10.pdf -- "Revisiting Genetic Algorithms for the FPGA Placement Problem"
 
https://arxiv.org/pdf/1609.09296v1 -- "Comprehensive Evaluation of OpenCL-based Convolutional Neural Network Accelerators in Xilinx and Altera FPGAs"
 
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.409.7533&rep=rep1&type=pdf -- "FPGA-TARGETED NEURAL ARCHITECTURE FOR EMBEDDED ALERTNESS DETECTION"
 
https://arxiv.org/pdf/1611.02450v1 -- "PipeCNN: An OpenCL-Based FPGA Accelerator for Large-Scale Convolution Neuron Networks"
 
https://arxiv.org/pdf/1605.06402v1 -- "Ristretto: Hardware-Oriented Approximation of Convolutional Neural Networks"
 
https://homes.cs.washington.edu/~luisceze/publications/snnap-hpca-2015.pdf -- "SNNAP: Approximate Computing on Programmable SoCs via Neural Acceleration"
 
https://arxiv.org/pdf/1701.00485v2 -- "Two-Bit Networks for Deep Learning on Resource-Constrained Embedded Devices"
 
== VLSI ==
 
http://ncs.ethz.ch/pubs/pdf/Indiveri_etal06.pdf -- "A VLSI array of low-power spiking neurons and bistable synapses with spike–timing dependent plasticity"
 
== Pruning ==
 
http://papers.nips.cc/paper/5784-learning-both-weights-and-connections-for-efficient-neural-network.pdf --
"Learning both Weights and Connections for Efficient Neural Networks"
 
https://arxiv.org/pdf/1701.04465 -- "The Incredible Shrinking Neural Network: New Perspectives on Learning Representations Through The Lens of Pruning"
 
https://arxiv.org/pdf/1512.08571 -- "Structured Pruning of Deep Convolutional Neural Networks"
 
https://arxiv.org/pdf/1611.01427 -- "Sparsely-Connected Neural Networks: Towards Efficient VLSI Implementation of Deep Neural Networks"
 
== Efficient Neural Networks via Compression, Quantization, Model Reduction, etc ==
 
https://arxiv.org/pdf/1504.04788 -- "Compressing Neural Networks with the Hashing Trick"
 
https://arxiv.org/pdf/1509.08745 -- "Compression of Deep Neural Networks on the Fly"
 
https://arxiv.org/pdf/1502.03436 -- "An Exploration of Parameter Redundancy in Deep Networks with Circulant Projections"
 
https://arxiv.org/pdf/1510.00149 -- "Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding"
 
https://arxiv.org/pdf/1612.00891 -- "Parameter Compression of Recurrent Neural Networks and Degradation of Short-term Memory"
 
https://arxiv.org/pdf/1609.07061 -- "Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations"
 
https://arxiv.org/pdf/1607.05418 -- "Runtime Configurable Deep Neural Networks for Energy-Accuracy Trade-off"
 
https://arxiv.org/pdf/1602.08194 -- "Scalable and Sustainable Deep Learning via Randomized Hashing"
 
https://arxiv.org/pdf/1508.05463 -- "StochasticNet: Forming Deep Neural Networks via Stochastic Connectivity"
 
https://arxiv.org/pdf/1412.7024 -- "Training Deep Neural Networks with Low Precision Multiplications"
 
https://arxiv.org/pdf/1612.03940 -- "Understanding the Impact of Precision Quantization on the Accuracy and Energy of Neural Networks"
 
https://arxiv.org/pdf/1609.00222 -- "Ternary Neural Networks for Resource-Efficient AI Applications"
 
== Neural Network Hyperparameter Optimization ==
 
https://arxiv.org/pdf/1601.00917 -- "DrMAD: Distilling Reverse-Mode Automatic Differentiation for Optimizing Hyperparameters of Deep Neural Networks"
 
== Neural Network based EEG Analysis ==
end
http://www.seas.upenn.edu/~wulsin/docs/wulsin2010.pdf -- "Semi-Supervised Anomaly Detection for EEG Waveforms Using Deep Belief Nets"
 
http://inter-eng.upm.ro/2012/files/proceedings/papers/paper72.pdf --
"Neural Network Parallelization on FPGA Platform for EEG Signal Classification"
 
== Seizure Detection ==
 
also see https://noisebridge.net/wiki/Kaggle for a (September 2016) current project!
 
and https://github.com/kevinjos/kaggle-aes-seizure-prediction (some earlier exploration, November 2014)
 
(broken link, sorry) http://www.sersc.org/journals/ijsip/vol7_no5/26.pdf --
"A Neural Network Model for Predicting Epileptic Seizures based on Fourier-Bessel Functions"
 
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2230664/pdf/nihms-34528.pdf --
"A Stochastic Framework for Evaluating Seizure Prediction Algorithms Using Hidden Markov Models"
 
(another broken link) http://cs.uni-muenster.de/Professoren/Lippe/diplomarbeiten/html/eisenbach/Untersuchte%20Artikel/PPHD+00.pdf --
"Recurrent neural network based preenddiction of epileptic seizures in intra- and extracranial EEG"
 
== Visible Light Sensor Network ==
 
http://infoteh.etf.unssa.rs.ba/zbornik/2016/radovi/KST-1/KST-1-15.pdf --
"Analysis of Visible Light Communication System for Implementation in Sensor Networks"
 
== Neurophysiology ==
 
http://www.buzsakilab.com/content/PDFs/BuzsakiKoch2012.pdf -- "The origin of extracellular fields and
currents — EEG, ECoG, LFP and spikes"
 
== Signal Processing ==
 
http://www.ti.com/lit/an/slyt438/slyt438.pdf -- "How delta-sigma ADCs work, Part 2"
 
http://provideyourown.com/2011/analogwrite-convert-pwm-to-voltage/ -- "Arduino’s AnalogWrite – Converting PWM to a Voltage"
 
http://sim.okawa-denshi.jp/en/PWMtool.php -- "RC Low-pass Filter Design for PWM (Transient Analysis Calculator)"
 
== Hyperdimensional Computing ==
 
http://redwood.berkeley.edu/pkanerva/papers/kanerva09-hyperdimensional.pdf --
"Hyperdimensional Computing: An Introduction to Computing in Distributed Representation with High-Dimensional Random Vectors"
 
http://arxiv.org/pdf/1602.03032.pdf --
"Associative Long Short-Term Memory"
 
== Bird Flocks and Maximum Entropy ==
 
http://arxiv.org/pdf/1107.0604v1 --
"Statistical Mechanics and Flocks of Birds"
 
https://arxiv.org/pdf/1311.2319.pdf -- "Statistical Mechanics of Surjective Cellular Automata"
 
https://pdfs.semanticscholar.org/fa84/5d15a54e99519d83a3ae1510200dc2eca471.pdf -- "Inhomogeneous Cellular Automata and Statistical Mechanics"
 
http://arxiv.org/pdf/1307.5563v1 --
"Social interactions dominate speed control in driving natural flocks toward criticality"
 
http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-050j-information-and-entropy-spring-2008/syllabus/MIT6_050JS08_penfield.pdf --
"Information and Entropy (Course Notes)"
 
== Whale Songs ==
 
https://arxiv.org/pdf/1307.0589.pdf -- "The Orchive : Data mining a massive bioacoustic archive"
 
https://www.researchgate.net/profile/Herbert_Roitblat/publication/13429327_The_neural_network_classification_of_false_killer_whale_%28Pseudorca_crassidens%29_vocalizations/links/540d2ff60cf2df04e75478cd.pdf -- "The neural network classification of false killer whale (Pseudorca crassidens) vocalizations"
 
http://users.iit.demokritos.gr/~paliourg/papers/PhD.pdf -- "REFINEMENT OF TEMPORAL CONSTRAINTS IN AN EVENT RECOGNITION SYSTEM USING SMALL DATASETS"
 
https://www.nersc.no/sites/www.nersc.no/files/master_thesis_sebastian_menze.pdf -- "Estimating fin whale distribution from ambient noise spectra using Bayesian inversion"
 
http://sis.univ-tln.fr/~glotin/IJCNN2015_IHMMbioac_BartChamGlot.pdf -- "Hierarchical Dirichlet Process Hidden Markov Model for Unsupervised Bioacoustic Analysis"
 
https://www.inf.ed.ac.uk/publications/thesis/online/IM030057.pdf -- "Hidden Markov Model Clustering of Acoustic Data"
 
http://danielnouri.org/notes/2014/01/10/using-deep-learning-to-listen-for-whales/ -- Using deep learning to listen for whales
 
== Computational Cognitive Neuroscience ==
 
http://www.pnas.org/content/110/41/16390.full -- "Indirection and symbol-like processing in the prefrontal cortex and basal ganglia"
 
http://ruccs.rutgers.edu/images/personal-zenon-pylyshyn/docs/jaf.pdf -- "Connectionism and Cognitive Architecture: A Critical Analysis"
 
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2728678/pdf/nihms131814.pdf -- Neves et al 2008 "Cell Shape and Negative Links in Regulatory Motifs Together Control Spatial Information Flow in Signaling Networks"
 
http://psych.colorado.edu/~oreilly/papers/AisaMingusOReilly08.pdf -- "The Emergent Neural Modeling System"
 
https://grey.colorado.edu/emergent/index.php/Comparison_of_Neural_Network_Simulators
 
== Text Generation ==
 
http://www.cs.toronto.edu/~ilya/pubs/2011/LANG-RNN.pdf -- "Generating Text with Recurrent Neural Networks"
 
== Games ==
 
http://setgame.com/sites/default/files/teacherscorner/COGNITIVE%20MODELING%20WITH%20SET.pdf -- "How to Construct a Believable Opponent using Cognitive Modeling in the Game of Set"
 
http://www-personal.umich.edu/~charchan/SET.pdf -- "SETs and Anti-SETs: The Math Behind the Game of SET"
 
http://personal.plattsburgh.edu/quenelgt/talks/set.pdf -- "Introduction to Set"
 
http://web.engr.illinois.edu/~pbg/papers/set.pdf -- "On the Complexity of the Game of Set"
 
http://www.warwick.ac.uk/staff/D.Maclagan/papers/set.pdf -- "The Card Game Set"
 
http://www.math.ucdavis.edu/~anne/FQ2011/set_game.pdf -- "The Game Set"
 
https://www.youtube.com/watch?v=k2rgzZ2WXKo -- "Best Practices for Procedural Narrative Generation" Chris Martens
 
== Large Scale Brain Simulation ==
 
http://www.nowere.net/b/arch/96550/src/1378907656268.pdf -- "A world survey of artificial brain projects, Part I: Large-scale brain simulations"
 
== Music ==
 
http://cmr.soc.plymouth.ac.uk/publications/bci-wkshop.pdf -- "ON GENERATING EEG FOR CONTROLLING MUSICAL SYSTEMS"
 
== Code ==
 
https://github.com/nbdt/gotrain (our ANN code)
 
https://github.com/nbdt/openbci-golang-server (EEG data acquisition and visualization software for OpenBCI)
 
https://github.com/Micah1/neurotech (brainduino code)
 
== Hidden Markov Models ==
 
http://www.cs.sjsu.edu/~stamp/RUA/HMM.pdf
-- "A Revealing Introduction to Hidden Markov Models"
 
http://www.jelmerborst.nl/pubs/Borst2013b.pdf
-- "Discovering Processing Stages by combining EEG with Hidden Markov Models"
 
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2230664/pdf/nihms-34528.pdf
-- "A Stochastic Framework for Evaluating Seizure Prediction Algorithms Using Hidden Markov Models"
 
http://www.schalklab.org/sites/default/files/misc/Coupled%20hidden%20markov%20model%20for%20electrocorticographic%20signal%20classification.pdf
-- "Coupled Hidden Markov Model for Electrocorticographic Signal Classification"
 
== Long Short Term Memory ==
 
http://deeplearning.cs.cmu.edu/pdfs/Hochreiter97_lstm.pdf
-- "Long Short-Term Memory"
 
ftp://ftp.idsia.ch/pub/juergen/2002_icannMusic.pdf
-- "Learning The Long-Term Structure of the Blues"
 
http://www.overcomplete.net/papers/nn2012.pdf
-- "A generalized LSTM-like training algorithm for second-order recurrent neural networks"
 
http://www.iro.umontreal.ca/~eckdoug/papers/2003_nn.pdf
-- "Kalman filters improve LSTM network performance in problems unsolvable by traditional recurrent nets"
 
http://googleresearch.blogspot.ch/2015/09/google-voice-search-faster-and-more.html
-- "Long Short-Term Memory dramatically improves Google Voice etc"
 
https://arxiv.org/pdf/1511.05552v4.pdf --
"Recurrent Neural Networks Hardware Implementation on FPGA"
 
http://vast.cs.ucla.edu/sites/default/files/publications/ASP-DAC2017-1352-11.pdf --
"FPGA-based Accelerator for Long Short-Term Memory Recurrent Neural Networks"
 
== Question Answering ==
 
http://www.overcomplete.net/papers/bica2012.pdf
-- "Neural Architectures for Learning to Answer Questions"
 
https://cs.umd.edu/~miyyer/pubs/2014_qb_rnn.pdf
-- "A Neural Network for Factoid Question Answering over Paragraphs"
 
http://arxiv.org/pdf/1502.05698.pdf
-- "Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks"
 
http://www.peterschueller.com/pub/2014/2014_winograd_schemas_relevance_knowledge_graphs.pdf
-- "Tackling Winograd Schemas by Formalizing Relevance Theory in Knowledge Graphs"
 
http://ijcai.org/papers15/Papers/IJCAI15-190.pdf
-- "Towards Addressing the Winograd Schema Challenge — Building and Using a Semantic Parser and a Knowledge Hunting Module"
 
http://arxiv.org/pdf/1506.05869v2.pdf
-- "A Neural Conversational Model"
 
http://arxiv.org/pdf/1508.05508v1.pdf
-- "Towards Neural Network-based Reasoning"
 
http://www.visualqa.org/vqa_iccv2015.pdf
-- "VQA: Visual Question Answering"
 
== Propagators ==
Cells must support three operations:
*add some content
*collect the content currently accumulated
*register a propagator to be notified when the accumulated content changes
*When new content is added to a cell, the cell must merge the addition with the content already present. When a propagator asks for the content of a cell, the cell must deliver a complete summary of the information that has been added to it.
*The merging of content must be commutative, associative, and idempotent. The behavior of propagators must be monotonic with respect to the lattice induced by the merge operation.
*http://web.mit.edu/~axch/www/art.pdf *http://groups.csail.mit.edu/mac/users/gjs/propagators/
*http://dustycloud.org/blog/sussman-on-ai/
 
== Boosting ==
 
http://www.cs.princeton.edu/courses/archive/spr08/cos424/readings/Schapire2003.pdf
-- "The Boosting Approach to Machine Learning An Overview"
 
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.19.9955&rep=rep1&type=pdf
-- "Ensembling Neural Networks: Many Could Be Better Than All"
 
http://www.cs.columbia.edu/~rocco/Public/mlj9.pdf
-- "Random Classification Noise Defeats All Convex Potential Boosters"
 
== Support Vector Machines ==
 
http://research.microsoft.com/en-us/um/people/cburges/papers/svmtutorial.pdf
-- "A Tutorial on Support Vector Machines for Pattern Recognition"
 
== Wire Length / Small World Networks ==
 
http://koulakovlab.cshl.edu/publications/koulakov_chklovskii_2000.pdf
-- "A wire length minimization approach to ocular dominance patterns in mammalian visual cortex"
 
http://papers.nips.cc/paper/1910-foundations-for-a-circuit-complexity-theory-of-sensory-processing.pdf
-- "Foundations for a Circuit Complexity Theory of Sensory Processing"
 
https://www.nada.kth.se/~cjo/documents/small_world.pdf
-- "Small-World Connectivity and Attractor Neural Networks"
 
http://www.doc.ic.ac.uk/~mpsha/ShanahanPhysRevEPreprint.pdf
-- "The Dynamical Complexity of Small-World Networks of Spiking Neurons"
 
http://www.dam.brown.edu/people/elie/papers/small_world.pdf
-- "Autoassociative Memory Retrieval and Spontaneous Activity Bumps in Small-World Networks of Integrate-and-Fire Neurons"
 
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.332.4501&rep=rep1&type=pdf
-- "Transition from Random to Small-World Neural Networks by STDP Learning Rule"
 
http://tamar.tau.ac.il/~eshel/papers/J_Neural_Engineering.pdf
-- "Compact self-wiring in cultured neural networks"
 
== Backpropagation ==
 
http://page.mi.fu-berlin.de/rojas/neural/neuron.pdf -- "Neural Networks - A Systematic Introduction"
 
http://www4.rgu.ac.uk/files/chapter3%20-%20bp.pdf (works through simple example)
 
http://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/
 
http://page.mi.fu-berlin.de/rojas/neural/chapter/K7.pdf
 
also see http://page.mi.fu-berlin.de/rojas/neural/neuron.pdf (entire book "Neural Networks - a Systemic Introduction" by Raul Rojas)
 
http://work.caltech.edu/lectures.html Hoeffding's inequality, VC Dimension and Back Propagation ANN
 
http://www.cs.stir.ac.uk/research/publications/techreps/pdf/TR148.pdf ("Learning XOR: exploring the space of a classic problem")
 
http://www.sysc.pdx.edu/classes/Werbos-Backpropagation%20through%20time.pdf
-- "Backpropagation Through Time: What it Does and How to Do It"
 
== Computer Vision ==
 
http://www.cv-foundation.org/openaccess/CVPR2015.py -- some interesting papers here
 
http://www.cv-foundation.org/openaccess/content_cvpr_2015/papers/Nguyen_Deep_Neural_Networks_2015_CVPR_paper.pdf -- "Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images"
 
== Visual Perception (Biological Systems) ==
 
http://cbcl.mit.edu/publications/ps/Serre_etal_PBR07.pdf
-- "A quantitative theory of immediate visual recognition"
 
http://www.dam.brown.edu/ptg/REPORTS/Invariance.pdf
-- "Invariance and Selectivity in the Ventral Visual Pathway"
 
http://www.cnbc.cmu.edu/~tai/papers/lee_mumford_josa.pdf
-- "Hierarchical Bayesian inference in the visual cortex"
 
== Neural Synchrony ==
 
http://arxiv.org/pdf/1312.6115.pdf
-- "Neuronal Synchrony in Complex-Valued Deep Networks"
 
== Spiking Neural Networks ==
 
http://ncs.ethz.ch/projects/evospike/publications/ICONIP2011%20Springer%20LNCS%20Nutta.pdf -- "EEG Classification with BSA Spike Encoding Algorithm and Evolving Probabilistic Spiking Neural Network"
 
http://dai.fmph.uniba.sk/~benus/confers/CNGM_ICANN05.pdf -- "Computational Neurogenetic Modeling: Integration of Spiking Neural Networks, Gene Networks, and Signal Processing Techniques"
 
http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.97.3902 -- "Pattern Recognition in a Bucket"
 
http://www.igi.tugraz.at/maass/psfiles/221.pdf -- "Noise as a Resource for Computation and Learning in Spiking Neural Networks"
 
http://web.stanford.edu/group/brainsinsilicon/goals.html -- Neurogrid
 
http://apt.cs.manchester.ac.uk/projects/SpiNNaker/ -- SpiNNaker
 
http://personalpages.manchester.ac.uk/staff/javier.navaridas/pubs/2012.jpdc.pdf -- more on SpiNNaker
 
http://apt.cs.manchester.ac.uk/ftp/pub/apt/papers/JavN_ICS09.pdf -- "Understanding the Interconnection Network of SpiNNaker"
 
==Hierarchical Temporal Memory==
 
https://github.com/numenta/nupic/wiki/Hierarchical-Temporal-Memory-Theory
 
http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1000532 -- "Towards a Mathematical Theory of Cortical Micro-circuits"
 
==Distributed Neural Networks==
 
https://www.paypal-engineering.com/2015/01/12/deep-learning-on-hadoop-2-0-2/ -- paypal engineering blog, paper describes implementation of neural network as described by [http://www.maths.bris.ac.uk/~maxvd/reduce_dim.pdf Hinton 2006] on Hadoop
 
http://www.sciencemag.org/content/suppl/2006/08/04/313.5786.504.DC1/Hinton.SOM.pdf -- supplementary material for [http://www.maths.bris.ac.uk/~maxvd/reduce_dim.pdf Hinton 2006]
 
http://www.cs.otago.ac.nz/staffpriv/hzy/papers/pdcs03.pdf -- "Parallelization of a Backpropagation Neural Network on a Cluster Computer"
 
http://arxiv.org/pdf/1404.5997v2.pdf -- "One weird trick for parallelizing convolutional neural networks"
 
http://www.ics.uci.edu/~jmoorkan/pub/gpusnn-ijcnn.pdf -- "Efficient Simulation of Large-Scale Spiking Neural Networks Using CUDA Graphics Processors"
 
==Mixture of Experts==
 
http://www.cs.toronto.edu/~fritz/absps/jjnh91.pdf -- "Adaptive Mixtures of Local Experts"
 
==Hopfield nets and RBMs==
 
http://www.iro.umontreal.ca/~bengioy/papers/ftml_book.pdf -- Yoshua Bengio, 2009, Learning Deep Architectures for AI
 
http://deeplearning.cs.cmu.edu/ -- Syllabus for cs course on deep learning, possible source of literature for the library
 
https://class.coursera.org/neuralnets-2012-001/lecture -- Lecture 11 - 15 to aid in understanding of Hinton 2006 paper above
 
http://www.pnas.org/content/79/8/2554.full.pdf -- "Neural networks and physical systems with emergent collective computational abilities" (Hopfield 1982)
 
http://page.mi.fu-berlin.de/rojas/neural/chapter/K13.pdf -- "The Hopfield Model" (Rojas 1996)
 
http://www.seas.upenn.edu/~wulsin/docs/wulsin2010.pdf -- "Semi-Supervised Anomaly Detection for EEG Waveforms Using Deep Belief Nets"
 
http://www.cse.buffalo.edu/DBGROUP/bioinformatics/papers/cameraready_xwjiabibe.pdf -- "A Novel Semi-supervised Deep Learning Framework
for Affective State Recognition on EEG Signals"
 
http://www.cs.toronto.edu/~hinton/absps/guideTR.pdf -- "A Practical Guide to Training Restricted Boltzmann Machines"
 
http://arxiv.org/pdf/1503.07793v2.pdf
-- "Gibbs Sampling with Low-Power Spiking Digital Neurons"
 
http://arxiv.org/pdf/1311.0190v1 -- "On the typical properties of inverse problems in statistical mechanics" Iacopo Mastromatteo 2013
 
http://jmlr.org/proceedings/papers/v5/salakhutdinov09a/salakhutdinov09a.pdf -- "Deep Boltzmann Machines" Salakhutdinov & Hinton 2009
 
http://www.cs.toronto.edu/~hinton/absps/tr00-004.pdf -- "Training Products of Experts by Minimizing Contrastive Divergence"
 
http://www.eecg.toronto.edu/~pc/research/publications/ly.fpga2009.submitted.pdf -- "A High-Performance FPGA Architecture for Restricted
Boltzmann Machines" Ly & Chow 2009
 
https://pdfs.semanticscholar.org/85fa/f7c3c05388e2bcd097a416606bdd88fc0c7c.pdf -- "A MULTI-FPGA ARCHITECTURE FOR STOCHASTIC RESTRICTED BOLTZMANN MACHINES" Ly & Chow 2009
 
== Variational Renormalization ==
 
https://arxiv.org/pdf/1410.3831 -- "An exact mapping between the Variational Renormalization Group and Deep Learning"
 
== Neuromorphic Stuff ==
 
https://arxiv.org/pdf/1508.01008.pdf -- "INsight: A Neuromorphic Computing System for Evaluation of Large Neural Networks" Chung, Shin & Kang 2015
 
== Markov Chain Monte Carlo ==
 
http://www.cs.princeton.edu/courses/archive/spr06/cos598C/papers/AndrieuFreitasDoucetJordan2003.pdf
-- "An Introduction to MCMC for Machine Learning"
 
http://jmlr.org/proceedings/papers/v37/salimans15.pdf
-- "Markov Chain Monte Carlo and Variational Inference: Bridging the Gap"
 
https://www.umiacs.umd.edu/~resnik/pubs/LAMP-TR-153.pdf -- "Gibbs Sampling for the Uninitiated"
 
==Entrainment==
 
http://www.scirp.org/Journal/PaperDownload.aspx?paperID=33251 -- "Alpha Rhythms Response to 10 Hz Flicker Is Wavelength Dependent"
 
http://www.brainmachine.co.uk/wp-content/uploads/Herrmann_Flicker.pdf -- "EEG responses to 1–100 Hz flicker: resonance phenomena in visual cortex and their potential correlation to cognitive phenomena"
 
http://www.jneurosci.org/content/23/37/11621.full.pdf -- "Human Cerebral Activation during Steady-State Visual-Evoked Responses"
 
http://www.dauwels.com/Papers/CogDyn%202009.pdf -- "On the synchrony of steady state visual evoked potentials and oscillatory burst events"
 
https://www.tu-ilmenau.de/fileadmin/public/lorentz-force/publications/peer/2012/haueisen2012/Halbleib_JCN_2012_Topographic_analysis_photic_driving.pdf -- "Topographic Analysis of Engagement and Disengagement of Neural Oscillators in Photic Driving: A Combined Electroencephalogram/Magnetoencephalogram Study"
 
==Mining Scientific Literature==
 
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC153503/pdf/1471-2105-4-11.pdf -- "PreBIND and Textomy – mining the biomedical literature for protein-protein interactions using a support vector machine" Donaldson 2003 BMC Bioinformatics
 
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4674139/pdf/pcbi.1004630.pdf -- "Text Mining for Protein Docking" Badal 2015 PLoS Comput Biol.
 
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4691339/pdf/bav116.pdf -- "Biocuration with insufficient resources and fixed timelines" Rodriguez-Esteban 2015 Database: The Journal of Biological Databases and Curation
 
==(not necessarilly very) Current Discussion==
 
re Tononi's "Integrated Information Theory" http://www.scottaaronson.com/blog/?p=1799
 
(19 February 2014) starting to think about possibility for experiments (loosely) related to [https://en.wikipedia.org/wiki/Visual_evoked_potential Visual Evoked Potential] research again - for instance:
 
http://www.biosemi.com/publications/pdf/Bierman.pdf -- Wave function collapse
 
[[File:Schak99InstantaneousCoherenceStroopTask.pdf]] -- "Instantaneous EEG Coherence Analysis During the Stroop Task" -- Schack et al 1999
 
[[File:CoherentEEGAmbiguousFigureBinding.pdf]] -- "Coherent EEG Indicators of Cognitive Binding During Ambiguous Figure Tasks" -- Klemm, Li, and Hernandez 2000
 
Note these two papers flog coherence measures - not trying to focus so much on that analysis right now, more interested in general understanding of what these experiments are about with possible goal of designing simpler experiments & analysis of similar perceptual/cognitive phenomena.
 
Here is an article that looks more directly at visual evoked potential measures:
 
[[File:ERP_Stereoscopic.pdf]] -- "Evoked Responses to Distinct and Nebulous Stereoscopic Stimuli" -- Dunlop et al 1983
 
 
(11 September 2013) more on analysis methods:
 
http://slesinsky.org/brian/misc/eulers_identity.html


http://www.dspguide.com/ch8/1.htm
http://www.dspguide.com/ch8/1.htm
Line 68: Line 863:
(following previous discussion) - we might select a few to study in more depth
(following previous discussion) - we might select a few to study in more depth
(... or not!  Plenty more to explore - suggestions (random or otherwise) are welcome.
(... or not!  Plenty more to explore - suggestions (random or otherwise) are welcome.
http://www.meltingasphalt.com/neurons-gone-wild/ --
Neurons Gone Wild - Levels of agency in the brain.


'''stereoscopic perception:'''
'''stereoscopic perception:'''
Line 134: Line 932:


[[File:NeuroSkyCommunicationsProtocol.pdf‎]]
[[File:NeuroSkyCommunicationsProtocol.pdf‎]]
==Android Neutral Network Fuzzy Learning app==
[https://play.google.com/store/apps/details?id=com.faadooengineers.free_neuralnetworkandfuzzysystems Android Neutral Network Fuzzy Learning app in Play Store]
==Learning about Neural Networks==
* What type of network? [http://stats.stackexchange.com/questions/114385/what-is-the-difference-between-convolutional-neural-networks-restricted-boltzma RMB (Restricted Boltzmann Machine) vs Autoencoder/MLP vs CNN (Convolutional Neural Networks)]
* Andrej Karpathy's [http://cs.stanford.edu/people/karpathy/convnetjs/ Convolutional Neural Network coded in JavaScript (ConvNetJS)]
* Andrej Karpathy's [http://karpathy.github.io/2015/10/25/selfie/ What a Deep Neural Network thinks about your #selfie  (background on Convolutional Neural Networks for image recognition and classification)]
* [https://blog.webkid.io/neural-networks-in-javascript/ Neural Networks in JavaScript w/MNIST]
* [http://www.antoniodeluca.info/blog/10-08-2016/neural-networks-in-javascript.html Another NN in JS]
* [http://caza.la/synaptic/ The Synaptic "architecture-free" neural network library in JS]

Latest revision as of 00:02, 9 January 2020

This is essentially the groups meeting notes – a trail of bread crumbs of topics of conversation and projects entertained by the group

note: Learn more about previous neuro research at Noisebridge on the wiki... For example, the Analog_EEG_Amp page describes some project ideas and work done by others here in 2012

Websites and events that have piqued our interest[edit]

http://cs375.stanford.edu/ -- Dan Yamins Large-Scale Neural Network Models for Neuroscience CS375

https://faculty.washington.edu/chudler/facts.html -- brain facts

http://onlinehub.stanford.edu/cs224 -- Natural Language Processing with Deep Learning

http://neurable.com/

https://noisebridge.net/wiki/NBDSM -- noiseBridge Deepnet and Statistical Mechanics -- first meetup @ noisebridge 7/6/17 at 7PM

https://metacademy.org/ -- machine learning knowledge graph

https://machinelearningguide.libsyn.com/rss -- machine learning guide podcast

http://www.thetalkingmachines.com/ -- podcast

https://karpathy.github.io/2015/05/21/rnn-effectiveness/

http://alexandre.barachant.org/papers/

http://ncs.ethz.ch/publications -- neuromorphic cognitive systems

https://github.com/crillab/gophersat/blob/master/examples/sat-for-noobs.md -- SAT solvers

https://media.ccc.de/v/34c3-8948-low_cost_non-invasive_biomedical_imaging -- Open EIT 34c3 talk https://github.com/OpenEIT

http://acrovirt.org/ -- sensors

http://www.neuroeducate.com/ -- citizen neuroscience

https://www.youtube.com/watch?v=9mZuyUzyN4Q -- "Categories for the Working Hacker"

http://radicalsciencenews.org/599-2/ -- "Deep Learning Fuels Nvidia’s Self-Driving Car Technology"

https://arxiv.org/abs/1803.03635 -- "The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks" Jonathan Frankle, Michael Carbin

Global Workspace Theory[edit]

http://bernardbaars.pbworks.com/f/BaarsJCS1997.pdf -- "IN THE THEATRE OF CONSCIOUSNESS"


Symbolic Mathematics[edit]

https://arxiv.org/pdf/1912.01412.pdf -- "Deep Learning for Symbolic Mathematics"

https://www.scottaaronson.com/busybeaver.pdf -- "A Relatively Small Turing Machine Whose Behavior Is Independent of Set Theory"

https://www.scottaaronson.com/blog/?p=2725 -- "The 8000th Busy Beaver number eludes ZF set theory: new paper by Adam Yedidia and me"

Vision[edit]

https://webvision.med.utah.edu/

Language Models[edit]

https://blog.scaleway.com/2019/building-a-machine-reading-comprehension-system-using-the-latest-advances-in-deep-learning-for-nlp/ -- "Natural Language Processing: the age of Transformers"

https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf -- "Language Models are Unsupervised Multitask Learners"

https://arxiv.org/pdf/1706.03762 -- "Attention Is All You Need"

https://arxiv.org/pdf/1705.03122.pdf -- "Convolutional Sequence to Sequence Learning"

https://papers.nips.cc/paper/5346-sequence-to-sequence-learning-with-neural-networks.pdf -- "Sequence to Sequence Learning with Neural Networks"

https://arxiv.org/pdf/1508.07909.pdf -- "Neural Machine Translation of Rare Words with Subword Units"

https://github.com/rsennrich/subword-nmt

https://github.com/rowanz/grover - grover GPU/TPU based GPT-2 transformer implementation

https://arxiv.org/pdf/1905.12616.pdf -- "Defending Against Neural Fake News"

https://lilianweng.github.io/lil-log/2018/06/24/attention-attention.html - blog post on attention

https://github.com/lilianweng/transformer-tensorflow - sample implementation of "Attention Is All You Need"

https://github.com/tensorflow/tensor2tensor/blob/master/tensor2tensor/models/transformer.py - "official" implementation of "Attention Is All You Need"

https://jalammar.github.io/illustrated-gpt2/ - The Illustrated GPT-2 (Visualizing Transformer Language Models)

Bioengineering[edit]

https://nips.cc/Conferences/2018/Schedule?showEvent=12487 -- "What Bodies Think About: Bioelectric Computation Outside the Nervous System, Primitive Cognition, and Synthetic Morphology"

Data Visualization[edit]

https://www.csc2.ncsu.edu/faculty/healey/download/tvcg.12b.pdf -- "Interest Driven Navigation in Visualization"

Fractal Dementia[edit]

https://pdfs.semanticscholar.org/0018/7c742e60d35d5034a63251e31e1b8d96c70b.pdf -- "Comparison of Fractal Dimension Algorithms for the Computation of Eeg Biomarkers for Dementia"

Brain Activity Dynamics[edit]

https://arxiv.org/pdf/1802.02523.pdf -- "Plasma Brain Dynamics (PBD): a Mechanism for EEG Waves Under Human Consciousness"

https://arxiv.org/pdf/1206.1108.pdf -- "Thermodynamic Model of Criticality in the Cortex Based On EEG/ECOG Data"

https://www.bm-science.com/images/bms/publ/art63.pdf -- "Topographic Mapping of Rapid Transitions in EEG Multiple Frequencies"

Silent Speech[edit]

https://dam-prod.media.mit.edu/x/2018/03/23/p43-kapur_BRjFwE6.pdf -- "AlterEgo: A Personalized Wearable Silent Speech Interface"

Image Reconstruction[edit]

https://www.biorxiv.org/content/biorxiv/early/2017/12/28/240317.full.pdf -- "Deep image reconstruction from human brain activity"

EEG Electrodes[edit]

https://sites.google.com/site/biofeedbackpages/velcro-sensors -- Saline electrodes

https://www.commsp.ee.ic.ac.uk/~mandic/Ear_EEG_IEEE_Pulse_2012.pdf -- "The In-the-Ear Recording Concept"

https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8357918 -- "Dry-Contact Electrode Ear-EEG"

Generative Adversarial Networks (GAN)[edit]

https://arxiv.org/pdf/1710.08864 -- "One pixel attack for fooling deep neural networks"

Kolmolgorov Complexity[edit]

ftp://ftp.idsia.ch/pub/juergen/loconet.pdf -- "Discovering Neural Nets with Low Kolmolgorov Complexity and High Generalization Capability"

https://papers.nips.cc/paper/394-chaitin-kolmogorov-complexity-and-generalization-in-neural-networks.pdf -- "Chaitin-Kolmogorov Complexity and Generalization in Neural Networks"

OpenCV[edit]

http://arnab.org/blog/so-i-suck-24-automating-card-games-using-opencv-and-python

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.185.585&rep=rep1&type=pdf -- "Pyramidal Implementation of the Lucas Kanade Feature Tracker Description of the algorithm"

Category Theory[edit]

https://arxiv.org/pdf/1711.10455 -- "Backprop as Functor: A compositional perspective on supervised learning"

http://math.ucr.edu/home/baez/rosetta.pdf -- "Physics, Topology, Logic and Computation: A Rosetta Stone"

https://www.youtube.com/watch?v=BF6kHD1DAeU -- "Category theory foundations 1.0 — Steve Awodey"

Proof Searcher[edit]

https://arxiv.org/pdf/cs/0207097 -- "Optimal Ordered Problem Solver"

http://people.idsia.ch/~juergen/ultimatecognition.pdf -- "Ultimate Cognition a la Gödel"

http://people.idsia.ch/~juergen/selfreflection.pdf -- "Towards an Actual Gödel Machine Implementation"

Capsule Models[edit]

https://arxiv.org/pdf/1710.09829.pdf -- "Dynamic Routing Between Capsules"

https://openreview.net/pdf?id=HJWLfGWRb -- "Matrix Capsules with EM Routing"

Multivariate Coherence Training[edit]

https://www.youtube.com/watch?v=qGYjvLki0WY

Infrared Neuroimaging[edit]

http://www.ecse.rpi.edu/~yazici/bio_book.pdf -- "Near Infrared Imaging and Spectroscopy for Brain Activity Monitoring"

http://fangyenlab.seas.upenn.edu/pubs/isr.pdf -- "Intrinsic optical signals in neural tissues: measurements, mechanisms, and applications"

Geometry[edit]

http://arxiv.org/abs/1710.10784 -- "How deep learning works --The geometry of deep learning"

Affective Computing[edit]

http://affect.media.mit.edu/pdfs/05.ahn-picard-acii.pdf -- "Affective-Cognitive Learning and Decision Making: A Motivational Reward Framework For Affective Agents"

Explainability[edit]

http://arxiv.org/abs/1708.01785 -- "Interpreting CNN knowledge via an Explanatory Graph"

NLP[edit]

https://arxiv.org/pdf/1605.06640 -- "Programming with a Differentiable Forth Interpreter"

https://pdfs.semanticscholar.org/f683/dbe8a22d633ad3a2cff379b055b26684a838.pdf -- "Solving General Arithmetic Word Problems"

https://arxiv.org/pdf/1611.04558.pdf -- "Google’s Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation"

http://emnlp2014.org/papers/pdf/EMNLP2014162.pdf -- "GloVe: Global Vectors for Word Representation"

RNNs[edit]

https://arxiv.org/pdf/1611.01576.pdf -- "Quasi Recurrent Neural Networks"

Hyper-parameter Optimization[edit]

https://arxiv.org/abs/1603.06560 -- "Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization"

Transfer Learning[edit]

http://arxiv.org/abs/1710.10776v1 -- "Transfer Learning to Learn with Multitask Neural Model Search"

Reinforcement Learning[edit]

http://www2.hawaii.edu/~sstill/StillPrecup2011.pdf -- "An information-theoretic approach to curiosity-driven reinforcement learning"

https://arxiv.org/abs/1605.06676 -- "Learning to Communicate with Deep Multi-Agent Reinforcement Learning"

Learning to Learn[edit]

https://arxiv.org/pdf/1703.01041.pdf -- "Large-Scale Evolution of Image Classifiers"

https://arxiv.org/pdf/1611.01578 -- "Neural Architecture Search with Reinforcement Learning"

The Utility of "Noise" in ML[edit]

https://www.cs.toronto.edu/~hinton/absps/JMLRdropout.pdf -- "Dropout: A Simple Way to Prevent Neural Networks from Overfitting"

http://yann.lecun.com/exdb/publis/pdf/lecun-90b.pdf -- "Optimal Brain Damage"

https://arxiv.org/pdf/1502.01852.pdf -- "Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification"

One-shot learning[edit]

https://arxiv.org/abs/1605%2E06065 -- "One-shot Learning with Memory-Augmented Neural Networks"

Program Synthesis[edit]

https://pdfs.semanticscholar.org/0163/35ce7e0a073623e1deac7138b28913dbf594.pdf -- "Human-level concept learning through probabilistic program induction"

https://arxiv.org/pdf/1511.06279.pdf -- "Neural Programmer: Inducing Latent Programs with Gradient Descent"

https://arxiv.org/abs/1608.04428 -- "TerpreT: A Probabilistic Programming Language for Program Induction" Gaunt et al 2016

Machine Learning Interaction[edit]

https://teachablemachine.withgoogle.com/#

Game Theory[edit]

https://arxiv.org/abs/1707.01068v1 - Maintaining cooperation in complex social dilemmas using deep reinforcement learning

Questions of Physics and Free Will[edit]

http://www.scottaaronson.com/papers/giqtm3.pdf - The Ghost in the Quantum Turing Machine

CNN[edit]

https://adeshpande3.github.io/adeshpande3.github.io/A-Beginner's-Guide-To-Understanding-Convolutional-Neural-Networks/ - "A Beginner's Guide To Understanding Convolutional Neural Networks"

https://adeshpande3.github.io/A-Beginner's-Guide-To-Understanding-Convolutional-Neural-Networks-Part-2/ - "A Beginner's Guide To Understanding Convolutional Neural Networks Part 2"

http://scs.ryerson.ca/~aharley/vis/harley_vis_isvc15.pdf -- "An Interactive Node-Link Visualization of Convolutional Neural Networks"

http://white.stanford.edu/teach/index.php/An_Introduction_to_Convolutional_Neural_Networks

http://www.cv-foundation.org/openaccess/content_cvpr_2015/papers/Dosovitskiy_Learning_to_Generate_2015_CVPR_paper.pdf -- "Learning to Generate Chairs With Convolutional Neural Networks"

http://www.pamitc.org/cvpr15/files/lecun-20150610-cvpr-keynote.pdf -- "What's Wrong With Deep Learning?"

Mind-Body Relations[edit]

http://www.pnas.org/content/111/20/7379.full.pdf -- "Voluntary activation of the sympathetic nervous system and attenuation of the innate immune response in humans"

Math[edit]

https://arxiv.org/pdf/1311.1090.pdf -- "Polyhedrons and Perceptrons Are Functionally Equivalent"

Example code and training data using polyhedrons developed by author of above paper: https://www.noisebridge.net/wiki/DreamTeam#Code

Bayesian Inference[edit]

https://noisebridge.net/images/e/ef/Perception_is_in_the_Details12.pdf -- "Perception is in the Details: A Predictive Coding Account of the Psychedelic Phenomenon"

http://rsif.royalsocietypublishing.org/content/10/86/20130475 -- "Life as we know it"

http://jmlr.csail.mit.edu/proceedings/papers/v31/wang13b.pdf -- "Collapsed Variational Bayesian Inference for Hidden Markov Models"

http://www.datalab.uci.edu/papers/nips06_cvb.pdf -- "A Collapsed Variational Bayesian Inference Algorithm for Latent Dirichlet Allocation"

http://www.cnbc.cmu.edu/~tai/papers/lee_mumford_josa.pdf -- "Hierarchical Bayesian inference in the visual cortex"

https://www.researchgate.net/profile/Til_Bergmann/publication/262423308_Temporal_coding_organized_by_coupled_alpha_and_gamma_oscillations_prioritize_visual_processing/links/0deec537d1bfda474c000000/Temporal-coding-organized-by-coupled-alpha-and-gamma-oscillations-prioritize-visual-processing.pdf -- "Temporal coding organized by coupled alpha and gamma oscillations prioritize visual processing"

http://www.cell.com/neuron/pdf/S0896-6273(15)00823-5.pdf -- "Rhythms for Cognition: Communication through Coherence"

http://www.biorxiv.org/content/biorxiv/early/2014/05/06/004804.full.pdf -- "Visual Areas Exert Feedforward and Feedback Influences through Distinct Frequency Channels"

Speech Recognition[edit]

https://arxiv.org/pdf/1612.00694v1 -- "ESE: Efficient Speech Recognition Engine with Compressed LSTM on FPGA"

Sound Classification[edit]

https://arxiv.org/pdf/1608.04363v2 -- "Deep Convolutional Neural Networks and Data Augmentation for Environmental Sound Classification"

https://arxiv.org/pdf/1605.09507 "Deep convolutional neural networks for predominant instrument recognition in polyphonic music"

Hardware Implementations - FPGA, GPU, etc[edit]

https://www.cse.iitk.ac.in/users/isaha/Publications/Journals/NC10.pdf -- "Artificial neural networks in hardware: A survey of two decades of progress"

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.133.9185&rep=rep1&type=pdf "A Self-Repairing Multiplexer-Based FPGA Inspired by Biological Processes"

http://www.genetic-programming.com/jkpdf/fpga1998.pdf -- "Evolving Computer Programs using Rapidly Reconfigurable Field-Programmable Gate Arrays and Genetic Programming"

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.114.2588&rep=rep1&type=pdf -- "Flexible Implementation of Genetic Algorithms on FPGAs"

http://www.users.muohio.edu/jamiespa/html_papers/gem_10.pdf -- "Revisiting Genetic Algorithms for the FPGA Placement Problem"

https://arxiv.org/pdf/1609.09296v1 -- "Comprehensive Evaluation of OpenCL-based Convolutional Neural Network Accelerators in Xilinx and Altera FPGAs"

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.409.7533&rep=rep1&type=pdf -- "FPGA-TARGETED NEURAL ARCHITECTURE FOR EMBEDDED ALERTNESS DETECTION"

https://arxiv.org/pdf/1611.02450v1 -- "PipeCNN: An OpenCL-Based FPGA Accelerator for Large-Scale Convolution Neuron Networks"

https://arxiv.org/pdf/1605.06402v1 -- "Ristretto: Hardware-Oriented Approximation of Convolutional Neural Networks"

https://homes.cs.washington.edu/~luisceze/publications/snnap-hpca-2015.pdf -- "SNNAP: Approximate Computing on Programmable SoCs via Neural Acceleration"

https://arxiv.org/pdf/1701.00485v2 -- "Two-Bit Networks for Deep Learning on Resource-Constrained Embedded Devices"

VLSI[edit]

http://ncs.ethz.ch/pubs/pdf/Indiveri_etal06.pdf -- "A VLSI array of low-power spiking neurons and bistable synapses with spike–timing dependent plasticity"

Pruning[edit]

http://papers.nips.cc/paper/5784-learning-both-weights-and-connections-for-efficient-neural-network.pdf -- "Learning both Weights and Connections for Efficient Neural Networks"

https://arxiv.org/pdf/1701.04465 -- "The Incredible Shrinking Neural Network: New Perspectives on Learning Representations Through The Lens of Pruning"

https://arxiv.org/pdf/1512.08571 -- "Structured Pruning of Deep Convolutional Neural Networks"

https://arxiv.org/pdf/1611.01427 -- "Sparsely-Connected Neural Networks: Towards Efficient VLSI Implementation of Deep Neural Networks"

Efficient Neural Networks via Compression, Quantization, Model Reduction, etc[edit]

https://arxiv.org/pdf/1504.04788 -- "Compressing Neural Networks with the Hashing Trick"

https://arxiv.org/pdf/1509.08745 -- "Compression of Deep Neural Networks on the Fly"

https://arxiv.org/pdf/1502.03436 -- "An Exploration of Parameter Redundancy in Deep Networks with Circulant Projections"

https://arxiv.org/pdf/1510.00149 -- "Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding"

https://arxiv.org/pdf/1612.00891 -- "Parameter Compression of Recurrent Neural Networks and Degradation of Short-term Memory"

https://arxiv.org/pdf/1609.07061 -- "Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations"

https://arxiv.org/pdf/1607.05418 -- "Runtime Configurable Deep Neural Networks for Energy-Accuracy Trade-off"

https://arxiv.org/pdf/1602.08194 -- "Scalable and Sustainable Deep Learning via Randomized Hashing"

https://arxiv.org/pdf/1508.05463 -- "StochasticNet: Forming Deep Neural Networks via Stochastic Connectivity"

https://arxiv.org/pdf/1412.7024 -- "Training Deep Neural Networks with Low Precision Multiplications"

https://arxiv.org/pdf/1612.03940 -- "Understanding the Impact of Precision Quantization on the Accuracy and Energy of Neural Networks"

https://arxiv.org/pdf/1609.00222 -- "Ternary Neural Networks for Resource-Efficient AI Applications"

Neural Network Hyperparameter Optimization[edit]

https://arxiv.org/pdf/1601.00917 -- "DrMAD: Distilling Reverse-Mode Automatic Differentiation for Optimizing Hyperparameters of Deep Neural Networks"

Neural Network based EEG Analysis[edit]

end http://www.seas.upenn.edu/~wulsin/docs/wulsin2010.pdf -- "Semi-Supervised Anomaly Detection for EEG Waveforms Using Deep Belief Nets"

http://inter-eng.upm.ro/2012/files/proceedings/papers/paper72.pdf -- "Neural Network Parallelization on FPGA Platform for EEG Signal Classification"

Seizure Detection[edit]

also see https://noisebridge.net/wiki/Kaggle for a (September 2016) current project!

and https://github.com/kevinjos/kaggle-aes-seizure-prediction (some earlier exploration, November 2014)

(broken link, sorry) http://www.sersc.org/journals/ijsip/vol7_no5/26.pdf -- "A Neural Network Model for Predicting Epileptic Seizures based on Fourier-Bessel Functions"

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2230664/pdf/nihms-34528.pdf -- "A Stochastic Framework for Evaluating Seizure Prediction Algorithms Using Hidden Markov Models"

(another broken link) http://cs.uni-muenster.de/Professoren/Lippe/diplomarbeiten/html/eisenbach/Untersuchte%20Artikel/PPHD+00.pdf -- "Recurrent neural network based preenddiction of epileptic seizures in intra- and extracranial EEG"

Visible Light Sensor Network[edit]

http://infoteh.etf.unssa.rs.ba/zbornik/2016/radovi/KST-1/KST-1-15.pdf -- "Analysis of Visible Light Communication System for Implementation in Sensor Networks"

Neurophysiology[edit]

http://www.buzsakilab.com/content/PDFs/BuzsakiKoch2012.pdf -- "The origin of extracellular fields and currents — EEG, ECoG, LFP and spikes"

Signal Processing[edit]

http://www.ti.com/lit/an/slyt438/slyt438.pdf -- "How delta-sigma ADCs work, Part 2"

http://provideyourown.com/2011/analogwrite-convert-pwm-to-voltage/ -- "Arduino’s AnalogWrite – Converting PWM to a Voltage"

http://sim.okawa-denshi.jp/en/PWMtool.php -- "RC Low-pass Filter Design for PWM (Transient Analysis Calculator)"

Hyperdimensional Computing[edit]

http://redwood.berkeley.edu/pkanerva/papers/kanerva09-hyperdimensional.pdf -- "Hyperdimensional Computing: An Introduction to Computing in Distributed Representation with High-Dimensional Random Vectors"

http://arxiv.org/pdf/1602.03032.pdf -- "Associative Long Short-Term Memory"

Bird Flocks and Maximum Entropy[edit]

http://arxiv.org/pdf/1107.0604v1 -- "Statistical Mechanics and Flocks of Birds"

https://arxiv.org/pdf/1311.2319.pdf -- "Statistical Mechanics of Surjective Cellular Automata"

https://pdfs.semanticscholar.org/fa84/5d15a54e99519d83a3ae1510200dc2eca471.pdf -- "Inhomogeneous Cellular Automata and Statistical Mechanics"

http://arxiv.org/pdf/1307.5563v1 -- "Social interactions dominate speed control in driving natural flocks toward criticality"

http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-050j-information-and-entropy-spring-2008/syllabus/MIT6_050JS08_penfield.pdf -- "Information and Entropy (Course Notes)"

Whale Songs[edit]

https://arxiv.org/pdf/1307.0589.pdf -- "The Orchive : Data mining a massive bioacoustic archive"

https://www.researchgate.net/profile/Herbert_Roitblat/publication/13429327_The_neural_network_classification_of_false_killer_whale_%28Pseudorca_crassidens%29_vocalizations/links/540d2ff60cf2df04e75478cd.pdf -- "The neural network classification of false killer whale (Pseudorca crassidens) vocalizations"

http://users.iit.demokritos.gr/~paliourg/papers/PhD.pdf -- "REFINEMENT OF TEMPORAL CONSTRAINTS IN AN EVENT RECOGNITION SYSTEM USING SMALL DATASETS"

https://www.nersc.no/sites/www.nersc.no/files/master_thesis_sebastian_menze.pdf -- "Estimating fin whale distribution from ambient noise spectra using Bayesian inversion"

http://sis.univ-tln.fr/~glotin/IJCNN2015_IHMMbioac_BartChamGlot.pdf -- "Hierarchical Dirichlet Process Hidden Markov Model for Unsupervised Bioacoustic Analysis"

https://www.inf.ed.ac.uk/publications/thesis/online/IM030057.pdf -- "Hidden Markov Model Clustering of Acoustic Data"

http://danielnouri.org/notes/2014/01/10/using-deep-learning-to-listen-for-whales/ -- Using deep learning to listen for whales

Computational Cognitive Neuroscience[edit]

http://www.pnas.org/content/110/41/16390.full -- "Indirection and symbol-like processing in the prefrontal cortex and basal ganglia"

http://ruccs.rutgers.edu/images/personal-zenon-pylyshyn/docs/jaf.pdf -- "Connectionism and Cognitive Architecture: A Critical Analysis"

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2728678/pdf/nihms131814.pdf -- Neves et al 2008 "Cell Shape and Negative Links in Regulatory Motifs Together Control Spatial Information Flow in Signaling Networks"

http://psych.colorado.edu/~oreilly/papers/AisaMingusOReilly08.pdf -- "The Emergent Neural Modeling System"

https://grey.colorado.edu/emergent/index.php/Comparison_of_Neural_Network_Simulators

Text Generation[edit]

http://www.cs.toronto.edu/~ilya/pubs/2011/LANG-RNN.pdf -- "Generating Text with Recurrent Neural Networks"

Games[edit]

http://setgame.com/sites/default/files/teacherscorner/COGNITIVE%20MODELING%20WITH%20SET.pdf -- "How to Construct a Believable Opponent using Cognitive Modeling in the Game of Set"

http://www-personal.umich.edu/~charchan/SET.pdf -- "SETs and Anti-SETs: The Math Behind the Game of SET"

http://personal.plattsburgh.edu/quenelgt/talks/set.pdf -- "Introduction to Set"

http://web.engr.illinois.edu/~pbg/papers/set.pdf -- "On the Complexity of the Game of Set"

http://www.warwick.ac.uk/staff/D.Maclagan/papers/set.pdf -- "The Card Game Set"

http://www.math.ucdavis.edu/~anne/FQ2011/set_game.pdf -- "The Game Set"

https://www.youtube.com/watch?v=k2rgzZ2WXKo -- "Best Practices for Procedural Narrative Generation" Chris Martens

Large Scale Brain Simulation[edit]

http://www.nowere.net/b/arch/96550/src/1378907656268.pdf -- "A world survey of artificial brain projects, Part I: Large-scale brain simulations"

Music[edit]

http://cmr.soc.plymouth.ac.uk/publications/bci-wkshop.pdf -- "ON GENERATING EEG FOR CONTROLLING MUSICAL SYSTEMS"

Code[edit]

https://github.com/nbdt/gotrain (our ANN code)

https://github.com/nbdt/openbci-golang-server (EEG data acquisition and visualization software for OpenBCI)

https://github.com/Micah1/neurotech (brainduino code)

Hidden Markov Models[edit]

http://www.cs.sjsu.edu/~stamp/RUA/HMM.pdf -- "A Revealing Introduction to Hidden Markov Models"

http://www.jelmerborst.nl/pubs/Borst2013b.pdf -- "Discovering Processing Stages by combining EEG with Hidden Markov Models"

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2230664/pdf/nihms-34528.pdf -- "A Stochastic Framework for Evaluating Seizure Prediction Algorithms Using Hidden Markov Models"

http://www.schalklab.org/sites/default/files/misc/Coupled%20hidden%20markov%20model%20for%20electrocorticographic%20signal%20classification.pdf -- "Coupled Hidden Markov Model for Electrocorticographic Signal Classification"

Long Short Term Memory[edit]

http://deeplearning.cs.cmu.edu/pdfs/Hochreiter97_lstm.pdf -- "Long Short-Term Memory"

ftp://ftp.idsia.ch/pub/juergen/2002_icannMusic.pdf -- "Learning The Long-Term Structure of the Blues"

http://www.overcomplete.net/papers/nn2012.pdf -- "A generalized LSTM-like training algorithm for second-order recurrent neural networks"

http://www.iro.umontreal.ca/~eckdoug/papers/2003_nn.pdf -- "Kalman filters improve LSTM network performance in problems unsolvable by traditional recurrent nets"

http://googleresearch.blogspot.ch/2015/09/google-voice-search-faster-and-more.html -- "Long Short-Term Memory dramatically improves Google Voice etc"

https://arxiv.org/pdf/1511.05552v4.pdf -- "Recurrent Neural Networks Hardware Implementation on FPGA"

http://vast.cs.ucla.edu/sites/default/files/publications/ASP-DAC2017-1352-11.pdf -- "FPGA-based Accelerator for Long Short-Term Memory Recurrent Neural Networks"

Question Answering[edit]

http://www.overcomplete.net/papers/bica2012.pdf -- "Neural Architectures for Learning to Answer Questions"

https://cs.umd.edu/~miyyer/pubs/2014_qb_rnn.pdf -- "A Neural Network for Factoid Question Answering over Paragraphs"

http://arxiv.org/pdf/1502.05698.pdf -- "Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks"

http://www.peterschueller.com/pub/2014/2014_winograd_schemas_relevance_knowledge_graphs.pdf -- "Tackling Winograd Schemas by Formalizing Relevance Theory in Knowledge Graphs"

http://ijcai.org/papers15/Papers/IJCAI15-190.pdf -- "Towards Addressing the Winograd Schema Challenge — Building and Using a Semantic Parser and a Knowledge Hunting Module"

http://arxiv.org/pdf/1506.05869v2.pdf -- "A Neural Conversational Model"

http://arxiv.org/pdf/1508.05508v1.pdf -- "Towards Neural Network-based Reasoning"

http://www.visualqa.org/vqa_iccv2015.pdf -- "VQA: Visual Question Answering"

Propagators[edit]

Cells must support three operations:

  • add some content
  • collect the content currently accumulated
  • register a propagator to be notified when the accumulated content changes
  • When new content is added to a cell, the cell must merge the addition with the content already present. When a propagator asks for the content of a cell, the cell must deliver a complete summary of the information that has been added to it.
  • The merging of content must be commutative, associative, and idempotent. The behavior of propagators must be monotonic with respect to the lattice induced by the merge operation.
  • http://web.mit.edu/~axch/www/art.pdf *http://groups.csail.mit.edu/mac/users/gjs/propagators/
  • http://dustycloud.org/blog/sussman-on-ai/

Boosting[edit]

http://www.cs.princeton.edu/courses/archive/spr08/cos424/readings/Schapire2003.pdf -- "The Boosting Approach to Machine Learning An Overview"

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.19.9955&rep=rep1&type=pdf -- "Ensembling Neural Networks: Many Could Be Better Than All"

http://www.cs.columbia.edu/~rocco/Public/mlj9.pdf -- "Random Classification Noise Defeats All Convex Potential Boosters"

Support Vector Machines[edit]

http://research.microsoft.com/en-us/um/people/cburges/papers/svmtutorial.pdf -- "A Tutorial on Support Vector Machines for Pattern Recognition"

Wire Length / Small World Networks[edit]

http://koulakovlab.cshl.edu/publications/koulakov_chklovskii_2000.pdf -- "A wire length minimization approach to ocular dominance patterns in mammalian visual cortex"

http://papers.nips.cc/paper/1910-foundations-for-a-circuit-complexity-theory-of-sensory-processing.pdf -- "Foundations for a Circuit Complexity Theory of Sensory Processing"

https://www.nada.kth.se/~cjo/documents/small_world.pdf -- "Small-World Connectivity and Attractor Neural Networks"

http://www.doc.ic.ac.uk/~mpsha/ShanahanPhysRevEPreprint.pdf -- "The Dynamical Complexity of Small-World Networks of Spiking Neurons"

http://www.dam.brown.edu/people/elie/papers/small_world.pdf -- "Autoassociative Memory Retrieval and Spontaneous Activity Bumps in Small-World Networks of Integrate-and-Fire Neurons"

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.332.4501&rep=rep1&type=pdf -- "Transition from Random to Small-World Neural Networks by STDP Learning Rule"

http://tamar.tau.ac.il/~eshel/papers/J_Neural_Engineering.pdf -- "Compact self-wiring in cultured neural networks"

Backpropagation[edit]

http://page.mi.fu-berlin.de/rojas/neural/neuron.pdf -- "Neural Networks - A Systematic Introduction"

http://www4.rgu.ac.uk/files/chapter3%20-%20bp.pdf (works through simple example)

http://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/

http://page.mi.fu-berlin.de/rojas/neural/chapter/K7.pdf

also see http://page.mi.fu-berlin.de/rojas/neural/neuron.pdf (entire book "Neural Networks - a Systemic Introduction" by Raul Rojas)

http://work.caltech.edu/lectures.html Hoeffding's inequality, VC Dimension and Back Propagation ANN

http://www.cs.stir.ac.uk/research/publications/techreps/pdf/TR148.pdf ("Learning XOR: exploring the space of a classic problem")

http://www.sysc.pdx.edu/classes/Werbos-Backpropagation%20through%20time.pdf -- "Backpropagation Through Time: What it Does and How to Do It"

Computer Vision[edit]

http://www.cv-foundation.org/openaccess/CVPR2015.py -- some interesting papers here

http://www.cv-foundation.org/openaccess/content_cvpr_2015/papers/Nguyen_Deep_Neural_Networks_2015_CVPR_paper.pdf -- "Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images"

Visual Perception (Biological Systems)[edit]

http://cbcl.mit.edu/publications/ps/Serre_etal_PBR07.pdf -- "A quantitative theory of immediate visual recognition"

http://www.dam.brown.edu/ptg/REPORTS/Invariance.pdf -- "Invariance and Selectivity in the Ventral Visual Pathway"

http://www.cnbc.cmu.edu/~tai/papers/lee_mumford_josa.pdf -- "Hierarchical Bayesian inference in the visual cortex"

Neural Synchrony[edit]

http://arxiv.org/pdf/1312.6115.pdf -- "Neuronal Synchrony in Complex-Valued Deep Networks"

Spiking Neural Networks[edit]

http://ncs.ethz.ch/projects/evospike/publications/ICONIP2011%20Springer%20LNCS%20Nutta.pdf -- "EEG Classification with BSA Spike Encoding Algorithm and Evolving Probabilistic Spiking Neural Network"

http://dai.fmph.uniba.sk/~benus/confers/CNGM_ICANN05.pdf -- "Computational Neurogenetic Modeling: Integration of Spiking Neural Networks, Gene Networks, and Signal Processing Techniques"

http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.97.3902 -- "Pattern Recognition in a Bucket"

http://www.igi.tugraz.at/maass/psfiles/221.pdf -- "Noise as a Resource for Computation and Learning in Spiking Neural Networks"

http://web.stanford.edu/group/brainsinsilicon/goals.html -- Neurogrid

http://apt.cs.manchester.ac.uk/projects/SpiNNaker/ -- SpiNNaker

http://personalpages.manchester.ac.uk/staff/javier.navaridas/pubs/2012.jpdc.pdf -- more on SpiNNaker

http://apt.cs.manchester.ac.uk/ftp/pub/apt/papers/JavN_ICS09.pdf -- "Understanding the Interconnection Network of SpiNNaker"

Hierarchical Temporal Memory[edit]

https://github.com/numenta/nupic/wiki/Hierarchical-Temporal-Memory-Theory

http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1000532 -- "Towards a Mathematical Theory of Cortical Micro-circuits"

Distributed Neural Networks[edit]

https://www.paypal-engineering.com/2015/01/12/deep-learning-on-hadoop-2-0-2/ -- paypal engineering blog, paper describes implementation of neural network as described by Hinton 2006 on Hadoop

http://www.sciencemag.org/content/suppl/2006/08/04/313.5786.504.DC1/Hinton.SOM.pdf -- supplementary material for Hinton 2006

http://www.cs.otago.ac.nz/staffpriv/hzy/papers/pdcs03.pdf -- "Parallelization of a Backpropagation Neural Network on a Cluster Computer"

http://arxiv.org/pdf/1404.5997v2.pdf -- "One weird trick for parallelizing convolutional neural networks"

http://www.ics.uci.edu/~jmoorkan/pub/gpusnn-ijcnn.pdf -- "Efficient Simulation of Large-Scale Spiking Neural Networks Using CUDA Graphics Processors"

Mixture of Experts[edit]

http://www.cs.toronto.edu/~fritz/absps/jjnh91.pdf -- "Adaptive Mixtures of Local Experts"

Hopfield nets and RBMs[edit]

http://www.iro.umontreal.ca/~bengioy/papers/ftml_book.pdf -- Yoshua Bengio, 2009, Learning Deep Architectures for AI

http://deeplearning.cs.cmu.edu/ -- Syllabus for cs course on deep learning, possible source of literature for the library

https://class.coursera.org/neuralnets-2012-001/lecture -- Lecture 11 - 15 to aid in understanding of Hinton 2006 paper above

http://www.pnas.org/content/79/8/2554.full.pdf -- "Neural networks and physical systems with emergent collective computational abilities" (Hopfield 1982)

http://page.mi.fu-berlin.de/rojas/neural/chapter/K13.pdf -- "The Hopfield Model" (Rojas 1996)

http://www.seas.upenn.edu/~wulsin/docs/wulsin2010.pdf -- "Semi-Supervised Anomaly Detection for EEG Waveforms Using Deep Belief Nets"

http://www.cse.buffalo.edu/DBGROUP/bioinformatics/papers/cameraready_xwjiabibe.pdf -- "A Novel Semi-supervised Deep Learning Framework for Affective State Recognition on EEG Signals"

http://www.cs.toronto.edu/~hinton/absps/guideTR.pdf -- "A Practical Guide to Training Restricted Boltzmann Machines"

http://arxiv.org/pdf/1503.07793v2.pdf -- "Gibbs Sampling with Low-Power Spiking Digital Neurons"

http://arxiv.org/pdf/1311.0190v1 -- "On the typical properties of inverse problems in statistical mechanics" Iacopo Mastromatteo 2013

http://jmlr.org/proceedings/papers/v5/salakhutdinov09a/salakhutdinov09a.pdf -- "Deep Boltzmann Machines" Salakhutdinov & Hinton 2009

http://www.cs.toronto.edu/~hinton/absps/tr00-004.pdf -- "Training Products of Experts by Minimizing Contrastive Divergence"

http://www.eecg.toronto.edu/~pc/research/publications/ly.fpga2009.submitted.pdf -- "A High-Performance FPGA Architecture for Restricted Boltzmann Machines" Ly & Chow 2009

https://pdfs.semanticscholar.org/85fa/f7c3c05388e2bcd097a416606bdd88fc0c7c.pdf -- "A MULTI-FPGA ARCHITECTURE FOR STOCHASTIC RESTRICTED BOLTZMANN MACHINES" Ly & Chow 2009

Variational Renormalization[edit]

https://arxiv.org/pdf/1410.3831 -- "An exact mapping between the Variational Renormalization Group and Deep Learning"

Neuromorphic Stuff[edit]

https://arxiv.org/pdf/1508.01008.pdf -- "INsight: A Neuromorphic Computing System for Evaluation of Large Neural Networks" Chung, Shin & Kang 2015

Markov Chain Monte Carlo[edit]

http://www.cs.princeton.edu/courses/archive/spr06/cos598C/papers/AndrieuFreitasDoucetJordan2003.pdf -- "An Introduction to MCMC for Machine Learning"

http://jmlr.org/proceedings/papers/v37/salimans15.pdf -- "Markov Chain Monte Carlo and Variational Inference: Bridging the Gap"

https://www.umiacs.umd.edu/~resnik/pubs/LAMP-TR-153.pdf -- "Gibbs Sampling for the Uninitiated"

Entrainment[edit]

http://www.scirp.org/Journal/PaperDownload.aspx?paperID=33251 -- "Alpha Rhythms Response to 10 Hz Flicker Is Wavelength Dependent"

http://www.brainmachine.co.uk/wp-content/uploads/Herrmann_Flicker.pdf -- "EEG responses to 1–100 Hz flicker: resonance phenomena in visual cortex and their potential correlation to cognitive phenomena"

http://www.jneurosci.org/content/23/37/11621.full.pdf -- "Human Cerebral Activation during Steady-State Visual-Evoked Responses"

http://www.dauwels.com/Papers/CogDyn%202009.pdf -- "On the synchrony of steady state visual evoked potentials and oscillatory burst events"

https://www.tu-ilmenau.de/fileadmin/public/lorentz-force/publications/peer/2012/haueisen2012/Halbleib_JCN_2012_Topographic_analysis_photic_driving.pdf -- "Topographic Analysis of Engagement and Disengagement of Neural Oscillators in Photic Driving: A Combined Electroencephalogram/Magnetoencephalogram Study"

Mining Scientific Literature[edit]

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC153503/pdf/1471-2105-4-11.pdf -- "PreBIND and Textomy – mining the biomedical literature for protein-protein interactions using a support vector machine" Donaldson 2003 BMC Bioinformatics

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4674139/pdf/pcbi.1004630.pdf -- "Text Mining for Protein Docking" Badal 2015 PLoS Comput Biol.

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4691339/pdf/bav116.pdf -- "Biocuration with insufficient resources and fixed timelines" Rodriguez-Esteban 2015 Database: The Journal of Biological Databases and Curation

(not necessarilly very) Current Discussion[edit]

re Tononi's "Integrated Information Theory" http://www.scottaaronson.com/blog/?p=1799

(19 February 2014) starting to think about possibility for experiments (loosely) related to Visual Evoked Potential research again - for instance:

http://www.biosemi.com/publications/pdf/Bierman.pdf -- Wave function collapse

File:Schak99InstantaneousCoherenceStroopTask.pdf -- "Instantaneous EEG Coherence Analysis During the Stroop Task" -- Schack et al 1999

File:CoherentEEGAmbiguousFigureBinding.pdf -- "Coherent EEG Indicators of Cognitive Binding During Ambiguous Figure Tasks" -- Klemm, Li, and Hernandez 2000

Note these two papers flog coherence measures - not trying to focus so much on that analysis right now, more interested in general understanding of what these experiments are about with possible goal of designing simpler experiments & analysis of similar perceptual/cognitive phenomena.

Here is an article that looks more directly at visual evoked potential measures:

File:ERP Stereoscopic.pdf -- "Evoked Responses to Distinct and Nebulous Stereoscopic Stimuli" -- Dunlop et al 1983


(11 September 2013) more on analysis methods:

http://slesinsky.org/brian/misc/eulers_identity.html

http://www.dspguide.com/ch8/1.htm

File:Fftw3.pdf

File:ParametricEEGAnalysis.pdf

File:ICATutorial.pdf

File:ICAFrequencyDomainEEG.pdf


(21 August 2013) - readings relating statistical (etc math / signal processing / pattern recognition / machine learning) methods for EEG data interpretation. A lot of stuff, a bit of nonsense ... and ... statistics!

Would be good to identify any papers suitable for more in-depth study. Currently have a wide field to graze for selections:

File:DWTandFFTforEEG.pdf "EEG Classifier using Fourier Transform and Wavelet Transform" -- Maan Shaker, 2007

File:Schak99InstantaneousCoherenceStroopTask.pdf -- "Instantaneous EEG Coherence Analysis During the Stroop Task" -- Schack et al 1999

File:KulaichevCoherence.pdf -- "The Informativeness of Coherence Analysis in EEG Studies" -- A. P. Kulaichev 2009 note: interesting critical perspective re limitations, discussion of alternative analytics

File:ContinuousAndDiscreteWaveletTransforms.pdf -- review of (pre-1990) wavelet literature -- Christopher Heil and David Walnut, 1989

File:EEGGammaMeditation.pdf -- "Brain sources of EEG gamma frequency during volitionally meditation-induced, altered states of consciousness, and experience of the self" -- Dietrich Lehman et al 2001

http://neuro.hut.fi/~pavan/home/Hyvarinen2010_FourierICA_Neuroimage.pdf - "Independent component analysis of short-time Fourier transforms for spontaneous EEG/MEG analysis" -- Aapo Hyvarinen, Pavan Ramkumar, Lauri Parkkonen, Riitta Hari - paper published in Neuroimage vol 49 (2010)


OpenSource Machine Learning Algs from NG @MIT
Consumer grade EEG used to see "P300" reponse and for thoes with a short attention span tldr
(discussed at meetup Wednesday 31 July 2013)
"Coherent EEG Indicators of Cognitive Binding During Ambiguous Figure Tasks" Klemm, Li, and Hernandez 2000
File:CoherentEEGAmbiguousFigureBinding.pdf
"We tested the hypothesis that perception of an alternative image in ambiguous figures would be manifest as high-frequency (gamma) components that become synchronized over multiple scalp sites as a "cognitive binding" process occurs."


art, dream, and eeg


mind v brain, hobson v solms
http://en.wikipedia.org/wiki/Activation-synthesis_hypothesis
File:HobsomREMDreamProtoconsciousness.pdf

"Hobson and McCarley originally proposed in the 1970s that the differences in the waking-NREM-REM sleep cycle was the result of interactions between aminergic REM-off cells and cholinergic REM-on cells.[4] This was perceived as the activation-synthesis model, stating that brain activation during REM sleep results in synthesis of dream creation.[1][1] Hobson's five cardinal characteristics include: intense emotions, illogical content, apparent sensory impressions, uncritical acceptance of dream events, and difficulty in being remembered."


Berkeley Labs

Gallant Group
Walker Group
Palmer Group

Sleep Research[edit]

Comment on the AASM Manual for the Scoring of Sleep and Associated Events

random tangents[edit]

(following previous discussion) - we might select a few to study in more depth (... or not! Plenty more to explore - suggestions (random or otherwise) are welcome. http://www.meltingasphalt.com/neurons-gone-wild/ -- Neurons Gone Wild - Levels of agency in the brain.


stereoscopic perception:


some (maybe) interesting background on Information Theory (cool title...)

Claude Shannon: "Communication in the Presence of Noise"
File:Shannon noise.pdf
"We will call a system that transmits without errors at the rate C an ideal system.
 Such a system cannot be achieved with any finite encoding process
 but can be approximated as closely as desired."

wikipedia etc quick reads:

https://en.wikipedia.org/wiki/Eeg
https://en.wikipedia.org/wiki/Neural_synchronization
https://en.wikipedia.org/wiki/Event-related_potentials
http://www.scholarpedia.org/article/Spike-and-wave_oscillations
http://www.scholarpedia.org/article/Thalamocortical_oscillations

Previously[edit]

Masahiro's EEG Device/IBVA Software

and ... open source hardware design and kits on instructables.com

Puzzlebox - Opensource BCI Developers

Morgan from GazzLab @ MissionBay/UCSF


https://github.com/gazzlab

Let's ease into a lightweight "journal club" discussion with this technical report from NeuroSky.

Name: A user-friendly SSVEP-based brain-computer interface using a time-domain classifier, Luo A and Sullivan TJ 2010

URL: File:NeuroSkyVEP.pdf

Please add your comments & questions here.

Background Reading[edit]

http://nanosouffle.net/ (view into Arxiv.org)

Name: Hunting for Meaning after Midnight, Miller 2007

URL: <http://www.noisebridge.net/pipermail/neuro/attachments/20130501/4b992eb6/attachment-0002.pdf>

Name: Broken mirrors, Ram, VS, & Oberman, LM, 2006, Nov

URL: <http://www.noisebridge.net/pipermail/neuro/attachments/20130501/4b992eb6/attachment-0003.pdf>

Ramachandran Critique

http://blogs.scientificamerican.com/guest-blog/2012/11/06/whats-so-special-about-mirror-neurons/

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2773693/

Sleep/Dream Studies

http://www.cns.atr.jp/dni/en/publications/

NeuroSky Docs[edit]

File:NeuroSkyDongleProtocol.pdf

File:NeuroSkyCommunicationsProtocol.pdf

Android Neutral Network Fuzzy Learning app[edit]

Android Neutral Network Fuzzy Learning app in Play Store

Learning about Neural Networks[edit]