Difference between revisions of "DreamTeam/Reading"

From Noisebridge
Jump to: navigation, search
(PWM / low pass filter)
Line 1: Line 1:
(note wiki contains some useful clues re previous neuro research at Noisebridge ... For example, the [[Analog_EEG_Amp]] page describes some project ideas and work done by others here in 2012)  
(note wiki contains some useful clues re previous neuro research at Noisebridge ... For example, the [[Analog_EEG_Amp]] page describes some project ideas and work done by others here in 2012)  

Revision as of 03:01, 4 August 2016

File:Orc.jpg (note wiki contains some useful clues re previous neuro research at Noisebridge ... For example, the Analog_EEG_Amp page describes some project ideas and work done by others here in 2012)

https://metacademy.org/ -- machine learning knowledge graph

http://www.thetalkingmachines.com/ -- podcast


http://www.buzsakilab.com/content/PDFs/BuzsakiKoch2012.pdf -- "The origin of extracellular fields and currents — EEG, ECoG, LFP and spikes"

Signal Processing

http://provideyourown.com/2011/analogwrite-convert-pwm-to-voltage/ -- "Arduino’s AnalogWrite – Converting PWM to a Voltage"

http://sim.okawa-denshi.jp/en/PWMtool.php -- "RC Low-pass Filter Design for PWM (Transient Analysis Calculator)"

Hyperdimensional Computing

http://redwood.berkeley.edu/pkanerva/papers/kanerva09-hyperdimensional.pdf -- "Hyperdimensional Computing: An Introduction to Computing in Distributed Representation with High-Dimensional Random Vectors"

http://arxiv.org/pdf/1602.03032.pdf -- "Associative Long Short-Term Memory"

Bird Flocks and Maximum Entropy

http://arxiv.org/pdf/1107.0604v1 -- "Statistical Mechanics and Flocks of Birds"

http://arxiv.org/pdf/1307.5563v1 -- "Social interactions dominate speed control in driving natural flocks toward criticality"

http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-050j-information-and-entropy-spring-2008/syllabus/MIT6_050JS08_penfield.pdf -- "Information and Entropy (Course Notes)"

Whale Songs

https://www.researchgate.net/profile/Herbert_Roitblat/publication/13429327_The_neural_network_classification_of_false_killer_whale_%28Pseudorca_crassidens%29_vocalizations/links/540d2ff60cf2df04e75478cd.pdf -- "The neural network classification of false killer whale (Pseudorca crassidens) vocalizations"


https://www.nersc.no/sites/www.nersc.no/files/master_thesis_sebastian_menze.pdf -- "Estimating fin whale distribution from ambient noise spectra using Bayesian inversion"

http://sis.univ-tln.fr/~glotin/IJCNN2015_IHMMbioac_BartChamGlot.pdf -- "Hierarchical Dirichlet Process Hidden Markov Model for Unsupervised Bioacoustic Analysis"

https://www.inf.ed.ac.uk/publications/thesis/online/IM030057.pdf -- "Hidden Markov Model Clustering of Acoustic Data"

Computational Cognitive Neuroscience

http://www.pnas.org/content/110/41/16390.full -- "Indirection and symbol-like processing in the prefrontal cortex and basal ganglia"

http://ruccs.rutgers.edu/images/personal-zenon-pylyshyn/docs/jaf.pdf -- "Connectionism and Cognitive Architecture: A Critical Analysis"

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2728678/pdf/nihms131814.pdf -- Neves et al 2008 "Cell Shape and Negative Links in Regulatory Motifs Together Control Spatial Information Flow in Signaling Networks"

http://psych.colorado.edu/~oreilly/papers/AisaMingusOReilly08.pdf -- "The Emergent Neural Modeling System"


Text Generation

http://www.cs.toronto.edu/~ilya/pubs/2011/LANG-RNN.pdf -- "Generating Text with Recurrent Neural Networks"


http://setgame.com/sites/default/files/teacherscorner/COGNITIVE%20MODELING%20WITH%20SET.pdf -- "How to Construct a Believable Opponent using Cognitive Modeling in the Game of Set"

http://www-personal.umich.edu/~charchan/SET.pdf -- "SETs and Anti-SETs: The Math Behind the Game of SET"

http://personal.plattsburgh.edu/quenelgt/talks/set.pdf -- "Introduction to Set"

http://web.engr.illinois.edu/~pbg/papers/set.pdf -- "On the Complexity of the Game of Set"

http://www.warwick.ac.uk/staff/D.Maclagan/papers/set.pdf -- "The Card Game Set"

http://www.math.ucdavis.edu/~anne/FQ2011/set_game.pdf -- "The Game Set"

Large Scale Brain Simulation

http://www.nowere.net/b/arch/96550/src/1378907656268.pdf -- "A world survey of artificial brain projects, Part I: Large-scale brain simulations"


http://cmr.soc.plymouth.ac.uk/publications/bci-wkshop.pdf -- "ON GENERATING EEG FOR CONTROLLING MUSICAL SYSTEMS"


https://github.com/nbdt/gotrain (our ANN code)

https://github.com/nbdt/openbci-golang-server (EEG data acquisition and visualization software for OpenBCI)

Hidden Markov Models

http://www.cs.sjsu.edu/~stamp/RUA/HMM.pdf -- "A Revealing Introduction to Hidden Markov Models"

http://www.jelmerborst.nl/pubs/Borst2013b.pdf -- "Discovering Processing Stages by combining EEG with Hidden Markov Models"

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2230664/pdf/nihms-34528.pdf -- "A Stochastic Framework for Evaluating Seizure Prediction Algorithms Using Hidden Markov Models"

http://www.schalklab.org/sites/default/files/misc/Coupled%20hidden%20markov%20model%20for%20electrocorticographic%20signal%20classification.pdf -- "Coupled Hidden Markov Model for Electrocorticographic Signal Classification"

Long Short Term Memory

http://deeplearning.cs.cmu.edu/pdfs/Hochreiter97_lstm.pdf -- "Long Short-Term Memory"

ftp://ftp.idsia.ch/pub/juergen/2002_icannMusic.pdf -- "Learning The Long-Term Structure of the Blues"

http://www.overcomplete.net/papers/nn2012.pdf -- "A generalized LSTM-like training algorithm for second-order recurrent neural networks"

http://www.iro.umontreal.ca/~eckdoug/papers/2003_nn.pdf -- "Kalman filters improve LSTM network performance in problems unsolvable by traditional recurrent nets"

http://googleresearch.blogspot.ch/2015/09/google-voice-search-faster-and-more.html -- "Long Short-Term Memory dramatically improves Google Voice etc"

Question Answering

http://www.overcomplete.net/papers/bica2012.pdf -- "Neural Architectures for Learning to Answer Questions"

https://cs.umd.edu/~miyyer/pubs/2014_qb_rnn.pdf -- "A Neural Network for Factoid Question Answering over Paragraphs"

http://arxiv.org/pdf/1502.05698.pdf -- "Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks"

http://www.peterschueller.com/pub/2014/2014_winograd_schemas_relevance_knowledge_graphs.pdf -- "Tackling Winograd Schemas by Formalizing Relevance Theory in Knowledge Graphs"

http://ijcai.org/papers15/Papers/IJCAI15-190.pdf -- "Towards Addressing the Winograd Schema Challenge — Building and Using a Semantic Parser and a Knowledge Hunting Module"

http://arxiv.org/pdf/1506.05869v2.pdf -- "A Neural Conversational Model"

http://arxiv.org/pdf/1508.05508v1.pdf -- "Towards Neural Network-based Reasoning"

http://www.visualqa.org/vqa_iccv2015.pdf -- "VQA: Visual Question Answering"


Cells must support three operations:

  • add some content
  • collect the content currently accumulated
  • register a propagator to be notified when the accumulated content changes
  • When new content is added to a cell, the cell must merge the addition with the content already present. When a propagator asks for the content of a cell, the cell must deliver a complete summary of the information that has been added to it.
  • The merging of content must be commutative, associative, and idempotent. The behavior of propagators must be monotonic with respect to the lattice induced by the merge operation.
  • http://web.mit.edu/~axch/www/art.pdf *http://groups.csail.mit.edu/mac/users/gjs/propagators/
  • http://dustycloud.org/blog/sussman-on-ai/


http://www.cs.princeton.edu/courses/archive/spr08/cos424/readings/Schapire2003.pdf -- "The Boosting Approach to Machine Learning An Overview"

http://citeseerx.ist.psu.edu/viewdoc/download?doi= -- "Ensembling Neural Networks: Many Could Be Better Than All"

http://www.cs.columbia.edu/~rocco/Public/mlj9.pdf -- "Random Classification Noise Defeats All Convex Potential Boosters"

Support Vector Machines

http://research.microsoft.com/en-us/um/people/cburges/papers/svmtutorial.pdf -- "A Tutorial on Support Vector Machines for Pattern Recognition"

Wire Length / Small World Networks

http://koulakovlab.cshl.edu/publications/koulakov_chklovskii_2000.pdf -- "A wire length minimization approach to ocular dominance patterns in mammalian visual cortex"

http://papers.nips.cc/paper/1910-foundations-for-a-circuit-complexity-theory-of-sensory-processing.pdf -- "Foundations for a Circuit Complexity Theory of Sensory Processing"

https://www.nada.kth.se/~cjo/documents/small_world.pdf -- "Small-World Connectivity and Attractor Neural Networks"

http://www.doc.ic.ac.uk/~mpsha/ShanahanPhysRevEPreprint.pdf -- "The Dynamical Complexity of Small-World Networks of Spiking Neurons"

http://www.dam.brown.edu/people/elie/papers/small_world.pdf -- "Autoassociative Memory Retrieval and Spontaneous Activity Bumps in Small-World Networks of Integrate-and-Fire Neurons"

http://citeseerx.ist.psu.edu/viewdoc/download?doi= -- "Transition from Random to Small-World Neural Networks by STDP Learning Rule"

http://tamar.tau.ac.il/~eshel/papers/J_Neural_Engineering.pdf -- "Compact self-wiring in cultured neural networks"


http://page.mi.fu-berlin.de/rojas/neural/neuron.pdf -- "Neural Networks - A Systematic Introduction"

http://www4.rgu.ac.uk/files/chapter3%20-%20bp.pdf (works through simple example)



also see http://page.mi.fu-berlin.de/rojas/neural/neuron.pdf (entire book "Neural Networks - a Systemic Introduction" by Raul Rojas)

http://work.caltech.edu/lectures.html Hoeffding's inequality, VC Dimension and Back Propagation ANN

http://www.cs.stir.ac.uk/research/publications/techreps/pdf/TR148.pdf ("Learning XOR: exploring the space of a classic problem")

http://www.sysc.pdx.edu/classes/Werbos-Backpropagation%20through%20time.pdf -- "Backpropagation Through Time: What it Does and How to Do It"

Convolutional Neural Networks

http://scs.ryerson.ca/~aharley/vis/harley_vis_isvc15.pdf -- "An Interactive Node-Link Visualization of Convolutional Neural Networks"


http://www.cv-foundation.org/openaccess/content_cvpr_2015/papers/Dosovitskiy_Learning_to_Generate_2015_CVPR_paper.pdf -- "Learning to Generate Chairs With Convolutional Neural Networks"

http://www.pamitc.org/cvpr15/files/lecun-20150610-cvpr-keynote.pdf -- "What's Wrong With Deep Learning?"

Computer Vision

http://www.cv-foundation.org/openaccess/CVPR2015.py -- some interesting papers here

http://www.cv-foundation.org/openaccess/content_cvpr_2015/papers/Nguyen_Deep_Neural_Networks_2015_CVPR_paper.pdf -- "Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images"

Visual Perception (Biological Systems)

http://cbcl.mit.edu/publications/ps/Serre_etal_PBR07.pdf -- "A quantitative theory of immediate visual recognition"

http://www.dam.brown.edu/ptg/REPORTS/Invariance.pdf -- "Invariance and Selectivity in the Ventral Visual Pathway"

http://www.cnbc.cmu.edu/~tai/papers/lee_mumford_josa.pdf -- "Hierarchical Bayesian inference in the visual cortex"

Neural Synchrony

http://arxiv.org/pdf/1312.6115.pdf -- "Neuronal Synchrony in Complex-Valued Deep Networks"

Spiking Neural Networks

http://ncs.ethz.ch/projects/evospike/publications/ICONIP2011%20Springer%20LNCS%20Nutta.pdf -- "EEG Classification with BSA Spike Encoding Algorithm and Evolving Probabilistic Spiking Neural Network"

http://dai.fmph.uniba.sk/~benus/confers/CNGM_ICANN05.pdf -- "Computational Neurogenetic Modeling: Integration of Spiking Neural Networks, Gene Networks, and Signal Processing Techniques"

http://citeseerx.ist.psu.edu/viewdoc/summary?doi= -- "Pattern Recognition in a Bucket"

http://www.igi.tugraz.at/maass/psfiles/221.pdf -- "Noise as a Resource for Computation and Learning in Spiking Neural Networks"

http://web.stanford.edu/group/brainsinsilicon/goals.html -- Neurogrid

http://apt.cs.manchester.ac.uk/projects/SpiNNaker/ -- SpiNNaker

http://personalpages.manchester.ac.uk/staff/javier.navaridas/pubs/2012.jpdc.pdf -- more on SpiNNaker

Hierarchical Temporal Memory


http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1000532 -- "Towards a Mathematical Theory of Cortical Micro-circuits"

Distributed Neural Networks

https://www.paypal-engineering.com/2015/01/12/deep-learning-on-hadoop-2-0-2/ -- paypal engineering blog, paper describes implementation of neural network as described by Hinton 2006 on Hadoop

http://www.sciencemag.org/content/suppl/2006/08/04/313.5786.504.DC1/Hinton.SOM.pdf -- supplementary material for Hinton 2006

http://www.cs.otago.ac.nz/staffpriv/hzy/papers/pdcs03.pdf -- "Parallelization of a Backpropagation Neural Network on a Cluster Computer"

http://arxiv.org/pdf/1404.5997v2.pdf -- "One weird trick for parallelizing convolutional neural networks"

http://www.ics.uci.edu/~jmoorkan/pub/gpusnn-ijcnn.pdf -- "Efficient Simulation of Large-Scale Spiking Neural Networks Using CUDA Graphics Processors"

Hopfield nets and RBMs

http://www.iro.umontreal.ca/~bengioy/papers/ftml_book.pdf -- Yoshua Bengio, 2009, Learning Deep Architectures for AI

http://deeplearning.cs.cmu.edu/ -- Syllabus for cs course on deep learning, possible source of literature for the library

https://class.coursera.org/neuralnets-2012-001/lecture -- Lecture 11 - 15 to aid in understanding of Hinton 2006 paper above

http://www.pnas.org/content/79/8/2554.full.pdf -- "Neural networks and physical systems with emergent collective computational abilities" (Hopfield 1982)

http://page.mi.fu-berlin.de/rojas/neural/chapter/K13.pdf -- "The Hopfield Model" (Rojas 1996)

http://www.seas.upenn.edu/~wulsin/docs/wulsin2010.pdf -- "Semi-Supervised Anomaly Detection for EEG Waveforms Using Deep Belief Nets"

http://www.cse.buffalo.edu/DBGROUP/bioinformatics/papers/cameraready_xwjiabibe.pdf -- "A Novel Semi-supervised Deep Learning Framework for Affective State Recognition on EEG Signals"

http://www.cs.toronto.edu/~hinton/absps/guideTR.pdf -- "A Practical Guide to Training Restricted Boltzmann Machines"

http://arxiv.org/pdf/1503.07793v2.pdf -- "Gibbs Sampling with Low-Power Spiking Digital Neurons"

http://arxiv.org/pdf/1311.0190v1 -- "On the typical properties of inverse problems in statistical mechanics" Iacopo Mastromatteo 2013

Markov Chain Monte Carlo

http://www.cs.princeton.edu/courses/archive/spr06/cos598C/papers/AndrieuFreitasDoucetJordan2003.pdf -- "An Introduction to MCMC for Machine Learning"

http://jmlr.org/proceedings/papers/v37/salimans15.pdf -- "Markov Chain Monte Carlo and Variational Inference: Bridging the Gap"

https://www.umiacs.umd.edu/~resnik/pubs/LAMP-TR-153.pdf -- "Gibbs Sampling for the Uninitiated"


http://www.scirp.org/Journal/PaperDownload.aspx?paperID=33251 -- "Alpha Rhythms Response to 10 Hz Flicker Is Wavelength Dependent"

http://www.brainmachine.co.uk/wp-content/uploads/Herrmann_Flicker.pdf -- "EEG responses to 1–100 Hz flicker: resonance phenomena in visual cortex and their potential correlation to cognitive phenomena"

http://www.jneurosci.org/content/23/37/11621.full.pdf -- "Human Cerebral Activation during Steady-State Visual-Evoked Responses"

http://www.dauwels.com/Papers/CogDyn%202009.pdf -- "On the synchrony of steady state visual evoked potentials and oscillatory burst events"

https://www.tu-ilmenau.de/fileadmin/public/lorentz-force/publications/peer/2012/haueisen2012/Halbleib_JCN_2012_Topographic_analysis_photic_driving.pdf -- "Topographic Analysis of Engagement and Disengagement of Neural Oscillators in Photic Driving: A Combined Electroencephalogram/Magnetoencephalogram Study"

Mining Scientific Literature

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC153503/pdf/1471-2105-4-11.pdf -- "PreBIND and Textomy – mining the biomedical literature for protein-protein interactions using a support vector machine" Donaldson 2003 BMC Bioinformatics

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4674139/pdf/pcbi.1004630.pdf -- "Text Mining for Protein Docking" Badal 2015 PLoS Comput Biol.

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4691339/pdf/bav116.pdf -- "Biocuration with insufficient resources and fixed timelines" Rodriguez-Esteban 2015 Database: The Journal of Biological Databases and Curation

(not necessarilly very) Current Discussion

re Tononi's "Integrated Information Theory" http://www.scottaaronson.com/blog/?p=1799

(19 February 2014) starting to think about possibility for experiments (loosely) related to Visual Evoked Potential research again - for instance:

http://www.biosemi.com/publications/pdf/Bierman.pdf -- Wave function collapse

File:Schak99InstantaneousCoherenceStroopTask.pdf -- "Instantaneous EEG Coherence Analysis During the Stroop Task" -- Schack et al 1999

File:CoherentEEGAmbiguousFigureBinding.pdf -- "Coherent EEG Indicators of Cognitive Binding During Ambiguous Figure Tasks" -- Klemm, Li, and Hernandez 2000

Note these two papers flog coherence measures - not trying to focus so much on that analysis right now, more interested in general understanding of what these experiments are about with possible goal of designing simpler experiments & analysis of similar perceptual/cognitive phenomena.

Here is an article that looks more directly at visual evoked potential measures:

File:ERP Stereoscopic.pdf -- "Evoked Responses to Distinct and Nebulous Stereoscopic Stimuli" -- Dunlop et al 1983

(11 September 2013) more on analysis methods:







(21 August 2013) - readings relating statistical (etc math / signal processing / pattern recognition / machine learning) methods for EEG data interpretation. A lot of stuff, a bit of nonsense ... and ... statistics!

Would be good to identify any papers suitable for more in-depth study. Currently have a wide field to graze for selections:

File:DWTandFFTforEEG.pdf "EEG Classifier using Fourier Transform and Wavelet Transform" -- Maan Shaker, 2007

File:Schak99InstantaneousCoherenceStroopTask.pdf -- "Instantaneous EEG Coherence Analysis During the Stroop Task" -- Schack et al 1999

File:KulaichevCoherence.pdf -- "The Informativeness of Coherence Analysis in EEG Studies" -- A. P. Kulaichev 2009 note: interesting critical perspective re limitations, discussion of alternative analytics

File:ContinuousAndDiscreteWaveletTransforms.pdf -- review of (pre-1990) wavelet literature -- Christopher Heil and David Walnut, 1989

File:EEGGammaMeditation.pdf -- "Brain sources of EEG gamma frequency during volitionally meditation-induced, altered states of consciousness, and experience of the self" -- Dietrich Lehman et al 2001

http://neuro.hut.fi/~pavan/home/Hyvarinen2010_FourierICA_Neuroimage.pdf - "Independent component analysis of short-time Fourier transforms for spontaneous EEG/MEG analysis" -- Aapo Hyvarinen, Pavan Ramkumar, Lauri Parkkonen, Riitta Hari - paper published in Neuroimage vol 49 (2010)

OpenSource Machine Learning Algs from NG @MIT
Consumer grade EEG used to see "P300" reponse and for thoes with a short attention span tldr
(discussed at meetup Wednesday 31 July 2013)
"Coherent EEG Indicators of Cognitive Binding During Ambiguous Figure Tasks" Klemm, Li, and Hernandez 2000
"We tested the hypothesis that perception of an alternative image in ambiguous figures would be manifest as high-frequency (gamma) components that become synchronized over multiple scalp sites as a "cognitive binding" process occurs."

art, dream, and eeg

mind v brain, hobson v solms

"Hobson and McCarley originally proposed in the 1970s that the differences in the waking-NREM-REM sleep cycle was the result of interactions between aminergic REM-off cells and cholinergic REM-on cells.[4] This was perceived as the activation-synthesis model, stating that brain activation during REM sleep results in synthesis of dream creation.[1][1] Hobson's five cardinal characteristics include: intense emotions, illogical content, apparent sensory impressions, uncritical acceptance of dream events, and difficulty in being remembered."

Berkeley Labs

Gallant Group
Walker Group
Palmer Group

Sleep Research

Comment on the AASM Manual for the Scoring of Sleep and Associated Events

random tangents

(following previous discussion) - we might select a few to study in more depth (... or not! Plenty more to explore - suggestions (random or otherwise) are welcome.

stereoscopic perception:

some (maybe) interesting background on Information Theory (cool title...)

Claude Shannon: "Communication in the Presence of Noise"
File:Shannon noise.pdf
"We will call a system that transmits without errors at the rate C an ideal system.
 Such a system cannot be achieved with any finite encoding process
 but can be approximated as closely as desired."

wikipedia etc quick reads:



Masahiro's EEG Device/IBVA Software

and ... open source hardware design and kits on instructables.com

Puzzlebox - Opensource BCI Developers

Morgan from GazzLab @ MissionBay/UCSF


Let's ease into a lightweight "journal club" discussion with this technical report from NeuroSky.

Name: A user-friendly SSVEP-based brain-computer interface using a time-domain classifier, Luo A and Sullivan TJ 2010

URL: File:NeuroSkyVEP.pdf

Please add your comments & questions here.

Background Reading

http://nanosouffle.net/ (view into Arxiv.org)

Name: Hunting for Meaning after Midnight, Miller 2007

URL: <http://www.noisebridge.net/pipermail/neuro/attachments/20130501/4b992eb6/attachment-0002.pdf>

Name: Broken mirrors, Ram, VS, & Oberman, LM, 2006, Nov

URL: <http://www.noisebridge.net/pipermail/neuro/attachments/20130501/4b992eb6/attachment-0003.pdf>

Ramachandran Critique



Sleep/Dream Studies


NeuroSky Docs