# Difference between revisions of "Machine Learning"

(→Datasets and Websites) |
(→Classes) |
||

(155 intermediate revisions by 26 users not shown) | |||

Line 1: | Line 1: | ||

− | === | + | === Join the Mailing List === |

− | + | ||

+ | https://www.noisebridge.net/mailman/listinfo/ml | ||

=== Next Meeting=== | === Next Meeting=== | ||

− | *When: | + | *When: Thursday, August 14, 2014 @ 6:00pm |

− | *Where: 2169 Mission St. ( | + | *Where: 2169 Mission St. (Church classroom) |

− | *Topic: | + | *Topic: |

− | * | + | *Details: |

+ | *Who: Andy McMurry | ||

+ | |||

+ | === Learn about Data Science and Machine Learning === | ||

+ | |||

+ | ===== Classes ===== | ||

+ | *[https://www.coursera.org/course/ml Coursera Machine Learning Course with Andrew Ng] | ||

+ | *[https://www.coursera.org/course/compneuro Coursera Computational Neuroscience Course with Rajesh P N Rao and Adrienne Fairhall] | ||

+ | *[http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-867-machine-learning-fall-2006/ MIT Machine Learning Class with Tommi Jaakkola] | ||

+ | *[http://cs229.stanford.edu/materials.html Stanford CS229] | ||

+ | *[http://www.cs.cmu.edu/~tom/10701_sp11/lectures.shtml Carnegie Mellon Machine Learning Course with Tom Mitchell] | ||

+ | *[http://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-spring-2010/ Linear Algebra with Gilbert Strang] | ||

+ | *[https://www.youtube.com/playlist?list=PL6Xpj9I5qXYEcOhn7TqghAJ6NAPrNmUBH Neural Networks Class with Hugo Larochelle] | ||

− | === | + | ==== Books ==== |

− | + | *[http://statweb.stanford.edu/~tibs/ElemStatLearn/ Elements of Statistical Learning] | |

+ | *[https://www.google.com/search?client=ubuntu&channel=fs&q=pattern+recognition+and+machine+learning&ie=utf-8&oe=utf-8#channel=fs&q=pattern+recognition+and+machine+learning+pdf Pattern Recognition and Machine Learning] | ||

+ | *[https://www.google.com/search?&channel=fs&q=+Information+Theory%2C+Inference%2C+and+Learning+Algorithms.&ie=utf-8&oe=utf-8#channel=fs&q=Information+Theory%2C+Inference%2C+and+Learning+Algorithms+pdf Information Theory, Inference, and Learning Algorithms] | ||

+ | *[http://chimera.labs.oreilly.com/books/1230000000345 Interactive Data Visualization for the Web (D3)] | ||

+ | *[http://cran.r-project.org/doc/manuals/R-intro.pdf Introduction to R] | ||

+ | *[http://www.dartmouth.edu/~chance/teaching_aids/books_articles/probability_book/amsbook.mac.pdf Introduction to Probability (Grinstead and Snell)] | ||

+ | *[http://www.cis.temple.edu/~latecki/Courses/CIS2033-Spring12/A_modern_intro_probability_statistics_Dekking05.pdf Modern Introduction to Probability and Statistics (Kraaikamp and Meester)] | ||

+ | *[http://web4.cs.ucl.ac.uk/staff/D.Barber/textbook/090310.pdf Bayesian Reasoning and Machine Learning] | ||

+ | |||

+ | ==== Tutorials ==== | ||

+ | *[http://nbviewer.ipython.org/github/unpingco/Python-for-Signal-Processing/tree/master/ Signal Processing IPython Notebooks] | ||

+ | *[http://scikit-learn.org/stable/tutorial/basic/tutorial.html Introduction to ML with scikits.learn] | ||

+ | *[http://www.sagemath.org/doc/tutorial/ Learn how to use SAGE] | ||

+ | |||

+ | ==== Noisebridge ML Class Slides ==== | ||

*[[NBML/Workshops/Intro to Machine Learning|Intro to Machine Learning]] | *[[NBML/Workshops/Intro to Machine Learning|Intro to Machine Learning]] | ||

*[[NBML/Workshops/Brief Tour of Statistics|A Brief Tour of Statistics]] | *[[NBML/Workshops/Brief Tour of Statistics|A Brief Tour of Statistics]] | ||

Line 27: | Line 54: | ||

=== Future Talks and Topics, Ideas === | === Future Talks and Topics, Ideas === | ||

− | *Restricted Boltzmann Machines (Mike S, | + | *Random Forests in R |

− | *Deep Nets w/ Stacked Autoencoders (Mike S, | + | *Restricted Boltzmann Machines (Mike S, some day) |

− | *Generalized Linear Models (Mike S, | + | *Analyzing brain cells (Mike S) |

− | *Graphical Models | + | *Deep Nets w/ Stacked Autoencoders (Mike S, some day) |

+ | *Generalized Linear Models (Mike S, Erin L? some day) | ||

+ | *Graphical Models | ||

*Working with the Kinect | *Working with the Kinect | ||

*Computer Vision with OpenCV | *Computer Vision with OpenCV | ||

− | |||

− | |||

− | |||

− | |||

=== Projects === | === Projects === | ||

Line 58: | Line 83: | ||

*[http://mlcomp.org/ MLcomp] | *[http://mlcomp.org/ MLcomp] | ||

**Upload your algorithm and objectively compare it's performance to other algorithms | **Upload your algorithm and objectively compare it's performance to other algorithms | ||

+ | *[http://www.ntis.gov/products/ssa-dmf.aspx Social Security Death Master File!] | ||

+ | *[http://www.sipri.org/databases SIPRI Social Databases] | ||

+ | **Wealth of information on international arms transfers and peace missions. | ||

+ | *[http://aws.amazon.com/publicdatasets/ Amazon AWS Public Datasets] | ||

+ | *[http://www.prio.no/Data/Armed-Conflict/ UCDP/PRIO Armed Conflict Datasets] | ||

+ | *[https://opendata.socrata.com/browse Socrata Government Datasets] | ||

+ | *[http://us-city.census.okfn.org/ US City Census Data] | ||

+ | *[http://webscope.sandbox.yahoo.com/catalog.php Yahoo Labs Datasets] | ||

− | === | + | === Software Tools === |

− | + | ||

− | + | ==== Generic ML Libraries ==== | |

− | |||

− | |||

− | |||

− | |||

*[http://www.cs.waikato.ac.nz/ml/weka/ Weka] | *[http://www.cs.waikato.ac.nz/ml/weka/ Weka] | ||

**a collection of data mining tools and machine learning algorithms. | **a collection of data mining tools and machine learning algorithms. | ||

− | |||

− | |||

*[http://scikit-learn.sourceforge.net/ scikits.learn] | *[http://scikit-learn.sourceforge.net/ scikits.learn] | ||

**Machine learning Python package | **Machine learning Python package | ||

+ | *[http://pypi.python.org/pypi/scikits.statsmodels scikits.statsmodels] | ||

+ | **Statistical models to go with scipy | ||

*[http://pybrain.org PyBrain] | *[http://pybrain.org PyBrain] | ||

**Does feedforward, recurrent, SOM, deep belief nets. | **Does feedforward, recurrent, SOM, deep belief nets. | ||

Line 78: | Line 107: | ||

*[http://pyml.sourceforge.net PyML] | *[http://pyml.sourceforge.net PyML] | ||

*[http://mdp-toolkit.sourceforge.net/ MDP] | *[http://mdp-toolkit.sourceforge.net/ MDP] | ||

− | ** | + | **Modular framework, has lots of stuff! |

− | |||

− | |||

− | |||

*[[Machine Learning/VirtualBox|VirtualBox]] Virtual Box Image with Pre-installed Libraries listed here | *[[Machine Learning/VirtualBox|VirtualBox]] Virtual Box Image with Pre-installed Libraries listed here | ||

− | *[http:// | + | *[http://sympy.org sympy] Does symbolic math |

*[http://waffles.sourceforge.net/ Waffles] | *[http://waffles.sourceforge.net/ Waffles] | ||

**Open source C++ set of machine learning command line tools. | **Open source C++ set of machine learning command line tools. | ||

*[http://rapid-i.com/content/view/181/196/ RapidMiner] | *[http://rapid-i.com/content/view/181/196/ RapidMiner] | ||

*[http://www.mrpt.org/ Mobile Robotic Programming Toolkit] | *[http://www.mrpt.org/ Mobile Robotic Programming Toolkit] | ||

+ | *[http://nipy.sourceforge.net/nitime/ nitime] | ||

+ | **NeuroImaging in Python, has some good time series analysis stuff and multi-variate response fitting. | ||

+ | *[http://pandas.pydata.org/ Pandas] | ||

+ | **Data analysis workflow in python | ||

+ | *[http://www.pytables.org/moin PyTables] | ||

+ | **Adds querying capabilities to HDF5 files | ||

+ | *[http://statsmodels.sourceforge.net/ statsmodels] | ||

+ | **Regression, time series analysis, statistics stuff for python | ||

+ | *[https://github.com/JohnLangford/vowpal_wabbit/wiki Vowpal Wabbit] | ||

+ | **"Intrinsically Fast" implementation of gradient descent for large datasets | ||

+ | *[http://www.shogun-toolbox.org/ Shogun] | ||

+ | **Fast implementations of SVMs | ||

+ | *[http://www.mlpack.org/ MLPACK] | ||

+ | **High performance scalable ML Library | ||

+ | *[http://www.torch.ch/ Torch] | ||

+ | **MATLAB-like environment for state-of-the art ML libraries written in LUA | ||

+ | |||

+ | ==== Deep Nets ==== | ||

+ | *[http://deeplearning.net/software/theano/ Theano] | ||

+ | **Symbolic Expressions and Transparent GPU Integration | ||

+ | *[http://caffe.berkeleyvision.org/ Caffe] | ||

+ | **Convolutional Neural Networks on GPU | ||

+ | *[https://code.google.com/p/neurolab/ Neurolab] | ||

+ | **Has support for recurrent neural nets | ||

+ | |||

+ | ==== Online ML ==== | ||

+ | *[http://moa.cs.waikato.ac.nz/ MOA (Massive Online Analysis)] | ||

+ | **Offshoot of weka, has all online-algorithms | ||

+ | *[http://jubat.us/en/ Jubatus] | ||

+ | **Distributed Online ML | ||

+ | *[http://dogma.sourceforge.net/ DOGMA] | ||

+ | **MATLAB-based online learning stuff | ||

+ | *[http://code.google.com/p/libol/ libol] | ||

+ | *[http://code.google.com/p/oll/ oll] | ||

+ | *[http://code.google.com/p/scw-learning/ scw-learning] | ||

+ | |||

+ | ==== Graphical Models ==== | ||

+ | *[http://www.mrc-bsu.cam.ac.uk/bugs/ BUGS] | ||

+ | **MCMC for Bayesian Models | ||

+ | *[http://mcmc-jags.sourceforge.net/ JAGS] | ||

+ | **Hierarchical Bayesian Models | ||

+ | *[http://mc-stan.org/ Stan] | ||

+ | **A graphical model compiler | ||

+ | *[https://github.com/kutschkem/Jayes Jayes] | ||

+ | **Bayesian networks in Java | ||

+ | *[http://tops.sourceforge.net/ ToPS] | ||

+ | **Probabilistic models of sequences | ||

+ | *[http://pymc-devs.github.io/pymc/ PyMC] | ||

+ | **Bayesian Models in Python | ||

+ | |||

+ | ==== Text Stuff ==== | ||

+ | *[http://www.crummy.com/software/BeautifulSoup/ Beautiful Soup] | ||

+ | **Screen-scraping tools | ||

+ | *[http://www.mlsec.org/sally/ SALLY] | ||

+ | **Tool for embedding strings into vector spaces | ||

+ | *[http://radimrehurek.com/gensim/ Gensim] | ||

+ | **Topic modeling | ||

+ | |||

+ | ==== Collaborative Filtering ==== | ||

+ | *[http://prea.gatech.edu/ PREA] | ||

+ | **Personalized Recommendation Algorithms Toolkit | ||

+ | *[http://svdfeature.apexlab.org/wiki/Main_Page SVDFeature] | ||

+ | **Collaborative Filtering and Ranking Toolkit | ||

+ | |||

+ | ==== Computer Vision ==== | ||

+ | *[http://opencv.willowgarage.com/documentation/index.html OpenCV] | ||

+ | **Computer Vision Library | ||

+ | **Has ML component (SVM, trees, etc) | ||

+ | **Online tutorials [http://www.pages.drexel.edu/~nk752/tutorials.html here] | ||

+ | *[http://drwn.anu.edu.au/ DARWIN] | ||

+ | **Generic C++ ML and Computer Vision Library | ||

+ | *[http://sourceforge.net/projects/petavision/ PetaVision] | ||

+ | **Developing a real-time, full-scale model of the primate visual cortex. | ||

+ | |||

+ | ==== Audio Processing ==== | ||

+ | *[http://tlecomte.github.com/friture/ Friture] | ||

+ | **Real-time spectrogram generation | ||

+ | *[http://code.google.com/p/pyo/ pyo] | ||

+ | **Real-time audio signal processing | ||

+ | *[https://github.com/jsawruk/pymir PYMir] | ||

+ | **A library for reading mp3's into python, and doing analysis | ||

+ | *[http://www.fon.hum.uva.nl/praat/ PRAAT] | ||

+ | **Speech analysis toolkit | ||

+ | *[http://ofer.sci.ccny.cuny.edu/sound_analysis_pro Sound Analysis Pro] | ||

+ | **Tool for analyzing animal sounds | ||

+ | *[http://luscinia.sourceforge.net/ Luscinia] | ||

+ | **Software for archiving, measuring, and analyzing bioacoustic data | ||

+ | *[http://wiki.python.org/moin/PythonInMusic List of Sound Tools for Python] | ||

+ | *[http://jasperproject.github.io/ Jasper] | ||

+ | **Voice-control anything! | ||

+ | |||

+ | ==== Data Visualization ==== | ||

+ | *[http://www.ailab.si/orange/ Orange] | ||

+ | **Strong data visualization component | ||

*[http://gephi.org/ Gephi] | *[http://gephi.org/ Gephi] | ||

**Graph Visualization | **Graph Visualization | ||

*[http://had.co.nz/ggplot2/ ggplot] | *[http://had.co.nz/ggplot2/ ggplot] | ||

**Nice plotting package for R | **Nice plotting package for R | ||

+ | *[http://code.enthought.com/projects/mayavi/ MayaVi2] | ||

+ | **3D Scientific Data Visualization | ||

+ | *[http://cytoscape.github.io/cytoscape.js/ Cytoscape] | ||

+ | **A JavaScript graph library for analysis and visualisation | ||

+ | *[https://plot.ly/ plot.ly] | ||

+ | **Web-based plotting | ||

+ | *[http://chimera.labs.oreilly.com/books/1230000000345/ch02.html D3 Ebook] | ||

+ | **Has a good list of HTML/CSS/Javascript data visualization tools. | ||

+ | *[https://plot.ly/ plotly] | ||

+ | **Python plotting tool | ||

+ | ==== Cluster Computing ==== | ||

+ | *[http://lucene.apache.org/mahout/ Mahout] | ||

+ | **Hadoop cluster based ML package. | ||

+ | *[http://web.mit.edu/star/cluster/ STAR: Cluster] | ||

+ | **Easily build your own Python computing cluster on Amazon EC2 | ||

+ | |||

+ | ==== Database Stuff ==== | ||

+ | *[http://madlib.net/ MADlib] | ||

+ | **Machine learning algorithms for in-database data | ||

+ | *[http://www.joyent.com/products/manta Manta] | ||

+ | **Distributed object storage | ||

+ | |||

+ | ==== Neural Simulation ==== | ||

+ | *[http://nengo.ca/ Nengo] | ||

+ | |||

+ | ==== Other ==== | ||

+ | *[http://jmlr.csail.mit.edu/mloss/ Journal of Machine Learning Software List] | ||

=== Presentations and other Materials === | === Presentations and other Materials === | ||

Line 100: | Line 247: | ||

* [http://www.linkedin.com/groupAnswers?viewQuestionAndAnswers=&discussionID=20096092&gid=77616&trk=EML_anet_qa_ttle-0Nt79xs2RVr6JBpnsJt7dBpSBA LinkedIn] discussion on good resources for data mining and predictive analytics | * [http://www.linkedin.com/groupAnswers?viewQuestionAndAnswers=&discussionID=20096092&gid=77616&trk=EML_anet_qa_ttle-0Nt79xs2RVr6JBpnsJt7dBpSBA LinkedIn] discussion on good resources for data mining and predictive analytics | ||

* [http://www.face-rec.org/algorithms/ Face Recognition Algorithms] | * [http://www.face-rec.org/algorithms/ Face Recognition Algorithms] | ||

+ | * [http://www.ics.uci.edu/~welling/classnotes/classnotes.html Max Welling's ML classnotes] | ||

=== Topics to Learn and Teach === | === Topics to Learn and Teach === | ||

Line 144: | Line 292: | ||

=== [[Machine Learning/Meeting Notes|Meeting Notes]]=== | === [[Machine Learning/Meeting Notes|Meeting Notes]]=== | ||

+ | |||

+ | [[Category:Events]] | ||

+ | [[Category:Projects]] |

## Latest revision as of 11:17, 10 October 2016

## Contents

### Join the Mailing List[edit]

https://www.noisebridge.net/mailman/listinfo/ml

### Next Meeting[edit]

- When: Thursday, August 14, 2014 @ 6:00pm
- Where: 2169 Mission St. (Church classroom)
- Topic:
- Details:
- Who: Andy McMurry

### Learn about Data Science and Machine Learning[edit]

##### Classes[edit]

- Coursera Machine Learning Course with Andrew Ng
- Coursera Computational Neuroscience Course with Rajesh P N Rao and Adrienne Fairhall
- MIT Machine Learning Class with Tommi Jaakkola
- Stanford CS229
- Carnegie Mellon Machine Learning Course with Tom Mitchell
- Linear Algebra with Gilbert Strang
- Neural Networks Class with Hugo Larochelle

#### Books[edit]

- Elements of Statistical Learning
- Pattern Recognition and Machine Learning
- Information Theory, Inference, and Learning Algorithms
- Interactive Data Visualization for the Web (D3)
- Introduction to R
- Introduction to Probability (Grinstead and Snell)
- Modern Introduction to Probability and Statistics (Kraaikamp and Meester)
- Bayesian Reasoning and Machine Learning

#### Tutorials[edit]

#### Noisebridge ML Class Slides[edit]

- Intro to Machine Learning
- A Brief Tour of Statistics
- Generalized Linear Models
- Neural Nets Workshop
- Support Vector Machines
- Random Forests
- Independent Components Analysis
- Deep Nets

### Code and SourceForge Site[edit]

- We have a Sourceforge Project
- We have a git repository on the project page, accessible as:

git clone git://ml-noisebridge.git.sourceforge.net/gitroot/ml-noisebridge/ml-noisebridge

- Send an email to the list if you want to become an administrator on the site to get write access to the git repo!

### Future Talks and Topics, Ideas[edit]

- Random Forests in R
- Restricted Boltzmann Machines (Mike S, some day)
- Analyzing brain cells (Mike S)
- Deep Nets w/ Stacked Autoencoders (Mike S, some day)
- Generalized Linear Models (Mike S, Erin L? some day)
- Graphical Models
- Working with the Kinect
- Computer Vision with OpenCV

### Projects[edit]

- Small Group Subproblems
- Fundraising
- Noisebridge Machine Learning Course
- Kaggle Social Network Contest
- KDD Competition 2010
- HIV

### Datasets and Websites[edit]

- UCI Machine Learning Repository
- DataSF.org
- Infochimps
- Face Recognition Databases
- Time Series Data Library
- Data Q&A Forum
- Metaoptimize
- Quora ML Page
- A ton of Weather Data
- MLcomp
- Upload your algorithm and objectively compare it's performance to other algorithms

- Social Security Death Master File!
- SIPRI Social Databases
- Wealth of information on international arms transfers and peace missions.

- Amazon AWS Public Datasets
- UCDP/PRIO Armed Conflict Datasets
- Socrata Government Datasets
- US City Census Data
- Yahoo Labs Datasets

### Software Tools[edit]

#### Generic ML Libraries[edit]

- Weka
- a collection of data mining tools and machine learning algorithms.

- scikits.learn
- Machine learning Python package

- scikits.statsmodels
- Statistical models to go with scipy

- PyBrain
- Does feedforward, recurrent, SOM, deep belief nets.

- LIBSVM
- c-based SVM package

- PyML
- MDP
- Modular framework, has lots of stuff!

- VirtualBox Virtual Box Image with Pre-installed Libraries listed here
- sympy Does symbolic math
- Waffles
- Open source C++ set of machine learning command line tools.

- RapidMiner
- Mobile Robotic Programming Toolkit
- nitime
- NeuroImaging in Python, has some good time series analysis stuff and multi-variate response fitting.

- Pandas
- Data analysis workflow in python

- PyTables
- Adds querying capabilities to HDF5 files

- statsmodels
- Regression, time series analysis, statistics stuff for python

- Vowpal Wabbit
- "Intrinsically Fast" implementation of gradient descent for large datasets

- Shogun
- Fast implementations of SVMs

- MLPACK
- High performance scalable ML Library

- Torch
- MATLAB-like environment for state-of-the art ML libraries written in LUA

#### Deep Nets[edit]

- Theano
- Symbolic Expressions and Transparent GPU Integration

- Caffe
- Convolutional Neural Networks on GPU

- Neurolab
- Has support for recurrent neural nets

#### Online ML[edit]

- MOA (Massive Online Analysis)
- Offshoot of weka, has all online-algorithms

- Jubatus
- Distributed Online ML

- DOGMA
- MATLAB-based online learning stuff

- libol
- oll
- scw-learning

#### Graphical Models[edit]

- BUGS
- MCMC for Bayesian Models

- JAGS
- Hierarchical Bayesian Models

- Stan
- A graphical model compiler

- Jayes
- Bayesian networks in Java

- ToPS
- Probabilistic models of sequences

- PyMC
- Bayesian Models in Python

#### Text Stuff[edit]

- Beautiful Soup
- Screen-scraping tools

- SALLY
- Tool for embedding strings into vector spaces

- Gensim
- Topic modeling

#### Collaborative Filtering[edit]

- PREA
- Personalized Recommendation Algorithms Toolkit

- SVDFeature
- Collaborative Filtering and Ranking Toolkit

#### Computer Vision[edit]

- OpenCV
- Computer Vision Library
- Has ML component (SVM, trees, etc)
- Online tutorials here

- DARWIN
- Generic C++ ML and Computer Vision Library

- PetaVision
- Developing a real-time, full-scale model of the primate visual cortex.

#### Audio Processing[edit]

- Friture
- Real-time spectrogram generation

- pyo
- Real-time audio signal processing

- PYMir
- A library for reading mp3's into python, and doing analysis

- PRAAT
- Speech analysis toolkit

- Sound Analysis Pro
- Tool for analyzing animal sounds

- Luscinia
- Software for archiving, measuring, and analyzing bioacoustic data

- List of Sound Tools for Python
- Jasper
- Voice-control anything!

#### Data Visualization[edit]

- Orange
- Strong data visualization component

- Gephi
- Graph Visualization

- ggplot
- Nice plotting package for R

- MayaVi2
- 3D Scientific Data Visualization

- Cytoscape
- A JavaScript graph library for analysis and visualisation

- plot.ly
- Web-based plotting

- D3 Ebook
- Has a good list of HTML/CSS/Javascript data visualization tools.

- plotly
- Python plotting tool

#### Cluster Computing[edit]

- Mahout
- Hadoop cluster based ML package.

- STAR: Cluster
- Easily build your own Python computing cluster on Amazon EC2

#### Database Stuff[edit]

#### Neural Simulation[edit]

#### Other[edit]

### Presentations and other Materials[edit]

- Awesome Machine Learning Applications -- A list of cool applications of ML
- Hands-on Machine Learning, a presentation jbm gave on 2009-01-07.
- http://www.youtube.com/user/StanfordUniversity#g/c/A89DCFA6ADACE599 Stanford Machine Learning online course videos]
- Media:Brief_statistics_slides.pdf, a presentation given on statistics for the machine learning group
- LinkedIn discussion on good resources for data mining and predictive analytics
- Face Recognition Algorithms
- Max Welling's ML classnotes

### Topics to Learn and Teach[edit]

NBML Course - Noisebridge Machine Learning Curriculum (work-in-progress)

CS229 - The Stanford Machine learning Course @ noisebridge

- Supervised Learning
- Linear Regression
- Linear Discriminants
- Neural Nets/Radial Basis Functions
- Support Vector Machines
- Classifier Combination [1]
- A basic decision tree builder, recursive and using entropy metrics

- Unsupervised Learning
- Hidden Markov Models
- Clustering: PCA, k-Means, Expectation-Maximization
- Graphical Modeling
- Generative Models: gaussian distribution, multinomial distributions, HMMs, Naive Bayes
- Deep Belief Networks & Restricted Boltzmann Machines

- Reinforcement Learning
- Temporal Difference Learning

- Math, Probability & Statistics
- Metric spaces and what they mean
- Fundamentals of probabilities
- Decision Theory (Bayesian)
- Maximum Likelihood
- Bias/Variance Tradeoff, VC Dimension
- Bagging, Bootstrap, Jacknife [2]
- Information Theory: Entropy, Mutual Information, Gaussian Channels
- Estimation of Misclassification [3]
- No-Free Lunch Theorem [4]

- Machine Learning SDK's

- Applications
- Collective Intelligence & Recommendation Engines