Machine Learning

From Noisebridge
(Difference between revisions)
Jump to: navigation, search
m
m (Audio Processing)
 
(89 intermediate revisions by 12 users not shown)
Line 1: Line 1:
 +
=== Join the Mailing List ===
 +
 +
https://www.noisebridge.net/mailman/listinfo/ml
 +
 
=== Next Meeting===
 
=== Next Meeting===
  
*When: Some time in August?
+
*When: Thursday, February 13, 2014 @ 6:30pm
*Where: 2169 Mission St. (back NE corner, Church classroom)
+
*Where: 2169 Mission St. (Church classroom)
*Topic:  
+
*Topic: Bayesian Inference for everyone
*Details: We're having a summer vacation of sorts, but feel free to self-organize and have a meetup by emailing the [https://www.noisebridge.net/mailman/listinfo/ml mailing list]!
+
*Details:  
*Who:  
+
*Who: Sam Tepper
 +
 
 +
=== Learn about Data Science and Machine Learning ===
  
=== Take the Noisebridge ML Survey ===
+
===== Classes =====
[http://www.surveymonkey.com/s/W2T9ZB6 Take a survey] and vote for what you want to learn!
+
*[https://www.coursera.org/course/ml Coursera Machine Learning Course with Andrew Ng]
 +
*[https://www.coursera.org/course/compneuro Coursera Computational Neuroscience Course with Adrienne Fairhall]
 +
*[http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-867-machine-learning-fall-2006/ MIT Machine Learning Class with Tommi Jaakkola]
 +
*[http://cs229.stanford.edu/materials.html Stanford CS229]
 +
*[http://www.cs.cmu.edu/~tom/10701_sp11/lectures.shtml Carnegie Mellon Machine Learning Course with Tom Mitchell]
 +
*[http://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-spring-2010/ Linear Algebra with Gilbert Strang]
 +
*[https://www.youtube.com/playlist?list=PL6Xpj9I5qXYEcOhn7TqghAJ6NAPrNmUBH Neural Networks Class with Hugo Larochelle]
  
=== Crowdsourced Q&A ===
+
==== Books ====
Are you working on a data mining, machine learning, or statistics problem? Do you want some help? Consider sending an email to the [https://www.noisebridge.net/mailman/listinfo/ml mailing list] about it! Also consider setting up a day to come in and talk about the project you're working on and get input from <span class="plainlinks">[http://www.andrewflusche.com/services/spotsylvania-reckless-driving-defense/<span style="color:black;font-weight:normal; text-decoration:none!important; background:none!important; text-decoration:none;">Spotsylvania reckless driving</span>] other ML people.
+
*[http://statweb.stanford.edu/~tibs/ElemStatLearn/ Elements of Statistical Learning]
 +
*[https://www.google.com/search?client=ubuntu&channel=fs&q=pattern+recognition+and+machine+learning&ie=utf-8&oe=utf-8#channel=fs&q=pattern+recognition+and+machine+learning+pdf Pattern Recognition and Machine Learning]
 +
*[https://www.google.com/search?&channel=fs&q=+Information+Theory%2C+Inference%2C+and+Learning+Algorithms.&ie=utf-8&oe=utf-8#channel=fs&q=Information+Theory%2C+Inference%2C+and+Learning+Algorithms+pdf Information Theory, Inference, and Learning Algorithms]
 +
*[http://chimera.labs.oreilly.com/books/1230000000345 Interactive Data Visualization for the Web (D3)]
 +
*[http://cran.r-project.org/doc/manuals/R-intro.pdf Introduction to R]
 +
*[http://www.dartmouth.edu/~chance/teaching_aids/books_articles/probability_book/amsbook.mac.pdf Introduction to Probability (Grinstead and Snell)]
 +
*[http://www.cis.temple.edu/~latecki/Courses/CIS2033-Spring12/A_modern_intro_probability_statistics_Dekking05.pdf Modern Introduction to Probability and Statistics (Kraaikamp and Meester)]
 +
*[http://web4.cs.ucl.ac.uk/staff/D.Barber/textbook/090310.pdf Bayesian Reasoning and Machine Learning]
  
=== About Us ===
+
==== Tutorials ====
We're a loosely-knit stochastic federation of people who like Noisebridge and like machine learning. What is machine learning? It's broad field that typically involves training computer models to solve problems. How can you <span class="plainlinks">[http://www.monoloop.com<span style="color:black;font-weight:normal; text-decoration:none!important; background:none!important; text-decoration:none;">website personalization</span>] participate? Join the [https://www.noisebridge.net/mailman/listinfo/ml mailing list], send an email and introduce yourself. Show up to the next meeting, share your thoughts. Participate in projects or start your own. Go to workshops, write code at workshops, learn stuff, give workshops of your own! All are welcome.
+
*[http://nbviewer.ipython.org/github/unpingco/Python-for-Signal-Processing/tree/master/ Signal Processing IPython Notebooks]
 +
*[http://scikit-learn.org/stable/tutorial/basic/tutorial.html Introduction to ML with scikits.learn]
 +
*[http://www.sagemath.org/doc/tutorial/ Learn how to use SAGE]
  
=== Talks and Workshops ===
+
==== Noisebridge ML Class Slides ====
We've given lots of workshops and talks over the past year or so, here's a few. Many of the workshops we've given previously are recurring and will be given again, especially upon request!
+
 
*[[NBML/Workshops/Intro to Machine Learning|Intro to Machine Learning]]
 
*[[NBML/Workshops/Intro to Machine Learning|Intro to Machine Learning]]
 
*[[NBML/Workshops/Brief Tour of Statistics|A Brief Tour of Statistics]]
 
*[[NBML/Workshops/Brief Tour of Statistics|A Brief Tour of Statistics]]
Line 42: Line 62:
 
*Working with the Kinect
 
*Working with the Kinect
 
*Computer Vision with OpenCV
 
*Computer Vision with OpenCV
 
=== Mailing List ===
 
 
https://www.noisebridge.net/mailman/listinfo/ml
 
  
 
=== Projects ===
 
=== Projects ===
Line 68: Line 84:
 
**Upload your algorithm and objectively compare it's performance to other algorithms
 
**Upload your algorithm and objectively compare it's performance to other algorithms
 
*[http://www.ntis.gov/products/ssa-dmf.aspx Social Security Death Master File!]
 
*[http://www.ntis.gov/products/ssa-dmf.aspx Social Security Death Master File!]
 +
*[http://www.sipri.org/databases SIPRI Social Databases]
 +
**Wealth of information on international arms transfers and peace missions.
 +
*[http://aws.amazon.com/publicdatasets/ Amazon AWS Public Datasets]
 +
*[http://www.prio.no/Data/Armed-Conflict/ UCDP/PRIO Armed Conflict Datasets]
 +
*[https://opendata.socrata.com/browse Socrata Government Datasets]
 +
*[http://us-city.census.okfn.org/ US City Census Data]
 +
*[http://webscope.sandbox.yahoo.com/catalog.php Yahoo Labs Datasets]
  
 
=== Software Tools ===
 
=== Software Tools ===
Line 74: Line 97:
 
*[http://www.cs.waikato.ac.nz/ml/weka/ Weka]
 
*[http://www.cs.waikato.ac.nz/ml/weka/ Weka]
 
**a collection of data mining tools and machine learning algorithms.
 
**a collection of data mining tools and machine learning algorithms.
*[http://moa.cs.waikato.ac.nz/ MOA (Massive Online Analysis)]
 
**Offshoot of weka, has all online-algorithms
 
 
*[http://scikit-learn.sourceforge.net/ scikits.learn]
 
*[http://scikit-learn.sourceforge.net/ scikits.learn]
 
**Machine learning Python package
 
**Machine learning Python package
Line 88: Line 109:
 
**Modular framework, has lots of stuff!
 
**Modular framework, has lots of stuff!
 
*[[Machine Learning/VirtualBox|VirtualBox]] Virtual Box Image with Pre-installed Libraries listed here
 
*[[Machine Learning/VirtualBox|VirtualBox]] Virtual Box Image with Pre-installed Libraries listed here
*[http://deeplearning.net/software/theano/ Theano: Symbolic Expressions and Transparent GPU Integration]
 
 
*[http://sympy.org sympy] Does symbolic math
 
*[http://sympy.org sympy] Does symbolic math
 
*[http://waffles.sourceforge.net/ Waffles]
 
*[http://waffles.sourceforge.net/ Waffles]
Line 100: Line 120:
 
*[http://www.pytables.org/moin PyTables]
 
*[http://www.pytables.org/moin PyTables]
 
**Adds querying capabilities to HDF5 files
 
**Adds querying capabilities to HDF5 files
 +
*[http://statsmodels.sourceforge.net/ statsmodels]
 +
**Regression, time series analysis, statistics stuff for python
 +
*[https://github.com/JohnLangford/vowpal_wabbit/wiki Vowpal Wabbit]
 +
**"Intrinsically Fast" implementation of gradient descent for large datasets
 +
*[http://www.shogun-toolbox.org/ Shogun]
 +
**Fast implementations of SVMs
 +
*[http://www.mlpack.org/ MLPACK]
 +
**High performance scalable ML Library
 +
*[http://www.torch.ch/ Torch]
 +
**MATLAB-like environment for state-of-the art ML libraries written in LUA
 +
 +
==== Deep Nets ====
 +
*[http://deeplearning.net/software/theano/ Theano]
 +
**Symbolic Expressions and Transparent GPU Integration
 +
*[http://caffe.berkeleyvision.org/ Caffe]
 +
**Convolutional Neural Networks on GPU
 +
*[https://code.google.com/p/neurolab/ Neurolab]
 +
**Has support for recurrent neural nets
 +
 +
==== Online ML ====
 +
*[http://moa.cs.waikato.ac.nz/ MOA (Massive Online Analysis)]
 +
**Offshoot of weka, has all online-algorithms
 +
*[http://jubat.us/en/ Jubatus]
 +
**Distributed Online ML
 +
*[http://dogma.sourceforge.net/ DOGMA]
 +
**MATLAB-based online learning stuff
 +
*[http://code.google.com/p/libol/ libol]
 +
*[http://code.google.com/p/oll/ oll]
 +
*[http://code.google.com/p/scw-learning/ scw-learning]
 +
 +
==== Graphical Models ====
 +
*[http://www.mrc-bsu.cam.ac.uk/bugs/ BUGS]
 +
**MCMC for Bayesian Models
 +
*[http://mcmc-jags.sourceforge.net/ JAGS]
 +
**Hierarchical Bayesian Models
 +
*[http://mc-stan.org/ Stan]
 +
**A graphical model compiler
 +
*[https://github.com/kutschkem/Jayes Jayes]
 +
**Bayesian networks in Java
 +
*[http://tops.sourceforge.net/ ToPS]
 +
**Probabilistic models of sequences
 +
*[http://pymc-devs.github.io/pymc/ PyMC]
 +
**Bayesian Models in Python
 +
 +
==== Text Stuff ====
 +
*[http://www.crummy.com/software/BeautifulSoup/ Beautiful Soup]
 +
**Screen-scraping tools
 +
*[http://www.mlsec.org/sally/ SALLY]
 +
**Tool for embedding strings into vector spaces
 +
*[http://radimrehurek.com/gensim/ Gensim]
 +
**Topic modeling
 +
 +
==== Collaborative Filtering ====
 +
*[http://prea.gatech.edu/ PREA]
 +
**Personalized Recommendation Algorithms Toolkit
 +
*[http://svdfeature.apexlab.org/wiki/Main_Page SVDFeature]
 +
**Collaborative Filtering and Ranking Toolkit
  
 
==== Computer Vision ====
 
==== Computer Vision ====
Line 106: Line 183:
 
**Has ML component (SVM, trees, etc)
 
**Has ML component (SVM, trees, etc)
 
**Online tutorials [http://www.pages.drexel.edu/~nk752/tutorials.html here]
 
**Online tutorials [http://www.pages.drexel.edu/~nk752/tutorials.html here]
 +
*[http://drwn.anu.edu.au/ DARWIN]
 +
**Generic C++ ML and Computer Vision Library
 +
*[http://sourceforge.net/projects/petavision/ PetaVision]
 +
**Developing a real-time, full-scale model of the primate visual cortex.
  
 
==== Audio Processing ====
 
==== Audio Processing ====
Line 114: Line 195:
 
*[https://github.com/jsawruk/pymir PYMir]  
 
*[https://github.com/jsawruk/pymir PYMir]  
 
**A library for reading mp3's into python, and doing analysis  
 
**A library for reading mp3's into python, and doing analysis  
 +
*[http://www.fon.hum.uva.nl/praat/ PRAAT]
 +
**Speech analysis toolkit
 +
*[http://ofer.sci.ccny.cuny.edu/sound_analysis_pro Sound Analysis Pro]
 +
**Tool for analyzing animal sounds
 +
*[http://luscinia.sourceforge.net/ Luscinia]
 +
**Software for archiving, measuring, and analyzing bioacoustic data
 
*[http://wiki.python.org/moin/PythonInMusic List of Sound Tools for Python]
 
*[http://wiki.python.org/moin/PythonInMusic List of Sound Tools for Python]
 +
*[http://jasperproject.github.io/ Jasper]
 +
**Voice-control anything!
  
 
==== Data Visualization ====
 
==== Data Visualization ====
Line 125: Line 214:
 
*[http://code.enthought.com/projects/mayavi/ MayaVi2]
 
*[http://code.enthought.com/projects/mayavi/ MayaVi2]
 
**3D Scientific Data Visualization
 
**3D Scientific Data Visualization
 
+
*[http://cytoscape.github.io/cytoscape.js/ Cytoscape]
 +
**A JavaScript graph library for analysis and visualisation
 +
*[https://plot.ly/ plot.ly]
 +
**Web-based plotting
 +
*[http://chimera.labs.oreilly.com/books/1230000000345/ch02.html D3 Ebook]
 +
**Has a good list of HTML/CSS/Javascript data visualization tools.
 +
*[https://plot.ly/ plotly]
 +
**Python plotting tool
 
==== Cluster Computing ====
 
==== Cluster Computing ====
 
*[http://lucene.apache.org/mahout/ Mahout]
 
*[http://lucene.apache.org/mahout/ Mahout]
Line 131: Line 227:
 
*[http://web.mit.edu/star/cluster/ STAR: Cluster]
 
*[http://web.mit.edu/star/cluster/ STAR: Cluster]
 
**Easily build your own Python computing cluster on Amazon EC2
 
**Easily build your own Python computing cluster on Amazon EC2
 +
 +
==== Database Stuff ====
 +
*[http://madlib.net/ MADlib]
 +
**Machine learning algorithms for in-database data
 +
*[http://www.joyent.com/products/manta Manta]
 +
**Distributed object storage
 +
 +
==== Neural Simulation ====
 +
*[http://nengo.ca/ Nengo]
  
 
==== Other ====
 
==== Other ====
Line 187: Line 292:
  
 
=== [[Machine Learning/Meeting Notes|Meeting Notes]]===
 
=== [[Machine Learning/Meeting Notes|Meeting Notes]]===
 +
 +
[[Category:Events]]
 +
[[Category:Projects]]

Latest revision as of 10:49, 9 April 2014

Contents

[edit] Join the Mailing List

https://www.noisebridge.net/mailman/listinfo/ml

[edit] Next Meeting

  • When: Thursday, February 13, 2014 @ 6:30pm
  • Where: 2169 Mission St. (Church classroom)
  • Topic: Bayesian Inference for everyone
  • Details:
  • Who: Sam Tepper

[edit] Learn about Data Science and Machine Learning

[edit] Classes

[edit] Books

[edit] Tutorials

[edit] Noisebridge ML Class Slides

[edit] Code and SourceForge Site

    git clone git://ml-noisebridge.git.sourceforge.net/gitroot/ml-noisebridge/ml-noisebridge
  • Send an email to the list if you want to become an administrator on the site to get write access to the git repo!

[edit] Future Talks and Topics, Ideas

  • Random Forests in R
  • Restricted Boltzmann Machines (Mike S, some day)
  • Analyzing brain cells (Mike S)
  • Deep Nets w/ Stacked Autoencoders (Mike S, some day)
  • Generalized Linear Models (Mike S, Erin L? some day)
  • Graphical Models
  • Working with the Kinect
  • Computer Vision with OpenCV

[edit] Projects

[edit] Datasets and Websites

[edit] Software Tools

[edit] Generic ML Libraries

  • Weka
    • a collection of data mining tools and machine learning algorithms.
  • scikits.learn
    • Machine learning Python package
  • scikits.statsmodels
    • Statistical models to go with scipy
  • PyBrain
    • Does feedforward, recurrent, SOM, deep belief nets.
  • LIBSVM
    • c-based SVM package
  • PyML
  • MDP
    • Modular framework, has lots of stuff!
  • VirtualBox Virtual Box Image with Pre-installed Libraries listed here
  • sympy Does symbolic math
  • Waffles
    • Open source C++ set of machine learning command line tools.
  • RapidMiner
  • Mobile Robotic Programming Toolkit
  • nitime
    • NeuroImaging in Python, has some good time series analysis stuff and multi-variate response fitting.
  • Pandas
    • Data analysis workflow in python
  • PyTables
    • Adds querying capabilities to HDF5 files
  • statsmodels
    • Regression, time series analysis, statistics stuff for python
  • Vowpal Wabbit
    • "Intrinsically Fast" implementation of gradient descent for large datasets
  • Shogun
    • Fast implementations of SVMs
  • MLPACK
    • High performance scalable ML Library
  • Torch
    • MATLAB-like environment for state-of-the art ML libraries written in LUA

[edit] Deep Nets

  • Theano
    • Symbolic Expressions and Transparent GPU Integration
  • Caffe
    • Convolutional Neural Networks on GPU
  • Neurolab
    • Has support for recurrent neural nets

[edit] Online ML

[edit] Graphical Models

  • BUGS
    • MCMC for Bayesian Models
  • JAGS
    • Hierarchical Bayesian Models
  • Stan
    • A graphical model compiler
  • Jayes
    • Bayesian networks in Java
  • ToPS
    • Probabilistic models of sequences
  • PyMC
    • Bayesian Models in Python

[edit] Text Stuff

[edit] Collaborative Filtering

  • PREA
    • Personalized Recommendation Algorithms Toolkit
  • SVDFeature
    • Collaborative Filtering and Ranking Toolkit

[edit] Computer Vision

  • OpenCV
    • Computer Vision Library
    • Has ML component (SVM, trees, etc)
    • Online tutorials here
  • DARWIN
    • Generic C++ ML and Computer Vision Library
  • PetaVision
    • Developing a real-time, full-scale model of the primate visual cortex.

[edit] Audio Processing

[edit] Data Visualization

  • Orange
    • Strong data visualization component
  • Gephi
    • Graph Visualization
  • ggplot
    • Nice plotting package for R
  • MayaVi2
    • 3D Scientific Data Visualization
  • Cytoscape
    • A JavaScript graph library for analysis and visualisation
  • plot.ly
    • Web-based plotting
  • D3 Ebook
    • Has a good list of HTML/CSS/Javascript data visualization tools.
  • plotly
    • Python plotting tool

[edit] Cluster Computing

  • Mahout
    • Hadoop cluster based ML package.
  • STAR: Cluster
    • Easily build your own Python computing cluster on Amazon EC2

[edit] Database Stuff

  • MADlib
    • Machine learning algorithms for in-database data
  • Manta
    • Distributed object storage

[edit] Neural Simulation

[edit] Other

[edit] Presentations and other Materials

[edit] Topics to Learn and Teach

NBML Course - Noisebridge Machine Learning Curriculum (work-in-progress)

CS229 - The Stanford Machine learning Course @ noisebridge

  • Supervised Learning
    • Linear Regression
    • Linear Discriminants
    • Neural Nets/Radial Basis Functions
    • Support Vector Machines
    • Classifier Combination [1]
    • A basic decision tree builder, recursive and using entropy metrics
  • Reinforcement Learning
    • Temporal Difference Learning
  • Math, Probability & Statistics
    • Metric spaces and what they mean
    • Fundamentals of probabilities
    • Decision Theory (Bayesian)
    • Maximum Likelihood
    • Bias/Variance Tradeoff, VC Dimension
    • Bagging, Bootstrap, Jacknife [2]
    • Information Theory: Entropy, Mutual Information, Gaussian Channels
    • Estimation of Misclassification [3]
    • No-Free Lunch Theorem [4]
  • Machine Learning SDK's
    • OpenCV ML component (SVM, trees, etc)
    • Mahout a Hadoop cluster based ML package.
    • Weka a collection of data mining tools and machine learning algorithms.
  • Applications
    • Collective Intelligence & Recommendation Engines

[edit] Meeting Notes

Personal tools