Editing Machine Learning/OmniscopeVisualization

== How to visualize data in a SQLITE database using Omniscope ==

To get an immediate sense of what this tutorial allows you to do, see [http://swarmfinancial.com/screencasts/nb/omniscopeVisualization.swf the screencast].


Download the [http://www.visokio.com/omniscope Omniscope] Trial, which will be good for a few weeks.  I have heard people say that they just create a new OSX user to install the program anew and to continue using Omniscope trial beyond its expiration date.  

Download the [http://www.zentus.com/sqlitejdbc/ Zentus] driver -- sqlitejdbc-v054.jar is the version that works on my OSX 10.5/10.6, your mileage may vary.

Connect to Database -> JDBC database -> enter details:

      /path/to/sqlitejdbc-v054.jar
      org.sqlite.JDBC
      jdbc:sqlite:/path/to/dbKdd.dbev
    
No authentication, click just next; then select table name -- in our case bridge or algebra. Click next in customize SQL;

Then you can select the fields to exclude in your visualization bu unchecking them: 
In the KDD case, I chose to remove all fields that are not relevant in test data: 
 all time/duration fields
 correct first attempt
 incorrects
 hints
 corrects

Click next then finish. It'll import your data into an IOK file -- which you need to save;  Once you've done the conversion into the IOK file you won't need the DB any more for omniscope (but can refresh if data changes).

If necessary, delete the first row (right click) which may contain header information, not actual data.

You can then format your data: Click Data -> Manage Fields.
Select the proper data types; e.g. row field is an Integer in our KDD data

Note: Many operations in Omniscope take a long time; expect to wait for a few minutes when you click certain things or make selections.  This is annoying but it occurs when you deal with datasets greater than a few hundred MB.  Save often, so that when your Omniscope hangs you can Force Quit it and open it again if necessary.

Andy