【转载】Getting started with Ramp: Detecting insults

Ramp is a python library for rapid machine learning prototyping. It provides a simple, declarative syntax for exploring features, algorithms and transformations quickly and efficiently. At its core it‘s a pandas wrapper around various python machine learning and statistics libraries (scikit-learn, rpy2, etc.). Some features:

• Fast caching and persistence of all intermediate and final calculations -- nothing is recomputed unnecessarily.
• Advanced training and preparation logic. Ramp respects the current training set, even when using complex trained features and blended predictions, and also tracks the given preparation set (the x values used in feature preparation -- e.g. the mean and stdev used for feature normalization.)
• A growing library of feature transformations, metrics and estimators. Ramp‘s simple API allows for easy extension.

 

 

Detecting insults

Let‘s try Ramp out on the Kaggle Detecting Insults in Social Commentary challenge. I recommend grabbing Ramp straight from the Github repo so you are up-to-date.

First, we load up the data using pandas. (You can download the data from the Kaggle site, you‘ll have to sign up.)
We‘ve got about 4000 comments along with the date they were posted and a boolean indicating whether or not the comment was classified as insulting. If you‘re curious, the insults in question range from the relatively civilized ("... you don‘t have a basic grasp on biology") to the mundane ("suck my d***, *sshole"), to the truly bottom-of-the-internet horrific (pass).

Anyways, let‘s set up a DataContext for Ramp. This involves providing a store (to save cached results to) and a pandas DataFrame with our actual data.
We just provided a directory path for the store, so Ramp will use the default HDFPickleStore, which attempts to store objects (on disk) in the fast HDF5 format and falls back to pickling if that is not an option.

Next, we‘ll specify a base configuration for our analysis. 
Here we have specified the DataFrame column ‘Insult‘ as the target for our classification and the AUC for our metric.

 

Model exploration

Now comes the fun part -- exploring features and algorithms. We create a ConfigFactory for this purpose, which takes our base config and provides an iterator over declared feature sets and estimators. 
We‘ve defined some base features along with four feature sets that seem promising.

 Now, let‘s run cross-validation and compare the results: 
Here are a couple snippets of the output: 
The Logistic Regression model has of course dominated Naive Bayes. The best feature sets are the 100-vector LSI and the 1000-word character 5-grams. Once a feature is computed, it does not need to be computed again in separate contexts. The binary feature selection is an exception to this though: because it uses target "y" values to select features, Ramp needs to recreate it for each cross validation fold using only the given training values (You can also cheat and tell it not to do this, training it just once against the entire data set.)

 

Predictions

We can also create a quick utility that processes a given comment and spit out it‘s probability of being an insult: 
And we can run it on some sample text:
Ramp will need to create the model for the full training data set the first time you make a prediction, but will then cache and store it, allowing you to quickly classify subsequent text.

 

And more