.. _example_applications_plot_out_of_core_classification.py:
======================================================
Out-of-core classification of text documents
======================================================
This is an example showing how scikit-learn can be used for classification
using an out-of-core approach: learning from data that doesn't fit into main
memory. We make use of an online classifier, i.e., one that supports the
partial_fit method, that will be fed with batches of examples. To guarantee
that the features space remains the same over time we leverage a
HashingVectorizer that will project each example into the same feature space.
This is especially useful in the case of text classification where new
features (words) may appear in each batch.
The dataset used in this example is Reuters-21578 as provided by the UCI ML
repository. It will be automatically downloaded and uncompressed on first run.
The plot represents the learning curve of the classifier: the evolution
of classification accuracy over the course of the mini-batches. Accuracy is
measured on the first 1000 samples, held out as a validation set.
To limit the memory consumption, we queue examples up to a fixed amount before
feeding them to the learner.
.. rst-class:: horizontal
*
.. image:: images/plot_out_of_core_classification_001.png
:scale: 47
*
.. image:: images/plot_out_of_core_classification_002.png
:scale: 47
*
.. image:: images/plot_out_of_core_classification_003.png
:scale: 47
*
.. image:: images/plot_out_of_core_classification_004.png
:scale: 47
**Script output**::
Test set is 982 documents (146 positive)
Passive-Aggressive classifier : 955 train docs ( 93 positive) 982 test docs ( 146 positive) accuracy: 0.889 in 2.26s ( 422 docs/s)
Perceptron classifier : 955 train docs ( 93 positive) 982 test docs ( 146 positive) accuracy: 0.875 in 2.27s ( 420 docs/s)
SGD classifier : 955 train docs ( 93 positive) 982 test docs ( 146 positive) accuracy: 0.906 in 2.28s ( 419 docs/s)
NB Multinomial classifier : 955 train docs ( 93 positive) 982 test docs ( 146 positive) accuracy: 0.852 in 2.34s ( 408 docs/s)
Passive-Aggressive classifier : 3794 train docs ( 398 positive) 982 test docs ( 146 positive) accuracy: 0.944 in 6.32s ( 600 docs/s)
Perceptron classifier : 3794 train docs ( 398 positive) 982 test docs ( 146 positive) accuracy: 0.930 in 6.33s ( 599 docs/s)
SGD classifier : 3794 train docs ( 398 positive) 982 test docs ( 146 positive) accuracy: 0.943 in 6.33s ( 599 docs/s)
NB Multinomial classifier : 3794 train docs ( 398 positive) 982 test docs ( 146 positive) accuracy: 0.868 in 6.40s ( 593 docs/s)
Passive-Aggressive classifier : 6733 train docs ( 759 positive) 982 test docs ( 146 positive) accuracy: 0.938 in 10.55s ( 638 docs/s)
Perceptron classifier : 6733 train docs ( 759 positive) 982 test docs ( 146 positive) accuracy: 0.923 in 10.56s ( 637 docs/s)
SGD classifier : 6733 train docs ( 759 positive) 982 test docs ( 146 positive) accuracy: 0.925 in 10.57s ( 637 docs/s)
NB Multinomial classifier : 6733 train docs ( 759 positive) 982 test docs ( 146 positive) accuracy: 0.885 in 10.63s ( 633 docs/s)
Passive-Aggressive classifier : 8634 train docs ( 984 positive) 982 test docs ( 146 positive) accuracy: 0.946 in 14.16s ( 609 docs/s)
Perceptron classifier : 8634 train docs ( 984 positive) 982 test docs ( 146 positive) accuracy: 0.926 in 14.16s ( 609 docs/s)
SGD classifier : 8634 train docs ( 984 positive) 982 test docs ( 146 positive) accuracy: 0.938 in 14.17s ( 609 docs/s)
NB Multinomial classifier : 8634 train docs ( 984 positive) 982 test docs ( 146 positive) accuracy: 0.897 in 14.23s ( 606 docs/s)
Passive-Aggressive classifier : 11463 train docs ( 1337 positive) 982 test docs ( 146 positive) accuracy: 0.952 in 18.47s ( 620 docs/s)
Perceptron classifier : 11463 train docs ( 1337 positive) 982 test docs ( 146 positive) accuracy: 0.908 in 18.47s ( 620 docs/s)
SGD classifier : 11463 train docs ( 1337 positive) 982 test docs ( 146 positive) accuracy: 0.947 in 18.48s ( 620 docs/s)
NB Multinomial classifier : 11463 train docs ( 1337 positive) 982 test docs ( 146 positive) accuracy: 0.907 in 18.54s ( 618 docs/s)
Passive-Aggressive classifier : 14330 train docs ( 1743 positive) 982 test docs ( 146 positive) accuracy: 0.936 in 22.49s ( 637 docs/s)
Perceptron classifier : 14330 train docs ( 1743 positive) 982 test docs ( 146 positive) accuracy: 0.944 in 22.49s ( 637 docs/s)
SGD classifier : 14330 train docs ( 1743 positive) 982 test docs ( 146 positive) accuracy: 0.949 in 22.50s ( 636 docs/s)
NB Multinomial classifier : 14330 train docs ( 1743 positive) 982 test docs ( 146 positive) accuracy: 0.916 in 22.57s ( 634 docs/s)
Passive-Aggressive classifier : 17257 train docs ( 2064 positive) 982 test docs ( 146 positive) accuracy: 0.955 in 26.66s ( 647 docs/s)
Perceptron classifier : 17257 train docs ( 2064 positive) 982 test docs ( 146 positive) accuracy: 0.951 in 26.67s ( 647 docs/s)
SGD classifier : 17257 train docs ( 2064 positive) 982 test docs ( 146 positive) accuracy: 0.949 in 26.68s ( 646 docs/s)
NB Multinomial classifier : 17257 train docs ( 2064 positive) 982 test docs ( 146 positive) accuracy: 0.913 in 26.74s ( 645 docs/s)
**Python source code:** :download:`plot_out_of_core_classification.py <plot_out_of_core_classification.py>`
.. literalinclude:: plot_out_of_core_classification.py
:lines: 25-
**Total running time of the example:** 29.40 seconds
( 0 minutes 29.40 seconds)