An efficient means to retrieve count summarizations of large data sets.

A count or summation can tell you a lot about particular fields of data, or your data as a whole. Hourly ingest record counts can be post processed to generate helpful histograms. This can be executed in a simple “word count” manner, in that for each input record, you output the same key, say the hour of data being processed, and a count of 1. The single reduce will sum all the input values and output the final record count with the hour. This works very well, but it can be done more efficiently using counters. Instead of writing any key value pairs at all, simply use the framework’s counting mechanism to keep track of the number of input records. This requires no reduce phase and no summation! The framework handles monitoring the names of the counters and their associated ....