分区排序案例
2019-01-23 本文已影响0人
bullion
1）在全排序的基础上实现
2）
ProvincePartitioner
public class ProvincePartitioner extends Partitioner<FlowBean, Text> {
@Override
public int getPartition(Text key, FlowBean value, int numPartitions) {
//按照手机号前三位分区
String prePhoneNum = key.toString().substring(0, 3);
int partition = 4;
if ("136".equals(prePhoneNum)) {
partition = 0;
} else if ("137".equals(prePhoneNum)) {
partition = 1;
} else if ("138".equals(prePhoneNum)) {
partition = 2;
} else if ("139".equals(prePhoneNum)) {
partition = 3;
}
return partition;
}
}
3）在FlowCountSortDriver中增加代码
job.setPartitionerClass(ProvincePartitioner.class);
job.setNumReduceTasks(5);
