java - How to Sort Reducer Output? -


i want sort output of reducer. sample of reducer output shown below:

0,0    2.5 0,1    3.0 1,0    4.0 1,1    1.5 

the reducer output sorted first element of key. wanted sort second element of key output this:

0,0    2.5 1,0    4.0 0,1    3.0 1,1    1.5 

any way can this?

please help!

this reducer:

import java.io.bufferedreader; import java.io.ioexception; import java.io.inputstreamreader; import java.util.hashmap; import org.apache.hadoop.fs.filesystem; import org.apache.hadoop.fs.path; import org.apache.hadoop.io.text; import org.apache.hadoop.mapreduce.reducer;   public class recreduce extends reducer<text, text, text, text> {     public static int n=0;     @override     public void setup(context context) throws ioexception, interruptedexception{         filesystem hdfs= filesystem.get(context.getconfiguration());         bufferedreader br = new bufferedreader(new inputstreamreader(hdfs.open(new path(context.getconfiguration().get("outfile")))));         string line=null;         while((line=br.readline())!=null){             n=integer.parseint(line);             break;         }         br.close();         hdfs.close();     }     public void reduce(text key, iterable<text> values, context context)             throws ioexception, interruptedexception {         string[] value;         hashmap<integer, float> hasha = new hashmap<integer, float>();         hashmap<integer, float> hashb = new hashmap<integer, float>();         (text val : values) {             value = val.tostring().split(",");             if (value[0].equals("a")) {                 for(int z=1;z<=n;z++){                     hasha.put(z, float.parsefloat(value[z]));}             } else{                 for(int a=1;a<=n;a++){                     hashb.put(a, float.parsefloat(value[a]));}             }         }         float result = 0.0f;         float a_ij;         float b_jk;         (int j=1;j<=n;j++) {             a_ij = hasha.containskey(j) ? hasha.get(j) : 0.0f;             b_jk = hashb.containskey(j) ? hashb.get(j) : 0.0f;             result +=a_ij*b_jk;         }         context.write(null, new text(key.tostring() + "," + float.tostring(result)));     } } 

you can use composite key , composite key comparator

create class e.g.

class pair(){     string key     string value; } 

and use in reducer output this

context.write(new pair(key.tostring(), float.tostring(result)), null);

then create comparator

public class paircomparator extends writablecomparator {     protected paircomparator() {         super(pair.class, true);     }        @override     public int compare(writablecomparable w1, writablecomparable w2) {         pair k1 = (pair)w1;         pair k2 = (pair)w2;                  return  k1.getvalue().compareto(k2.getvalue());     } } 

and use comparator in job definition
job.setsortcomparatorclass(paircomparator.class);

i didn't check code above. idea.

i hope


Comments

Popular posts from this blog

How has firefox/gecko HTML+CSS rendering changed in version 38? -

javascript - Complex json ng-repeat -

jquery - Cloning of rows and columns from the old table into the new with colSpan and rowSpan -