lundi 26 décembre 2016

Top N frequent words in a file - using Parallel Stream

Bellow, the simplest way to get Top N frequent words in a file using Parallel Streams :

  package org.service ;

  import java.util.Map ;
  import java.util.List ;
  import java.util.Arrays ;
  import java.nio.file.Path ;
  import java.io.IOException ;
  import java.nio.file.Paths ;
  import java.nio.file.Files ;
  import static java.util.Comparator.comparing ;
  import static java.util.stream.Collectors.toList ;
  import static java.util.stream.Collectors.counting ;
  import static java.util.stream.Collectors.groupingBy ;

  /**
   *
   * @author ryahiaoui
  */
  
  public class Counter {
    
    
    private static List<String> getTopNFrequentWords( Path path, int rank ) throws IOException {
        
      return Files.lines(path)
                  .parallel()
                  .flatMap( line -> Arrays.asList ( line.split("\\b")).stream() )
                  .collect( groupingBy( w-> w, counting() ) )
                  .entrySet()
                  .stream()
                  .sorted( comparing ( Map.Entry< String, Long>::getValue).reversed() )
                  .limit( rank )
                  .map( Map.Entry::getKey )
                  .collect( toList() ) ;
    }
    
    public static void main(String[] args) throws IOException {
        
        // Top 10 most frequent words 
        
        String path = "/testFile.txt" ;
        
        List<String> mostFrequentWords = getTopNFrequentWords ( Paths.get(path), 10 ) ;
        
        System.out.println( " mostFrequentWords = " + mostFrequentWords ) ;
    }
    
  }




That's all :-)


Aucun commentaire:

Enregistrer un commentaire