java 如何调用 hadoop分析

0 0

java 如何调用 hadoop分析

我想请教 java如何调用hadoop mapper/reduce?
我用的是mapper/reduce打成jar包，java项目用Runtime.getRuntime().exec调用shell脚本，在shell脚本中

  hadoop jar sas.jar $1 $2 $3

然后传递需要统计的参数，调用结束后，项目使用hadoop提供的工具类来访问统计结果

是这样吗？这样总有个问题就是调用hadoop分析时总有大量的socket未关闭，本来以为是hadoop版本的bug 但现在发现不是，请问大家是用什么方法解决这个问题的？

  java    32579 tasuser 4135u  IPv6   45885795               TCP racnode2:40331->racnode2:50010 (ESTABLISHED)
  

  java    32579 tasuser 4136u  IPv6   45886267               TCP racnode2:40339->racnode2:50010 (ESTABLISHED)
  

  java    32579 tasuser 4137u  IPv6   45886758               TCP racnode2:40347->racnode2:50010 (ESTABLISHED)
  

  java    32579 tasuser 4138u  IPv6   45887269               TCP racnode2:40356->racnode2:50010 (ESTABLISHED)
  

  java    32579 tasuser 4139u  IPv6   45887751               TCP racnode2:40364->racnode2:50010 (ESTABLISHED)
  

  java    32579 tasuser 4140u  IPv6   45888242               TCP racnode2:40372->racnode2:50010 (ESTABLISHED)
  

  java    32579 tasuser 4141u  IPv6   45888733               TCP racnode2:40380->racnode2:50010 (ESTABLISHED)
  

  java    32579 tasuser 4142u  IPv6   45889213               TCP racnode2:40388->racnode2:50010 (ESTABLISHED)
  

  java    32579 tasuser 4143u  IPv6   45889702               TCP racnode2:40396->racnode2:50010 (ESTABLISHED)
  

  java    32579 tasuser 4144u  IPv6   45890189               TCP racnode2:40405->racnode2:50010 (ESTABLISHED)
  

  java    32579 tasuser 4145u  IPv6   45890680               TCP racnode2:40413->racnode2:50010 (ESTABLISHED)
  

  java    32579 tasuser 4146u  IPv6   45891163               TCP racnode2:40421->racnode2:50010 (ESTABLISHED)
  

  java    32579 tasuser 4147u  IPv6   45891661               TCP racnode2:40431->racnode2:50010 (ESTABLISHED)
  

  java    32579 tasuser 4148u  IPv6   45892152               TCP racnode2:40439->racnode2:50010 (ESTABLISHED)
  

  java    32579 tasuser 4149u  IPv6   45892673               TCP racnode2:40448->racnode2:50010 (ESTABLISHED)
  

  java    32579 tasuser 4150u  IPv6   45893148               TCP racnode2:40457->racnode2:50010 (ESTABLISHED)
  

  java    32579 tasuser 4151u  IPv6   45893645               TCP racnode2:40466->racnode2:50010 (ESTABLISHED)
  

  java    32579 tasuser 4152u  IPv6   45894136               TCP racnode2:17485->racnode2:50010 (ESTABLISHED)
  

  java    32579 tasuser 4153u  IPv6   45906919               TCP racnode2:33024->racnode2:50010 (ESTABLISHED)
  

  java    32579 tasuser 4154u  IPv6   45908865               TCP racnode2:33059->racnode2:50010 (ESTABLISHED)
  

  java    32579 tasuser 4155u  IPv6   45909359               TCP racnode2:33067->racnode2:50010 (ESTABLISHED)
  

  java    32579 tasuser 4156u  IPv6   45909884               TCP racnode2:33075->racnode2:50010 (ESTABLISHED)
  

  java    32579 tasuser 4157u  IPv6   45910371               TCP racnode2:33085->racnode2:50010 (ESTABLISHED)
  

  java    32579 tasuser 4158u  IPv6   45910868               TCP racnode2:33096->racnode2:50010 (ESTABLISHED)
  

  java    32579 tasuser 4159u  IPv6   45907402               TCP racnode2:33032->racnode2:50010 (ESTABLISHED)
  

  java    32579 tasuser 4160u  IPv6   45911335               TCP racnode2:33105->racnode2:50010 (ESTABLISHED)
  

  java    32579 tasuser 4161u  IPv6   45907889               TCP racnode2:33042->racnode2:50010 (ESTABLISHED)
  

  java    32579 tasuser 4162u  IPv6   45908385               TCP racnode2:33050->racnode2:50010 (ESTABLISHED)
  

  java    32579 tasuser 4163u  IPv6   45911831               TCP racnode2:33113->racnode2:50010 (ESTABLISHED)
  

  java    32579 tasuser 4164u  IPv6   45912317               TCP racnode2:33121->racnode2:50010 (ESTABLISHED)
  

  java    32579 tasuser 4165u  IPv6   45912804               TCP racnode2:33131->racnode2:50010 (ESTABLISHED)
  

  java    32579 tasuser 4166u  IPv6   45913284               TCP racnode2:33142->racnode2:50010 (ESTABLISHED)
  

  java    32579 tasuser 4167u  IPv6   45913799               TCP racnode2:33151->racnode2:50010 (ESTABLISHED)
  

  java    32579 tasuser 4168u  IPv6   45914286               TCP racnode2:33162->racnode2:50010 (ESTABLISHED)
  

  java    32579 tasuser 4169u  IPv6   45914757               TCP racnode2:33173->racnode2:50010 (ESTABLISHED)
  

  java    32579 tasuser 4170u  IPv6   45915266               TCP racnode2:33181->racnode2:50010 (ESTABLISHED)
  

  java    32579 tasuser 4171u  IPv6   45915750               TCP racnode2:33192->racnode2:50010 (ESTABLISHED)
  

  java    32579 tasuser 4172u  IPv6   45916235               TCP racnode2:33203->racnode2:50010 (ESTABLISHED)
  

  java    32579 tasuser 4173u  IPv6   45916713               TCP racnode2:33213->racnode2:50010 (ESTABLISHED)
  

  java    32579 tasuser 4174u  IPv6   45917477               TCP racnode2:33225->racnode2:50010 (ESTABLISHED)
  

  java    32579 tasuser 4175u  IPv6   45918926               TCP racnode2:33258->racnode2:50010 (ESTABLISHED)
  

  java    32579 tasuser 4176u  IPv6   45919412               TCP racnode2:33266->racnode2:50010 (ESTABLISHED)
  

  java    32579 tasuser 4177u  IPv6   45919910               TCP racnode2:33278->racnode2:50010 (ESTABLISHED)
  

  java    32579 tasuser 4178u  IPv6   45920406               TCP racnode2:33287->racnode2:50010 (ESTABLISHED)
  

  java    32579 tasuser 4180u  IPv6   45917963               TCP racnode2:33237->racnode2:50010 (ESTABLISHED)
  

  java    32579 tasuser 4182u  IPv6   45918459               TCP racnode2:33247->racnode2:50010 (ESTABLISHED)

hadoop Linux

12 years, 6 months ago

LoveS

LoveS 12 years, 6 months ago

java 如何调用 hadoop分析

LoveS

Answers

Your Answer