hadoop 产生大量句柄导致系统报错

0 0

hadoop 产生大量句柄导致系统报错

项目部署在linux服务器上，运行hadoop分析任务，每运行一段时间就报错，后来发现错误是：

  java.io.IOException: error=24, Too many open files
  

  at java.lang.ProcessBuilder.start(ProcessBuilder.java:460)
  

  at java.lang.Runtime.exec(Runtime.java:593)
  

  at java.lang.Runtime.exec(Runtime.java:431)
  

  at java.lang.Runtime.exec(Runtime.java:328)
  

  at com.technicolor.tas.timer.sas.AbOperateSAS.exec(AbOperateSAS.java:145)
  

  at com.technicolor.tas.timer.sas.AbOperateSAS.deal(AbOperateSAS.java:71)
  

  at

这说明是linux系统文件句柄数开的太多，查看了一下是1024，网上说是要增加句丙数但这治标不治本
后来查看系统句柄发现有大量下面的句柄：

  java    26014 tasuser  819u  IPv6           38700078                TCP 10.9.0.12:56192->10.9.0.12:50010 (CLOSE_WAIT)
  

  java    26014 tasuser  820u  IPv6           38700937                TCP 10.9.0.12:56229->10.9.0.12:50010 (CLOSE_WAIT)
  

  java    26014 tasuser  821u  IPv6           38701836                TCP 10.9.0.12:56266->10.9.0.12:50010 (CLOSE_WAIT)
  

  java    26014 tasuser  822u  IPv6           38702699                TCP 10.9.0.12:56303->10.9.0.12:50010 (CLOSE_WAIT)
  

  java    26014 tasuser  823u  IPv6           38703619                TCP 10.9.0.12:56344->10.9.0.12:50010 (CLOSE_WAIT)
  

  java    26014 tasuser  824u  IPv6           38704533                TCP 10.9.0.12:56381->10.9.0.12:50010 (CLOSE_WAIT)
  

  java    26014 tasuser  825u  IPv6           38705392                TCP 10.9.0.12:56419->10.9.0.12:50010 (CLOSE_WAIT)
  

  java    26014 tasuser  826u  IPv6           38706274                TCP 10.9.0.12:56456->10.9.0.12:50010 (CLOSE_WAIT)
  

  java    26014 tasuser  827u  IPv6           38707135                TCP 10.9.0.12:56497->10.9.0.12:50010 (CLOSE_WAIT)
  

  java    26014 tasuser  828u  IPv6           38708040                TCP 10.9.0.12:33196->10.9.0.12:50010 (CLOSE_WAIT)
  

  java    26014 tasuser  829u  IPv6           38708927                TCP 10.9.0.12:33233->10.9.0.12:50010 (CLOSE_WAIT)
  

  java    26014 tasuser  830u  IPv6           38709780                TCP 10.9.0.12:33270->10.9.0.12:50010 (CLOSE_WAIT)
  

  java    26014 tasuser  831u  IPv6           38710709                TCP 10.9.0.12:33307->10.9.0.12:50010 (CLOSE_WAIT)

这应该是项目调用shell脚本进行hadoop分析后留下的，请问有什么好的方法关闭已经用过的句柄？

hadoop Linux

11 years, 4 months ago