hadoop多次格式化后，导致datanode启动不了-白红宇

hadoop多次格式化后，导致datanode启动不了

阅读量：4882 次

发布时间：2019-06-11

本文共 3879 字，大约阅读时间需要 12 分钟。

hadoop namenode -format

多次格式化后，datanode启动不了

org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /user/beifeng/core-site.xml._COPYING_ could only be replicated to 0 nodes instead of minReplication (=1).  There are 0 datanode(s) running and no node(s) are excluded in this operation.        at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget(BlockManager.java:1471)        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2791)        at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:606)        at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:455)        at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)        at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:585)        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:928)        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2013)        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2009)        at java.security.AccessController.doPrivileged(Native Method)        at javax.security.auth.Subject.doAs(Subject.java:415)        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2007)        at org.apache.hadoop.ipc.Client.call(Client.java:1411)        at org.apache.hadoop.ipc.Client.call(Client.java:1364)        at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)        at com.sun.proxy.$Proxy14.addBlock(Unknown Source)        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)        at java.lang.reflect.Method.invoke(Method.java:606)        at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)        at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)        at com.sun.proxy.$Proxy14.addBlock(Unknown Source)        at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.addBlock(ClientNamenodeProtocolTranslatorPB.java:368)        at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.locateFollowingBlock(DFSOutputStream.java:1449)        at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1270)        at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:526)put: File /user/beifeng/core-site.xml._COPYING_ could only be replicated to 0 nodes instead of minReplication (=1).  There are 0 datanode(s) running and no node(s) are excluded in this operation.

0.20.2版本解决方式:

1、查看名称节点上(即在配置文件中配置的hadoop.tmp.dir参数路径)/usr/hadoop0.20.2/hadoop-huanghz/dfs/name/current/文件夹中VERSION文件中的namespaceid；

2、在两个数据节点修改上dfs.data.dir配置的路径下current文件夹中的VERSION文件namespaceid与名称节点保持一致

2.20版本解决

/data/hadoop/dfs/name/current/VERSION

用name下面的clusterID，修改datanode的/data/hadoop/dfs/data/current/VERSION 里面的clusterID

每次格式化，name下面的VERSION的clusterID会产生一个新的ID，要去修改各个节点的VERSION的clusterID

……/dfs/data/current

storageID=DS-1959445666-10.161.138.100-50010-1386493413986

clusterID=CID-64a3a726-29e4-4d80-86a6-035ef33a225b

cTime=0

storageType=DATA_NODE