Hadoop - No data nodes are reported at Yarn / Resource Manager


If you look at the Resource Manager web page or running "yarn node -list" and you see no data nodes being reported

For example

yarn node -list
20/01/01 11:13:12 INFO client.RMProxy: Connecting to ResourceManager at hadoopNM/112.44.1.23:8032
Total Nodes:0
Node-Id Node-State Node-Http-Address Number-of-Running-Containers


If you are certain that your yarn-sites.xml is set up properly with "yarn.resourcemanager.address" pointing to namenode_hostname:port, you may want to take a look at the datanode log. That is a good starting place to find issue. For this example, we find the following error


org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exit code from container executor initialization is : 24
org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.privileged.PrivilegedOperationException: ExitCodeException exitCode=24: File /hadoop-path/container-executor.cfg must not be world or group writable, but is 664



Fixing the permission of that file and restart the cluster addressed the issue.

Comments

Popular Posts