hadoop 操作

hadoop fs -ls /   看根目录下的所有文件

显示HDFS块信息

:~$ hdfs fsck / -files
Connecting to namenode via http://node-master:9870/fsck?ugi=hadoop&files=1&path=%2F
FSCK started by hadoop (auth:SIMPLE) from /192.168.56.2 for path / at Sun Oct 27 17:37:28 AEDT 2019
/ <dir>
/alice.txt 173595 bytes, replicated: replication=2, 1 block(s):  OK
/readme.txt 152 bytes, replicated: replication=2, 1 block(s):  OK

Status: HEALTHY
 Number of data-nodes:    3
 Number of racks:        1
 Total dirs:            1
 Total symlinks:        0

Replicated Blocks:
 Total size:    173747 B
 Total files:    2
 Total blocks (validated):    2 (avg. block size 86873 B)
 Minimally replicated blocks:    2 (100.0 %)
 Over-replicated blocks:    0 (0.0 %)
 Under-replicated blocks:    0 (0.0 %)
 Mis-replicated blocks:        0 (0.0 %)
 Default replication factor:    2
 Average block replication:    2.0
 Missing blocks:        0
 Corrupt blocks:        0
 Missing replicas:        0 (0.0 %)

Erasure Coded Block Groups:
 Total size:    0 B
 Total files:    0
 Total block groups (validated):    0
 Minimally erasure-coded block groups:    0
 Over-erasure-coded block groups:    0
 Under-erasure-coded block groups:    0
 Unsatisfactory placement block groups:    0
 Average block group size:    0.0
 Missing block groups:        0
 Corrupt block groups:        0
 Missing internal blocks:    0
FSCK ended at Sun Oct 27 17:37:28 AEDT 2019 in 3 milliseconds


The filesystem under path ‘/‘ is HEALTHY

:~$ hdfs fsck / -blocks
Connecting to namenode via http://node-master:9870/fsck?ugi=hadoop&blocks=1&path=%2F
FSCK started by hadoop (auth:SIMPLE) from /192.168.56.2 for path / at Sun Oct 27 17:40:29 AEDT 2019

Status: HEALTHY
 Number of data-nodes:    3
 Number of racks:        1
 Total dirs:            1
 Total symlinks:        0

Replicated Blocks:
 Total size:    173747 B
 Total files:    2
 Total blocks (validated):    2 (avg. block size 86873 B)
 Minimally replicated blocks:    2 (100.0 %)
 Over-replicated blocks:    0 (0.0 %)
 Under-replicated blocks:    0 (0.0 %)
 Mis-replicated blocks:        0 (0.0 %)
 Default replication factor:    2
 Average block replication:    2.0
 Missing blocks:        0
 Corrupt blocks:        0
 Missing replicas:        0 (0.0 %)

Erasure Coded Block Groups:
 Total size:    0 B
 Total files:    0
 Total block groups (validated):    0
 Minimally erasure-coded block groups:    0
 Over-erasure-coded block groups:    0
 Under-erasure-coded block groups:    0
 Unsatisfactory placement block groups:    0
 Average block group size:    0.0
 Missing block groups:        0
 Corrupt block groups:        0
 Missing internal blocks:    0
FSCK ended at Sun Oct 27 17:40:29 AEDT 2019 in 4 milliseconds


The filesystem under path ‘/‘ is HEALTHY

:~$ hdfs fsck /readme.txt -files
Connecting to namenode via http://node-master:9870/fsck?ugi=hadoop&files=1&path=%2Freadme.txt
FSCK started by hadoop (auth:SIMPLE) from /192.168.56.2 for path /readme.txt at Sun Oct 27 17:43:58 AEDT 2019
/readme.txt 152 bytes, replicated: replication=2, 1 block(s):  OK

Status: HEALTHY
 Number of data-nodes:    3
 Number of racks:        1
 Total dirs:            0
 Total symlinks:        0

Replicated Blocks:
 Total size:    152 B
 Total files:    1
 Total blocks (validated):    1 (avg. block size 152 B)
 Minimally replicated blocks:    1 (100.0 %)
 Over-replicated blocks:    0 (0.0 %)
 Under-replicated blocks:    0 (0.0 %)
 Mis-replicated blocks:        0 (0.0 %)
 Default replication factor:    2
 Average block replication:    2.0
 Missing blocks:        0
 Corrupt blocks:        0
 Missing replicas:        0 (0.0 %)

Erasure Coded Block Groups:
 Total size:    0 B
 Total files:    0
 Total block groups (validated):    0
 Minimally erasure-coded block groups:    0
 Over-erasure-coded block groups:    0
 Under-erasure-coded block groups:    0
 Unsatisfactory placement block groups:    0
 Average block group size:    0.0
 Missing block groups:        0
 Corrupt block groups:        0
 Missing internal blocks:    0
FSCK ended at Sun Oct 27 17:43:58 AEDT 2019 in 1 milliseconds


The filesystem under path ‘/readme.txt‘ is HEALTHY