Hadoop HDFS BIG Data installations and setup, Online Trainings, Certification, Pricing, Downloads, Blogs, Interview Questions, projects and jobs..

HDFS Architecture and Design

(Note: This post is mainly focused on the platform level architecture, if you are looking for Hadoop BIG Data Application level architecture visit : BIG Data Hadoop Spark Application level Architecture.)

HDFS has a master and slave architecture. HDFS (Hadoop Distributed File System) is designed to run on commodity hardware. These commodity hardware providers can be Dell, HP, IBM, Huawei and others. Though there are similarities between HDFS and other distributed file systems, the unique differences making HDFS a market leader.

Fault-tolerant:


HDFS is highly fault-tolerant and is designed to be deployed on multiple nodes so that if one node fails the other node is available.
HDFS is good for applications that have large data sets, the throughput is very high in HDFS design. With some additional setup, HDFS can accommodate live streaming ability and real time data processing.

NameNode:


It is repository of HDFS metadata. The user data never flows through the NameNode, this design protects the metadata without corrupting the Name node. The NameNode acts as an arbitrator. The NameNode executes file system operations such as opening, closing and renaming of files and directories. HDFS cluster consists of a single NameNode, a master server that manages file system. Also, the mapping of blocks to DataNodes is determined by the NameNode.

...Click here for full details

Python for BEGINNERS and DUMMIES - INDEX


Python Variable, Expression and Statement


List of 'Reserved Words' in Python


Boolean Expressions and Comparison Operators in Python

Python Interview Questions and Answers - INDEX


What are the differences between Python 2.x vs Python 3.x (2.7 vs 3.3, 3.4, 3.5, 3.6, 3.7, 3.8, 3.9), when to you which version ?


What is Python Variable, Expression and Statement ?


What is a 'Reserved Word' in Python ? List of Reserved words in Python ?

How to Install Python, IDE, UI to Practice


Download and Install Python from Python.org


Download and Install Anaconda to Use SPIDER and Jupyter Notebook