Hadoop
Hadoop is a batch processing system for a cluster of nodes that gives the bases of the biggest Data analytic activities because it bundles two sets of functionality, most wanted to deal with huge unstructured datasets i.e Distributed file systems and MapReduce processing.
- It is a project from the Apache Software Foundation written in Java to assist data-intensive distributed applications.
- Hadoop allows applications to operate with thousands of nodes and petabytes of data.
- The incentive originates from Google’s MapReduce and Google File System papers.
- Hadoop’s biggest contributor has been the search giant Yahoo, where it is widely utilized across the business platform.
Map Reduce
Hadoop MapReduce is a programming model and software structure for writing applications that quickly make large amounts of data in parallel on big clusters of computer nodes. MapReduce uses the HDFS to access file parts and to save reduced results.
HDFS
Hadoop Distributed File System (HDFS) is the initial storage system handled by Hadoop applications. HDFS is, as its name implies, a distributed file system that gives high throughput access to application data creating multiple copies of data blocks and sharing them on computer nodes during a cluster to enable reliable and fast computations.
Architecture of Hadoop
- Hadoop is a Map/Reduce framework that works on HDFS or HBase.
- The central idea is to decompose a task into many identical tasks that can be executed closer to the data.
- Also, all tasks are parallelized: the Map phase. Then all these intermediate results are joined into one result: the Reduce phase.
- In Hadoop, The JobTracker is responsible for regulating the job, maintaining the Map/Reduce phase, retrying in case of failures.
- The TaskTrackers (Java process) are running on different DataNodes. Each Task Tracker performs the tasks of the job on the locally saved data.
If you have skills in PHP programming and you want to enhance your career in this field, a PHP certification from StudySection can help you reach your desired goals. Both beginner level and expert level PHP Certification Exams are offered by StudySection along with other programming certification exams.
