Hadoop Application Architectures

Name: Hadoop Application Architectures
Author: Mark Grover

Designing Real-World Big Data Applications

Specificaties

Paperback, 371 blz. | Engels

Wiley Computing | 1e druk, 2015

ISBN13: 9781491900086

Rubricering

Hoofdrubriek : Computer en informatica

Juridisch :

Wiley Computing 1e druk, 2015 9781491900086

Verwachte levertijd ongeveer 9 werkdagen

57,42

In winkelwagen

Samenvatting

Get expert guidance on architecting end-to-end data management solutions with Apache Hadoop. While many sources explain how to use various components in the Hadoop ecosystem, this practical book takes you through architectural considerations necessary to tie those components together into a complete tailored application, based on your particular use case.

To reinforce those lessons, the book’s second section provides detailed examples of architectures used in some of the most commonly found Hadoop applications. Whether you’re designing a new Hadoop application, or planning to integrate Hadoop into your existing data infrastructure, Hadoop Application Architectures will skillfully guide you through the process.

This book covers:
- Factors to consider when using Hadoop to store and model data
- Best practices for moving data in and out of the system
- Data processing frameworks, including MapReduce, Spark, and Hive
- Common Hadoop processing patterns, such as removing duplicate records and using windowing analytics
- Giraph, GraphX, and other tools for large graph processing on Hadoop
- Using workflow orchestration and scheduling tools such as Apache Oozie
- Near-real-time stream processing with Apache Storm, Apache Spark Streaming, and Apache Flume
- Architecture examples for clickstream analysis, fraud detection, and data warehousing

Specificaties

ISBN13:9781491900086

Trefwoorden:applicatieontwikkeling, Apache, Programmeertalen, Hadoop

Taal:Engels

Bindwijze:paperback

Aantal pagina's:371

Uitgever:Wiley Computing

Druk:1

Verschijningsdatum:30-4-2015

Hoofdrubriek:Computer en informatica

Inhoudsopgave

Part 1: Architectural Considerations for Hadoop Applications
1. Data Modeling in Hadoop
-Data Storage Options
-HDFS Schema Design
-HBase Schema Design
-Managing Metadata
-Conclusion

2. Data Movement
-Data Ingestion Considerations
-Data Ingestion Options
-Data Extraction
-Conclusion

3. Processing Data in Hadoop
-MapReduce
-Spark
-Abstractions
-Crunch
-Cascading
-Hive
-Impala
-Conclusion

4. Common Hadoop Processing Patterns
-Pattern: Removing Duplicate Records by Primary Key
-Pattern: Windowing Analysis
-Pattern: Time Series Modifications
-Conclusion

5. Graph Processing on Hadoop
-What Is a Graph?
-What Is Graph Processing?
-How Do You Process a Graph in a Distributed System?
-Giraph
-GraphX
-Which Tool to Use?
-Conclusion

6. Orchestration
-Why We Need Workflow Orchestration
-The Limits of Scripting
-The Enterprise Job Scheduler and Hadoop
-Orchestration Frameworks in the Hadoop Ecosystem
-Oozie Terminology
-Oozie Overview
-Oozie Workflow
-Workflow Patterns
-Parameterizing Workflows
-Classpath Definition
-Scheduling Patterns
-Executing Workflows
-Conclusion

7. Near-Real-Time Processing with Hadoop
-Stream Processing
-Apache Storm
-Trident
-Spark Streaming
-Flume Interceptors
-Which Tool to Use?
-Conclusion

Part 2: Case Studies
8. Clickstream Analysis
-Defining the Use Case
-Using Hadoop for Clickstream Analysis
-Design Overview
-Storage
-Ingestion
-Processing
-Analyzing
-Orchestration
-Conclusion

9. Fraud Detection
-Continuous Improvement
-Taking Action
-Architectural Requirements of Fraud Detection Systems
-Introducing Our Use Case
-High-Level Design
-Client Architecture
-Profile Storage and Retrieval
-Ingest
-Near-Real-Time and Exploratory Analytics
-Near-Real-Time Processing
-Exploratory Analytics
-What About Other Architectures?
-Conclusion

10. Data Warehouse
-Using Hadoop for Data Warehousing
-Defining the Use Case
-OLTP Schema
-Data Warehouse: Introduction and Terminology
-Data Warehousing with Hadoop
-High-Level Design
-Conclusion

Appendix A: Joins in Impala