Apache Hadoop Ecosystem

Apache Hadoop Ecosystem

  • Hadoop HDFS - 2007 - A distributed file system for reliably storing huge amounts of unstructured, semi-structured and structured data in the form of files. 
  • Hadoop MapReduce - 2007 - A distributed algorithm framework for the parallel processing of large datasets on HDFS filesystem. It runs on Hadoop cluster but also supports other database formats like Cassandra and HBase. 
  • Cassandra - 2008 - A key-value pair NoSQL database, with column family data representation and asynchronous masterless replication. 
  • HBase - 2008 - A key-value pair NoSQL database, with column family data representation, with master-slave replication. It uses HDFS as underlying storage. 
  • Zookeeper - 2008 - A distributed coordination service for distributed applications. It is based on Paxos algorithm variant called Zab. 
  • Pig - 2009 - Pig is a scripting interface over MapReduce for developers who prefer scripting interface over native Java MapReduce programming. 
  • Hive - 2009 - Hive is a SQL interface over MapReduce for developers and analysts who prefer SQL interface over native Java MapReduce programming. 
  • Mahout - 2009 - A library of machine learning algorithms, implemented on top of MapReduce, for finding meaningful patterns in HDFS datasets. 
  • Sqoop - 2010 - A tool to import data from RDBMS/DataWarehouse into HDFS/HBase and export back. 
  • YARN - 2011 - A system to schedule applications and services on an HDFS cluster and manage the cluster resources like memory and CPU. 
  • Flume - 2011 - A tool to collect, aggregate, reliably move and ingest large amounts of data into HDFS. 
  • Storm - 2011 - A system to process high-velocity streaming data with 'at least once' message semantics. 
  • Spark - 2012 - An in-memory data processing engine that can run a DAG of operations. It provides libraries for Machine Learning, SQL interface and near real-time Stream Processing. 
  • Kafka - 2012 - A distributed messaging system with partitioned topics for very high scalability. 
  • SolrCloud - 2012 - A distributed search engine with a REST-like interface for full-text search. It uses Lucene library for data indexing.

Comments

  1. for ict 9925 March 2018 at 08:00

    great

    ReplyDelete
  2. Malar Pretty10 July 2018 at 22:46

    This technical post helps me to improve my skills ,thanks for this wonder post I expect your upcoming blog, so keep sharing...
    Articles
    Technology updates

    ReplyDelete
  3. Anonymous20 July 2018 at 00:58

    Best tutorial on hadoop for freshers among lots of available. Super work
    Hadoop Training in Chennai

    ReplyDelete
  4. Vicky Ram20 July 2018 at 05:32

    Thanks for sharing your knowledge with us .This will absolutely going to help me in my future .

    Big Data Training Chennai

    Best hadoop training institute in chennai

    ReplyDelete
  5. Staarwd4 August 2018 at 22:38

    It's event-driven, and builders not should depend on the ops to check their code. They'll shortly run, check and deploy their code with out getting tangled within the conventional workflow.This is great blog. If you want to know more about this visit here Internet of Things.

    ReplyDelete
  6. pooja25 August 2018 at 02:56

    Your good knowledge and kindness in playing with all the pieces were very useful. I don’t know what I would have done if I had not encountered such a step like this.
    Online training in USA

    ReplyDelete
  7. sai25 August 2018 at 04:05

    Those guidelines additionally worked to become a good way to recognize that other people online have the identical fervor like mine to grasp great deal more around this condition.
    Click here:
    Online training in USA

    ReplyDelete
  8. pooja25 August 2018 at 05:19

    Thank you a lot for providing individuals with a very spectacular possibility to read critical reviews from this site.

    Online training in USA

    ReplyDelete
  9. genga g25 August 2018 at 07:00

    I feel really happy to have seen your webpage and look forward to so many more entertaining times reading here. Thanks once more for all the details.Online training in USA

    ReplyDelete
  10. Unknown28 September 2018 at 06:58

    It's interesting that many of the bloggers to helped clarify a few things for me as well as giving.Most of ideas can be nice content.The people to give them a good shake to get your point and across the command.
    aws training in chennai

    hadoop training in chennai

    ReplyDelete
  11. Vicky Ram4 October 2018 at 05:35

    Wonderful post!! Thanks for sharing this blog .

    Simple Truth

    Article submission sites

    ReplyDelete
  12. Vicky Ram12 November 2018 at 04:45

    Thanks for your sharing such a useful information. this was really helpful to me

    Education
    Technology

    ReplyDelete
  13. Madhu Bala23 December 2018 at 05:21



    Impressive content, keep doing more.
    JAVA Training in Chennai
    Hadoop Training in Chennai

    ReplyDelete
  14. velraj25 June 2019 at 05:58

    I am feeling happy to read this. You gave nice info to me. Please update more.
    Ethical Hacking course in Chennai
    Ethical Hacking Training in Chennai
    Hacking course in Chennai
    ccna course in Chennai
    Salesforce Training in Chennai
    AngularJS Training in Chennai
    PHP Training in Chennai
    Ethical Hacking course in Tambaram
    Ethical Hacking course in Velachery
    Ethical Hacking course in T Nagar

    ReplyDelete
  15. Balaji26 June 2019 at 23:38

    I am glad that I have visited your blog, really amazing. Waiting for further updates.

    Blue Prism Training in Chennai
    DevOps Training in Chennai
    MVC Training in Chennai

    ReplyDelete
  16. Aparna28 June 2019 at 03:52

    Innovative blog...!!! This is the best post and I got more ideas from your post. Keep continuous....
    JMeter Training in Chennai
    JMeter Training
    Power BI Training in Chennai
    Job Openings in Chennai
    Linux Training in Chennai
    Tableau Training in Chennai
    Oracle Training in Chennai
    Oracle DBA Training in Chennai
    JMeter Training in Velachery
    JMeter Training in Vadapalani

    ReplyDelete
  17. easylearn22 August 2019 at 02:30

    Hi, thank you very much for new information, i learned something new. Very well written.It was so good to read and usefull to improve knowledge.Keep posting. If you are looking for any big data hadoop related information please visit our website.
    big data hadoop training in bangalore.

    ReplyDelete
  18. Rakesh17 September 2019 at 04:29

    Fabulous post admin, it was too good and helpful. Waiting for more updates.
    Tally course in Chennai
    Tally Training in Chennai
    Tally institute in Chennai

    ReplyDelete
  19. Laura Bush5 October 2019 at 03:39

    asdfgh

    ReplyDelete
  20. Malcom Marshall21 January 2020 at 06:42

    This comment has been removed by the author.

    ReplyDelete
  21. rainbowr10 March 2020 at 04:05

    good blog

    Spark and Scala Online Training

    ReplyDelete
  22. jamuna30 March 2020 at 01:54

    This is good information and really helpful for the people who need information about this.
    German Classes in Chennai
    german language course
    best spoken english institute in chennai
    Japanese Language Course in Chennai
    top 10 ielts coaching in chennai
    ielts exam coaching centre in chennai
    Spoken English in Chennai
    TOEFL Training in Chennai
    spanish courses in chennai
    content writing course in chennai
    German Classes in Adyar
    German Classes in T Nagar

    ReplyDelete
  23. bill.wood12 July 2020 at 13:44

    Limiting the number of questions was not appealing because it made the sampling small and coverage uneven while placing more weight on the few remaining questions. machine learning training in hyderabad

    ReplyDelete
  24. siva30 December 2020 at 21:58

    Excellent Blog!!! Waiting for your new blog... thanks for sharing with us.
    salesforce architect certification
    interview techniques
    career after bsc
    tools used in data science
    oracle interview questions and answers
    pega interview questions for experienced

    ReplyDelete
  25. Anonymous21 February 2021 at 22:10

    Nice article please do visit my website for Bigdata Hadoop online training

    ReplyDelete
  26. Huongkv23 March 2021 at 01:06

    Aivivu - đại lý chuyên vé máy bay trong nước và quốc tế

    vé máy bay vietjet từ hàn quốc về việt nam

    vé máy bay thanh hóa sài gòn giá rẻ

    vé máy bay sg đi hà nội

    vé máy bay đi huế vietnam airlines

    vé quy nhơn

    ReplyDelete
  27. Royal Online Book13 May 2021 at 03:06

    Wonderful article, Which you have shared about the service. Your article is very important and I really enjoyed reading it. Get for more information online tennis betting sites

    ReplyDelete
  28. senthilpraveen5 July 2021 at 05:05

    Sometimes blogs were goes away from the topic what actually mentioned. But this is not like that. Thanks for sharing this.
    AWS Course in Chennai
    DevOps Certification in Chennai

    ReplyDelete
  29. Unknown18 July 2021 at 06:24

    Nice article eauctionsindia

    ReplyDelete
  30. Priya Rathod6 August 2021 at 00:50

    Very interesting blog. A lot of the blogs I see these days don't provide anything that interests me, but I'm really interested in this one. I just thought I would post and let you know.
    AWS Training in Hyderabad
    AWS Course in Hyderabad

    ReplyDelete
  31. Small Paw6 February 2022 at 06:05

    Hi, I do believe this is a great website. I stumbledupon it ;) I am going to revisit yet
    again since I book-marked it. Money and freedom is the greatest way to
    change, may you be rich and continue to help other people.
    teacup havanese puppies for sale

    https://thegorgeousdoodles.com/
    https://www.fluffyhavanese.com/
    https://www.pomeranianpuppiesforsales.com/
    https://thegorgeousragdolls.com/

    ReplyDelete
  32. preety17 February 2022 at 01:26

    Thank you for sharing valuable Content. Really amazing information.here also we can get more information about Best oracle fusion Courses online Training.

    Soft Online Training offers

    Oracle Fusion SCM Online Training

    Oracle Fusion HCM Online Training

    Oracle Fusion Financials Online Training

    Oracle Fusion Technical Online Training

    Oracle Fusion PPM Online Training

    Oracle Integration Cloud Online Training

    ReplyDelete
  33. Harshan17 March 2022 at 03:19


    Useful blog, it is very impressive.

    How JMeter is Used for Performance Testing
    Why JMeter for Performance Testing

    ReplyDelete
  34. Jeff....26 October 2023 at 21:58

    This comment has been removed by the author.

    ReplyDelete
  35. Jeff....26 October 2023 at 22:02

    This comment has been removed by the author.

    ReplyDelete

Post a Comment

Popular posts from this blog

Big Data After The Internet

Image
Till 1995 most of the people did not know about the internet. It was hard to use, till the Netscape browser arrived and its famous IPO happened. The arrival of Netscape meant anyone could create material and anyone with a connection could view it. Internet's popularity resulted in mushrooming of websites like AOL, MSN, Yahoo, CNN, Napster and so many more. They  provided free information sharing services like emails, chats, photograph sharing, video sharing, blogging, news, weather, music, games etc. These sites were generating, collecting and sharing an enormous amount of data, for the people all over the globe. There were, of course, new generation e-commerce companies like Amazon and eBay that also contributed to the overall information available, but sharing of information was not at the core of their strategy.   Why this phenomenon of information sharing noteworthy? There are two good reasons:  The data on the Internet was freely available to everyone on the Interne
Read more