Hortonworks : Introduction to Apache NiFi

Nearly ten years ago, I was presented with an amazing opportunity. I was fortunate enough to join a team of three incredibly talented engineers to build a new platform. It would be responsible for handling the ever-increasing volumes of data that would be streamed through my organization. The platform would have to allow users to quickly add new sources of data on the fly and route, analyze, process, and transform the data, all before delivering it to its final destination. In short, the goal was to get the right data to the right place, in the right format and schema, at the right time.

Thus, began a long and engaging journey. Over the next year, I would take on more and more responsibilities, ultimately taking full responsibility for the development of the framework. The volume and variety of the data continued to increase. My organization decided that if the software were to be open sourced, it would not only benefit us but also many others who were on a similar journey. So, in November of 2014, this software was donated to the Apache Software Foundation and became known as Apache® NiFi™.

Since its debut, NiFi has been adopted by companies and organizations across every industry. The authors of this book have been with them through it all-training the users, assessing the strengths and weaknesses of the platform, and even getting their hands dirty to improve the code. They have seen a vast number of different use cases for NiFi across these industries and been active in the day-to-day use of the software to solve their critical dataflow problems. I have had the pleasure of working alongside some of the authors to tackle their most difficult dataflow challenges and learn from their experiences. Others have written of their experiences using NiFi and established that they have a great understanding of not just the software itself but also where it came from and where it is heading.

Hortonworks and Attunity have collaborated on an Apache NiFi For Dummies book. It is not intended to provide an in-depth understanding of every aspect of NiFi but rather is meant to provide an understanding of what NiFi is and explain when, how, and why to use NiFi. Additionally, it will explore the features that make the software unique. Through tutorials, examples, and explanations, it provides an excellent overview and walkthrough of NiFi that will benefit the uninitiated and experienced users alike.

While reading this book, you will gain a firm grasp on NiFi fundamentals and how to use the software. You should also be able to relate them to some of the challenges that you are facing and understand how NiFi can help to address them. Most importantly, I hope that you enjoy the read and that it encourages you to read more about NiFi and explore it on your own.

Cheers,
Mark Payne
Apache NiFi Committer and PMC Member, Sr. Member of Technical Staff, Hortonworks