Over the past few years, Kafka has become the most exciting new addition in the big data distributed architecture. Originally developed at LinkedIn, its founders Jay Kreps, Jun Rao and Neha Narkhede have launched a company Confluent to develop its open-core business model. The software at its core, Apache Kafka reinvents the database log to provide a highly scalable and fault tolerant, high performance distributed system, which serves as the data pipeline backbone for stream data processing.
Eric Ries, a young American entrepreneur and ex-software engineer published “The Lean Startup” in 2011. The book is the collective wisdom of the valley and smart folks like Ries and Steve Blank about the purpose of a startup and the methods that help with it most efficiently. It’s because of the explosion of interest in the lean-startup methods that today concepts like “fail-fast”, “MVP”, “pivot” are commonplace. Coupled with lean-concepts and the experimentation afforded by agile methods, the lean-startup techniques improves the chances of success of a startup.
Scrum is the most popular agile framework out there. It uses 1 to 4 week iterations called sprints to deliver products. Developed by Jeff Sutherland and Ken Schwaber, it has been used since the nineties, i.e. even before the 2001 agile manifesto was written. The word scrum is taken from rugby.
Being a framework, scrum needs to be tailored to your situation. A scrum team is supposed to be cross-functional and self-organizing in order to minimize dependencies and wait times while improving collaboration.
DevOps is the new buzzword for IT. It arose out of the need to resolve issues with delivery of software as development processes adopted agile approaches. But what exactly is DevOps?
DevOps is a philosophy which advocates close collaboration between development (Dev) and IT Operations (Ops) with a goal of delivering value to the end-user with improved agility and quality. Historically developers and IT operations were separated in their own silos. While development focused on getting features out quickly, the IT operations team were focused on maintaining the stability and performance of the systems. With little visibility into each others’ work and conflicting / opposing priorities, there was little alignment on the context and the ultimate value provided to business. In the DevOps culture, there is increased collaboration and feedback, reduced silos and shared responsibility which improves outcomes. In essence, DevOps is the extension of agile values beyong development to IT Operations and collaborating together. Thus the portmanteau word: Development + IT Operations = DevOps
Lean thinking is based on eliminating waste in order to improve efficiency. Lean project management relies on lean thinking concepts for the elimination of waste. Due to its inception in the quality and productivity improvement processes in Japanese manufacturing, especially the Toyota Production System, the three main categories of waste (3M) have Japanese names: muda, mura and muri .
Muda - any activity that consumes resources but creates no value for the customer
