Categories: Big Data

Big Data is NOT Just about Hadoop Stack Implementation

That is something any one can with a decent technical skill and Java experience could do it.

Big Data has lot to do with Data science. And, to stand out as a Big Data solution provider in the IT marketplace, one needs to have a team of Data scientist who work with technologist to implement Big data solution suggested by them.

Thus, following is how the Big Data team may look like?

  • Project/Delivery Manager
  • Data Scientist
  • Technical Architect (Hadoop)
  • Technical team including team/tech lead, developers, testers etc
  • Build/Configuration Engineer: This may be important owing to the Big Data typical cluster configurations requirement and the complexities surrounding it.

What is a Data Scientist?

Basically, a data scientist is someone who could extract meaning from data and create/propose new data products or enhancement to existing data products. Some of the pre-requisite skills for a data scientist are following:

  • Data Engineering
  • Pattern recognition and learning
  • Visualization
  • Data warehousing
  • Maths & Statistics

Take a look at picture below to get a visualization of skills.

Data Science

Thus, if you as a software provider have not yet started on this, its not too late at all. Plan to hire one or get a person in house to develop the skills on data science. Both may be harder to achieve as this job requirement is pretty new. However, the job need for a data scientist is only going to grow. So better late than never. Go and get one for your Big Data team to start with.

And one another thing. While talking to a new prospect on Big Data, get Data scientist involved as well along with a Hadoop technical architect. Chances of impressing the prospect and closing the deal will be higher.

Ajitesh Kumar

I have been recently working in the area of Data analytics including Data Science and Machine Learning / Deep Learning. I am also passionate about different technologies including programming languages such as Java/JEE, Javascript, Python, R, Julia, etc, and technologies such as Blockchain, mobile computing, cloud-native technologies, application security, cloud computing platforms, big data, etc. I would love to connect with you on Linkedin. Check out my latest book titled as First Principles Thinking: Building winning products using first principles thinking.

Recent Posts

Retrieval Augmented Generation (RAG) & LLM: Examples

Last updated: 25th Jan, 2025 Have you ever wondered how to seamlessly integrate the vast…

6 days ago

How to Setup MEAN App with LangChain.js

Hey there! As I venture into building agentic MEAN apps with LangChain.js, I wanted to…

2 weeks ago

Build AI Chatbots for SAAS Using LLMs, RAG, Multi-Agent Frameworks

Software-as-a-Service (SaaS) providers have long relied on traditional chatbot solutions like AWS Lex and Google…

2 weeks ago

Creating a RAG Application Using LangGraph: Example Code

Retrieval-Augmented Generation (RAG) is an innovative generative AI method that combines retrieval-based search with large…

3 weeks ago

Building a RAG Application with LangChain: Example Code

The combination of Retrieval-Augmented Generation (RAG) and powerful language models enables the development of sophisticated…

3 weeks ago

Building an OpenAI Chatbot with LangChain

Have you ever wondered how to use OpenAI APIs to create custom chatbots? With advancements…

3 weeks ago