Categories: Big Data

Big Data is NOT Just about Hadoop Stack Implementation

That is something any one can with a decent technical skill and Java experience could do it.

Big Data has lot to do with Data science. And, to stand out as a Big Data solution provider in the IT marketplace, one needs to have a team of Data scientist who work with technologist to implement Big data solution suggested by them.

Thus, following is how the Big Data team may look like?

  • Project/Delivery Manager
  • Data Scientist
  • Technical Architect (Hadoop)
  • Technical team including team/tech lead, developers, testers etc
  • Build/Configuration Engineer: This may be important owing to the Big Data typical cluster configurations requirement and the complexities surrounding it.

What is a Data Scientist?

Basically, a data scientist is someone who could extract meaning from data and create/propose new data products or enhancement to existing data products. Some of the pre-requisite skills for a data scientist are following:

  • Data Engineering
  • Pattern recognition and learning
  • Visualization
  • Data warehousing
  • Maths & Statistics

Take a look at picture below to get a visualization of skills.

Data Science

Thus, if you as a software provider have not yet started on this, its not too late at all. Plan to hire one or get a person in house to develop the skills on data science. Both may be harder to achieve as this job requirement is pretty new. However, the job need for a data scientist is only going to grow. So better late than never. Go and get one for your Big Data team to start with.

And one another thing. While talking to a new prospect on Big Data, get Data scientist involved as well along with a Hadoop technical architect. Chances of impressing the prospect and closing the deal will be higher.

Ajitesh Kumar

I have been recently working in the area of Data analytics including Data Science and Machine Learning / Deep Learning. I am also passionate about different technologies including programming languages such as Java/JEE, Javascript, Python, R, Julia, etc, and technologies such as Blockchain, mobile computing, cloud-native technologies, application security, cloud computing platforms, big data, etc. I would love to connect with you on Linkedin. Check out my latest book titled as First Principles Thinking: Building winning products using first principles thinking.

Recent Posts

Coefficient of Variation in Regression Modelling: Example

When building a regression model or performing regression analysis to predict a target variable, understanding…

1 week ago

Chunking Strategies for RAG with Examples

If you've built a "Naive" RAG pipeline, you've probably hit a wall. You've indexed your…

2 weeks ago

RAG Pipeline: 6 Steps for Creating Naive RAG App

If you're starting with large language models, you must have heard of RAG (Retrieval-Augmented Generation).…

2 weeks ago

Python: List Comprehension Explained with Examples

If you've spent any time with Python, you've likely heard the term "Pythonic." It refers…

3 weeks ago

Large Language Models (LLMs): Four Critical Modeling Stages

Large language models (LLMs) have fundamentally transformed our digital landscape, powering everything from chatbots and…

3 months ago

Agentic Workflow Design Patterns Explained with Examples

As Large Language Models (LLMs) evolve into autonomous agents, understanding agentic workflow design patterns has…

4 months ago