Category Archives: Big Data
Hadoop Map-Reduce Described With Example
I came across a great page describing Hadoop map-reduce and HDFS architecture. The page presents some of the following details: HDFS responsibilities and execution flows Key characteristics of Map-Reduce lifecycle A sample example related with web crawler and Hadoop Map-reduce setup
Learn R – How to Get Data Frames Columns as Vectors
This article represents different ways in which one could get a data frame column as a vector. Please feel free to comment/suggest if I missed to mention one or more important points. Also, sorry for the typos. 4 Techniques to Get Data Frame Column as Vector In the examples below, diamonds dataset from ggplot2 package is considered. This is how a diamond dataset looks like: Following are four different technique/method using which one could retrieve a data frame column as a vector. # In the data set shown above, carat represents column name and hence, [[‘carat’]] carat1 <- diamonds[[‘carat’]] # In the data set shown above, carat represents 1st column …
Top 8 Data Science Training Institutes in India
This article lists down top 8 data science/analytics training institutes from India. Some of them including INSOFE just provide classroom coaching while others such as Edureka provide online training. Please feel free to comment/suggest if I missed to mention one or more important points. Also, sorry for the typos. Following is the list of training institues which are detailed later in this article: INSOFE Jigsaw Academy UReach Solutions AnalytixLabs Edureka SpringPeople SimpliLearn EduPristine INSOFE International School of Engineering was launched in 2011 with an aim to transform the applied engineering education space in India. Their current focus area is Big Data Analytics / Data Science. Out of all of …
Top 5 Usecases of Solr to Power Your Web & Mobile Search
This article represents top 5 usecases for using Solr to power your web and mobile search. Note that in case of mobile search requirements, Solr exposes APIs that could be used to retrieve data from Solr index server and serve to mobile client. It also presents a classification of websites which are using Solr to fulfill their search requirements. Please feel free to comment/suggest if I missed to mention one or more important points. Also, sorry for the typos. Following are the key points described later in this article: Top 5 Usecases for Solr Search Different Classes of Websites using Solr to Power Search Engines Top 5 Usecases for Solr Search Search Engine: Many …
Dummies Notes on How Distributed Computing Works using Hadoop
This article intends to present dummies notes on how distributed computing works using Hadoop. As Hadoop is inspired by Google GFS/Map-Reduce/BigTable paper,I have tried and refer to GFS/Map-Reduce/BigTable in this article appropriately wherever possible. One must note that distributed computing paradigm has become mainstream given the advent of Big Data related large scale project implementation going on in several companies. Please feel free to shout if you find discrepancies with my understanding and help me correct the mistakes. Simply speaking, distributed computing refers to the computing paradigm in which processing happens on multiple different boxes consisting of data and, the result is, then, aggregated appropriately to display the final result. In traditional …
60 Most Commonly Used R Packages in R Programming Language
This article represents a comprehensive list of 60 most commonly used R packages which helps to achieve some of the following objectives when working with data science/analytics projects: Predictive modeling Data handling/manipulation Visualization Integration Hadoop GUI Database 60 Most Commonly Used R Packages Following is the list of 60 or so R packages which help take care of different aspects when working to create predictive models: Predictive Modeling: Represents packages which help in working with various different predictive models (linear/multivariate/logistic regression models, SVM, neural network etc.) caret: Stands for Classification And REgression Training. Provides a set of functions which could be used to do some of the following when …
Data Science – Who could become a Data Scientist?
This article represents information related different classes of IT & Non-IT professionals who could take on different data science free courses (as mentioned) and get on to the path of becoming a data scientist. Please feel free to comment/suggest if I missed to mention one or more important points. Also, sorry for the typos. Following are the different classifications of IT/Non-IT professional which has been addressed later in this article: Software Development Stakeholders working on Non-analytics projects Datawarehouse/BI Developers Big Data Developers Statisticians Senior Management Executive Non-Software Professionals Could I become a Data Scientist? Anyone matching following criteria could become a data scientist. One is decent with Mathematics & Statistics …
Top 10 Solution Approaches for Supervised Learning Problems
This article represents top 10 solutions approaches that could be used to solve supervised learning problems. For those unaware of what is supervised learning problem, here is the supervised learning definition from Wikipedia: Supervised learning is the machine learning task of inferring a function from labeled training data.[1] The training data consist of a set of training examples. In supervised learning, each example is a pair consisting of an input object (typically a vector) and a desired output value (also called the supervisory signal). A supervised learning algorithm analyzes the training data and produces an inferred function, which can be used for mapping new examples. Following are two different kind of supervised …
Document Search Architecture to Search Millions of Documents
This article represents different document search architectural models using which one could create a search architecture that could search through 100s of millions of documents in faster time (milliseconds) with most up-to-date and fresh results. If you are planning to create a document search infrastructure which could search millions of documents, and shows up results in less than a second time, go ahead and explore different models and adopt the one that suits your needs at this stage. Note that the models given below could scale to multiple data centers. In this blog, we shall try and examine different architecture models that could achieve the search timing of less than a …
Learn R or Python for Becoming Data Scientist?
This article presents analysis on whether one should go for learning R or Python programming language to create one or more predictive models using different machine learning algorithms. It could be noted that both languages, R and Python, is equally doing good and sought after by developers and the companies hiring such developers. So, you could choose either one of these languages. However, majority has been found to be voted in favour of Python for ease of learning and greater community support. Data Scientist with expertise in R Following indeed.com plot represents the job trends for the search term, “Data Scientist R”. It clearly indicates the trend such as …
Machine Learning – Top 16 Learning Resources on Statistics
This article represents some of the top learning resources (webpages, videos etc) on my frequent visit list. Please feel free to comment/suggest if I missed to mention one or more important points. Also, sorry for the typos. Following are the key categories of webpages/videos that are expanded later in this article: Websites Quora Youtube Videos Coursera courses Khan Academy Top 16 Learning Resources on Statistics Folllowing is the list of URLs for these learning resources: Websites on Statistics Stattrek.com Elementary Statistics with R StatsDirect.com Usable Stats Quora.com Statistics Channel Probability & Statistics Statistics (Acacedmic Discipline) Bayesian Inference Youtube Videos Playlists on Statistics Brandon Foltz StatisticsFun JBStatistics Quantitative Specialists Coursera Courses …
Machine Learning Research in Top 10 US Universities
This article represents information related with machine learning departments & related research projects in top 10 US universities (as per USNews Ranking). I have put it together for my quick reference and thought to share with you for the same purpose. Please feel free to comment/suggest if I missed to mention one or more important points. Also, sorry for the typos. Following are top 10 universities covered later in this article: Princeton University Harvard University Yale University Columbia University Stanford University University of Chicago MIT Duke University University of Pennsylvania California Institue of Technology Machine Learning @ Top 10 US Universities Princeton University: Machine Learning Department at Princeton University …
Data Science – 175 Probability & Statistics Interview Questions
This article presents URL and short description of around 175 probability & statistics objective questions which could prove very useful and helpful for those who are planning to attend one or more data scientist interviews in time to come. These tests/quizzes were created when I was learning probability and statistics some time back and, found various concepts interesting enough to be converted into quizzes for my future references. As probability & statistics form key to data science, it may be worth spending some time on these tests and check your understanding. You may also use this for your future reference. These questions could also be used for checking your concepts …
Quick Cheat Sheet for Big Data Technologies
This article represents quick details on some of the key open-source technologies (tools & frameworks) associated with Big Data. The objective of this article is to present quick details on open-source tools & frameworks in a well-categorized manner using top-down approach where data engineering and data science aspects of Big Data is associated with relevant tools & framework. Most of these tools and frameworks could be found with commercial Hadoop distributions such as Cloudera, Hortonworks, MapR etc. Please feel free to comment/suggest if I missed to mention one or more important frameworks. Also, sorry for the typos. Following is the key classication of tools/frameworks that have been briefed later in …
Machine Learning – Top 5 Video Channels for Regression Models
This article represents top 5 video channels that one could use to learn and become expert at regression models. I make visits to watch these videos, once in a while, to clarify my doubts in relation with regression models. As I find these pages very useful, I thought it to share with you all. These are some real good videos from learning perspective that could help you get started with regression models and get a good hang of it within no time. Please feel free to share it with your community. Please feel free to comment/suggest if I missed to mention any other great video channels. Also, sorry for the …
Data Science – Top 10 Websites to Bookmark for Daily News
This article represents links and information in relation with top 10 websites that publishes data science related news and article on daily/regular basis. These links are my favorites and help me remain up-to-date with latest and greatest happening in the field of data science. Please feel free to comment/suggest if I missed to mention/include one or more important and interesting websites in the list given below. Also, sorry for the typos. Following are the key points described later in this article: Top 5 Data Science News Websites – Recommended Daily Visit Top 5 Data Science News Websites – Recommended Regular Visit Top 5 Data Science News Websites – Recommended …
Nice question to help us