Exploratory Data Analysis (EDA)

-


EDA is not identical to statistical graphics although the two terms are used almost interchangeably. Statistical graphics is a collection of techniques--all graphically based and all focusing on one data characterization aspect. EDA encompasses a larger venue; EDA is an approach to data analysis that postpones the usual assumptions about what kind of model the data follow with the more direct approach of allowing the data itself to reveal its underlying structure and model. EDA is not a mere collection of techniques; EDA is a philosophy as to how we dissect a data set; what we look for; how we look; and how we interpret. It is true that EDA heavily uses the collection of techniques that we call "statistical graphics", but it is not identical to statistical graphics per se.


A Gentle Introduction to Exploratory Data Analysis | by Daniel ...

Most EDA techniques are graphical in nature with a few quantitative techniques. The reason for the heavy reliance on graphics is that by its very nature the main role of EDA is to open-mindedly explore, and graphics gives the analysts unparalleled power to do so, enticing the data to reveal its structural secrets, and being always ready to gain some new, often unsuspected, insight into the data. In combination with the natural pattern-recognition capabilities that we all possess, graphics provides, of course, unparalleled power to carry this out.
The particular graphical techniques employed in EDA are often quite simple, consisting of various techniques of:


  • Plotting the raw data (such as data traces, histograms, bihistograms, probability plots, lag plots, block plots, and Youden plots.
  • Plotting simple statistics such as mean plots, standard deviation plots, box plots, and main effects plots of the raw data.
  • Positioning such plots so as to maximize our natural pattern-recognition abilities, such as using multiple plots per page.




Comments

Post a Comment

Popular posts from this blog

Project: GIS with Folium

-
Image
A geographic information system (GIS) is a conceptualized framework that provides the ability to capture and analyze spatial and geographic data. GIS applications (or GIS apps) are computer-based tools, that allow the user to create interactive queries (user-created searches), analyze spatial information output, edit data presented within maps, and visually share the results of these operations. Geographic information science (or, GIScience)—the scientific study of geographic concepts, applications, and systems—is commonly initialized as GIS, as well. Geographic information systems are utilized in multiple technologies, processes, techniques, and methods. It is attached to various operations and numerous applications, that relate to engineering, planning, management, transport/logistics, insurance, telecommunications, and business. For this reason, GIS and location intelligence applications are at the foundation of location-enabled services, that rely on geographic ana
Read more

tf-idf

-
Image
In information retrieval, tf–idf or TFIDF, short for term frequency–inverse document frequency, is a numerical statistic that is intended to reflect how important a word is to a document in a collection or corpus. It is often used as a weighting factor in searches of information retrieval, text mining, and user modeling. The tf–idf value increases proportionally to the number of times a word appears in the document and is offset by the number of documents in the corpus that contain the word, which helps to adjust for the fact that some words appear more frequently in general. tf–idf is one of the most popular term-weighting schemes today. A survey conducted in 2015 showed that 83% of text-based recommender systems in digital libraries use tf–idf. Variations of the tf–idf weighting scheme are often used by search engines as a central tool in scoring and ranking a document's relevance given a user query. tf–idf can be successfully used for stop-words filtering in various subjec
Read more

Project : Customer Churn (Customer Attrition)

-
Image
Customer attrition, also known as customer churn, customer turnover, or customer defection, is the loss of clients or customers. Banks, telephone service companies, Internet service providers, pay-TV companies, insurance firms, and alarm monitoring services, often use customer attrition analysis and customer attrition rates as one of their key business metrics (along with cash flow, EBITDA, etc.) because the cost of retaining an existing customer is far less than acquiring a new one. Companies from these sectors often have customer service branches which attempt to win back defecting clients, because recovered long-term customers can be worth much more to a company than newly recruited clients. Companies usually make a distinction between voluntary churn and involuntary churn. Voluntary churn occurs due to a decision by the customer to switch to another company or service provider, involuntary churn occurs due to circumstances such as a customer's relocation to a long-term
Read more