Transcript Data mining
A Best Practices Framework for Data Mining Mark Tabladillo, Ph.D., Data Mining Scientist Artus Krohn-Grimberghe, Ph.D., Consultant and Assistant Professor 13 June 2013 | Virtual Business Analytics Chapter About MarkTab Training and Consulting with http://marktab.com Data Mining Resources and Blog at http://marktab.net Ph.D. – Industrial Engineering, Georgia Tech Training and consulting internationally across many industries – SAS and Microsoft Contributed to peer-reviewed research and legislation ◦ Mentoring doctoral dissertations at the accredited University of Phoenix Presenter About Artus Assistant Professor for Analytic Information Systems and Business Intelligence PhD in computer science Research: data mining for e-commerce and mobile business Consultant Section One DATA MINING FOUNDATION 4 Definition 1 (Informal) Data mining is the automated or semi-automated process of discovering patterns in data. Definition 2 Data Mining is a process using 1. Exploratory Data Analysis Statistical and visual data analysis techniques. Forming a hypothesis 2. Data Modeling & Predictions Describe data using probability distributions and Machine Learning algorithms (“model”). Fitting a hypothesis 3. Statistical Learning Theory Model selection, model evaluation 6 Data Mining Visualized f( ) Input Target Target: attribute we are interested in. Input: data available for our predictions. Function f: describes the relationship between target and input. Regrettably, f is unknown and unknowable. 7 Data Mining Visualized Real world: Unknown f ( ) Input Target Data Mining model: Hypothesis h ( Need to find “good” h. h is your DM “algorithm”. ) Input data has to be appropriate. Select and transform as needed Correct modeling of target is crucial 8 Top 10 Expectations BEST PRACTICE: LEARN FROM EXPERIENCE 9 Expectation Ten Marketing More Scientific •People can start data •Better models come mining in 10 minutes… from days, weeks or months of iterative improvement 10 Expectation Nine Marketing More Scientific •Data miners can •Knowing the industry provide provably good and organizational models with little or goals helps orient the zero knowledge of the questions, modeling, specific industry… and analysis. 11 Expectation Eight Marketing •Open source software can provide quality results worthy of peerreviewed literature… More Scientific •Commercial software with years-long service options is required for enterprise scale. 12 Expectation Seven Marketing •We can learn a lot from the current data warehouses, cubes, and big data… More Scientific •We can improve our modeling by creating new data collection strategies. 13 Expectation Six Marketing •People can build data mining models with little or zero data cleaning… More Scientific •Better results happen when we organize and rearrange data for best success. 14 Expectation Five Marketing •Data mining can provide answers to problems… More Scientific •Most times we only get detail insights toward larger problems, and sometimes uncover more problems than we started with. 15 Expectation Four Marketing •A little data mining knowledge can provide an organization with a competitive edge… More Scientific •The edge grows along with experience and better study of the methodology and mathematics. 16 Expectation Three Marketing •Individual professionals can deliver excellent predictive analysis… More Scientific •Small teams working together can help quickly and efficiently conquer some of the most difficult analytic challenges. 17 Expectation Two Marketing •Numbers speak for themselves and can influence better decision making… More Scientific •Leadership strategy helps teams deliver results in the best way given the current culture. 18 Expectation One Marketing •A lot of data mining best practices and strategies can be communicated in an hour or a day… More Scientific •The best commitment is ongoing education on both data mining and machine learning technology. 19 Section Two ANALYZING AND PREPARING DATA 20 Best practice: study individual attributes Histograms and frequencies (discrete) Kernel density estimates Cumulative distribution function Rank-order plots and lift charts Summary statistics (continuous) Box-and-whisker plots 21 Best practice: study combinations Pivot tables Scatter plots Logarithmic plots Naïve Bayes Correlation matrices False-Color plots Scatter-Plot matrix Co-plot 22 Section Three MACHINE LEARNING ALGORITHMS 23 How to Choose an Algorithm Choosing an algorithm or series of algorithms is an art One algorithm could perform different tasks Be willing to experiment with algorithms and algorithm parameters 24 Algorithms for Data Mining Tasks (1 of 2) Algorithm Description Name Microsoft Time Analyzes time-related data by using a linear decision tree. Series Patterns can be used to predict future values in the time series. Microsoft Makes predictions based on the relationships between columns in the dataset, and models Decision Trees the relationships as a tree-like series of splits on specific values. Supports the prediction of both discrete and continuous attributes. Microsoft Linear Regression If there is a linear dependency between the target variable and the variables being examined, finds the most efficient relationship between the target and its inputs. Supports prediction of continuous attributes. Microsoft Clustering Identifies relationships in a dataset that you might not logically derive through casual observation. Uses iterative techniques to group records into clusters that contain similar characteristics. Algorithms for Data Mining Tasks (2 of 2) Algorithm Name Description Microsoft Naïve Bayes Finds the probability of the relationship between all input and predictable columns. This algorithm is useful for quickly generating mining models to discover relationships. Supports only discrete or discretized attributes. Treats all input attributes as independent. Analyzes the factors that contribute to an outcome, where the outcome is restricted to two values, usually the occurrence or non-occurrence of an event. Supports the prediction of both discrete and continuous attributes. Microsoft Logistic Regression Microsoft Neural Network Microsoft Association Rules Microsoft Sequence Clustering Analyzes complex input data or business problems for which a significant quantity of training data is available but for which rules cannot be easily derived by using other algorithms. Can predict multiple attributes. Can be used to classify discrete attributes and regression of continuous attributes. Builds rules that describe which items are likely to appear together in a transaction. Identifies clusters of similarly ordered events in a sequence. Provides a combination of sequence analysis and clustering. Best practice: Document your science Describe the business problem Determine how to measure success (including baseline) Document what was learned during data preparation and analysis Justify the algorithms used during the investigation List assumptions were made 27 Section Four ACHIEVING BUSINESS VALUE 28 Leadership challenges Build on organizational communications Consider redoing analysis Find results champions Celebrate the results 29 Best practice: prepare the next cycle Note strengths, weaknesses, opportunities, risks Build consensus on model expiration dates Encourage and improve the process Create insight into new future data collection 30 Conclusion Best Practices Framework Provide a data mining foundation Prepare the data Evaluate machine learning output Plan to move toward actionable decisions 31 Resources http://www.lfd.uci.edu/~gohlke/pythonlibs/ Free Win x64 Python libs http://www.enthought.com/products/epd.php Commercial Python http://www.burns-stat.com/pages/Tutor/R_inferno.pdf R Tutorial http://technet.microsoft.com/en-us/sqlserver/cc510301.aspx SQL Server Analysis Services Data Mining http://marktab.net Data Mining Portal http://sqlserverdatamining.com Data Mining Team Portal Books: “Data Mining with SQL Server 2008”, “Data Mining for Business Intelligence”, “Practical Time Series Forecasting” 32