Transcript pptx
Feature Engineering Studio September 23, 2013 Welcome to Mucking Around Day Sort into pairs • Partner with the person next to you • One group of 3 is allowed Sort into pairs • Do we have a group of 3? • One of the 3 will work with me Sort into pairs • Go over your reports together – A maximum of 5 minutes apiece 5 minutes for first person 5 minutes for second person Re-assemble into one big group Who here found something really cool while mucking around? • Show us, tell us Who here found a histogram with a normal distribution? • Show us, tell us Who here found a histogram with a hypermode? • Show us, tell us Who here found a histogram with a flat distribution? • Show us, tell us Who here found a histogram with a skewed distribution? • Show us, tell us Who here found a histogram with a bimodal distribution? • Show us, tell us Who here found a histogram with something else interesting? • Show us, tell us Who here found something surprising with their min, max, average, stdev? Categorical variables • Who here found something curious, weird, or interesting in the distribution of their categorical variables? Who here hasn’t spoken yet? (and analyzed data) • Tell us something interesting you found in your data Who here played with pivot tables? • What did you learn? My turn to play with pivot tables • Who wants to volunteer their data? • (I might request a 2nd or 3rd data set, depending on how the 1st one goes) Who here played with vlookup? • What did you learn? My turn to play with vlookup • Using the same volunteered data set(s) Other cool things you can create with a few simple formulas (plus demos!) Identifying specific cases of interest Did event of interest ever occur for student? Counts-so-far (and total value for student) Counts-last-N-actions First attempts Ratios between events of interest How many students had 3 (or 4, 5, 2,…) of an event Times-so-far Cutoff-based features Unitized actions (such as unitized time) Last 3 or 5 unitized Comparing earlier behaviors to later behaviors through caching Counts-if Percentages of action type Percentages of time spent per action/location/KC/etc. Questions? Comments? Other cool ideas? Assignment 3 • Feature Engineering 1 “Bring Me a Rock” • Get your data set • Open it in Excel • Create as many features as you feel inspired to create – Features should be created with the goal of predicting your ground truth variable – At least 12 separate features that are not just variations on a theme (e.g. “time for last 3 actions” and “time for last 4 actions” are variations on a theme; but “time for last 3 actions” and “total time between help requests and next action” are two separate features) • For each feature, write a 1-3 sentence “just so story” for why it might work • Test how good each features is Testing Feature Goodness • For this assignment, there are a bunch of ways to test feature goodness • Single-feature prediction models in data mining or stats package, giving correlation or kappa (special session this Wednesday) • Compute correlation in Excel (want to see?) – You can do this with binaries variables too, although it’s not really optimal • Compute t-test in Excel (want to see?) • Compute kappa in Excel (if you don’t know how, easier to do in RapidMiner) Were you right? • Which of your “just so stories” seem to be correct? • Did any of your feature correlate in the opposite direction from what you expected? Assignment 3 • • • • Write a brief report for me Email me an excel sheet with your features You don’t need to prepare a presentation But be ready to discuss your features in class Next Classes • 9/25 Special Session – Using RapidMiner to Produce Prediction Models – Come to this if you’ve never built a classifier or regressor in RapidMiner (or a similar tool) – Statistical significance tests using linear regression don’t count… • 9/30 Advanced Feature Distillation in Excel – Assignment 3 due – Online Equation Solver Tutorials should be in your INBOX Upcoming Classes • 10/2 Special session on prediction models – Come to this if you don’t know why student-level cross-validation is important, or if you don’t know what J48 is • 10/7 Advanced Feature Distillation in Google Refine • 10/9 Special session? TBD.