Transcript Data
Data François Bancilhon twitter.com/fbancilhon Data consumers Data Analytics Dataviz Data Producers Public Data Editors Private Data Editors BI tools providers Platform providers Tool providers Data developer Custom Off the shelf Web Open Data Internal Datasets Datasets Editor Socizl Networks Crawl Scraping ETL, etc. Text Mining Structuration Augmentation … Delivery Visualization Data Publica : expertise Content Transformation Open Data Web Social network Private data Crowdsourcing Crawl Scraping ETL Data cleansing Text & Web mining Databases Generic dataviz Specific dataviz Data Presentation Is Data Global? • • • • Software vendors operate in a global market Data is country dependent The shape of the data is country dependent The access to the data is country dependent • You cannot build a local software business • You can build a local data business Internet start ups Fortune 500 48.8742° N, 2.3470° E 533 735 932 1 48 04 07 019 125 Dataset Examples 1. Public Tender announcements 2. Public organisations 3. Economic activity (by vertical and by geography) 4. City dashboard 5. Employment dashboard 6. Real Estate dashboard 7. Enrichment of company data 8. Weak signal detection in company data 9. Directory of a specific category of stores 10. Elected officials Public tender announcements OJEU CUSTOMER PLATFORM CLASSIFICATION (CPV) ANALYSIS BOAMP DELIVERY DEMAT DEMAT DEMAT PLATFORM PLATFORM PLATFORM DEDUP Public organizations in France • • • • 175 000 instances 5 000 types 6 sources, all of them public Data: name, code, address, phone, fax, email, web site • Hard part: deduplication within and between sources • LOD with public tenders (65% up to now) Territory dashboard Geographic granularity Theme Country State Disctrict City Neighborhood etc. Dashboard Economy Démography Health Tourism Environment etc. Territory dashboards Manual process Sources Manual data extraction Other Sources Industrial process Automated data extraction Automated data update Manual report production Vizualisation Generic dataviz Your report Custom report Data set enrichment • Given a list of companies and some information about them • Find more information – With the company information, find the web site – Analyse the web site to find more information – Eventually look outside of the web site Company data enrichment Total Entreprises with Web site Confident web site >0.5 # of shares information 10 000 companies 10 000 companies (%) 5 779 companies with website (%) 5 779 58% 100% 2 084 21% 36% 468 5% 8% Contact information 1 071 11% 19% Professional e-mail 1 893 19% 32% Phone 3 618 37% 62% Generic email 1 212 12% 21% Web site analysis • Analyse the web site do diagnose the company Internet presence Weak signal detection • Given a set of company description • Detect some weak signal – Export – Change in management Web site detection Web site analysis Signal detection Hotel Room Prices • • • • • • In a given geographic zone For all the hotels For all the rooms For every reserved day of the year For every reservation day of the year Provide the price of the room French Elected Officials Scope • Government • Mayors • Senators • « députés » (congressmen) • « conseillers généraux » • « conseillers départementaux » Information • First Name, Las Name • Mandate • Location • Date • Political affiliation François Bancilhon twitter.com/fbancilhon www.data-publica.com