Transcript Data

Data
François Bancilhon
twitter.com/fbancilhon
Data
consumers
Data
Analytics
Dataviz
Data
Producers
Public Data
Editors
Private Data
Editors
BI tools
providers
Platform
providers
Tool
providers
Data
developer
Custom
Off the
shelf
Web
Open Data
Internal
Datasets
Datasets
Editor
Socizl
Networks
Crawl
Scraping
ETL, etc.
Text Mining
Structuration
Augmentation
…
Delivery
Visualization
Data Publica : expertise
Content
Transformation
Open Data
Web
Social network
Private data
Crowdsourcing
Crawl
Scraping
ETL
Data cleansing
Text & Web mining
Databases
Generic dataviz
Specific dataviz
Data
Presentation
Is Data Global?
•
•
•
•
Software vendors operate in a global market
Data is country dependent
The shape of the data is country dependent
The access to the data is country dependent
• You cannot build a local software business
• You can build a local data business
Internet
start ups
Fortune
500
48.8742° N, 2.3470° E
533 735 932
1 48 04 07 019 125
Dataset Examples
1. Public Tender announcements
2. Public organisations
3. Economic activity (by vertical and by geography)
4. City dashboard
5. Employment dashboard
6. Real Estate dashboard
7. Enrichment of company data
8. Weak signal detection in company data
9. Directory of a specific category of stores
10. Elected officials
Public tender announcements
OJEU
CUSTOMER
PLATFORM
CLASSIFICATION
(CPV)
ANALYSIS
BOAMP
DELIVERY
DEMAT
DEMAT
DEMAT
PLATFORM
PLATFORM
PLATFORM
DEDUP
Public organizations in France
•
•
•
•
175 000 instances
5 000 types
6 sources, all of them public
Data: name, code, address, phone, fax, email,
web site
• Hard part: deduplication within and between
sources
• LOD with public tenders (65% up to now)
Territory dashboard
Geographic
granularity
Theme
Country
State
Disctrict
City
Neighborhood
etc.
Dashboard
Economy
Démography
Health
Tourism
Environment
etc.
Territory dashboards
Manual
process
Sources
Manual data
extraction
Other
Sources
Industrial
process
Automated data
extraction
Automated data
update
Manual
report
production
Vizualisation
Generic
dataviz
Your report
Custom
report
Data set enrichment
• Given a list of companies and some
information about them
• Find more information
– With the company information, find the web site
– Analyse the web site to find more information
– Eventually look outside of the web site
Company data enrichment
Total
Entreprises with Web
site
Confident web site
>0.5
# of shares
information
10 000 companies
10 000 companies
(%)
5 779 companies with
website
(%)
5 779
58%
100%
2 084
21%
36%
468
5%
8%
Contact information
1 071
11%
19%
Professional e-mail
1 893
19%
32%
Phone
3 618
37%
62%
Generic email
1 212
12%
21%
Web site analysis
• Analyse the web site do diagnose the
company Internet presence
Weak signal detection
• Given a set of company description
• Detect some weak signal
– Export
– Change in management
Web site
detection
Web site
analysis
Signal
detection
Hotel Room Prices
•
•
•
•
•
•
In a given geographic zone
For all the hotels
For all the rooms
For every reserved day of the year
For every reservation day of the year
Provide the price of the room
French Elected Officials
Scope
• Government
• Mayors
• Senators
• « députés »
(congressmen)
• « conseillers généraux »
• « conseillers
départementaux »
Information
• First Name, Las Name
• Mandate
• Location
• Date
• Political affiliation
François Bancilhon
twitter.com/fbancilhon
www.data-publica.com