Transcript Artificial Artificial Intelligence: Using Amazon Mechanical Turk and

Amazon Web Services:
Fueling Innovation and
Entrepreneurship
SOA Web Services Edge 2006
Jeff Barr
Web Services Evangelist
Amazon Web Services
[email protected]
1
Introduction & Background
> Software development background
> Veteran of several startups
> Visual Studio team at Microsoft
> Almost 4 Years with Amazon
> Amazon Web Services Developer Relations Team
2
What Is Amazon?
> Online Retailer
•
US, UK, Germany, Japan, France, Canada, China
> Technology Consumer
•
Multi-National Web Sites
•
•
Vast Data Warehouses
World-Class Logistics
> Technology Provider
•
Amazon Associates
•
Marketplace Sellers
•
Software Developers
3
The Web Services Concept
> The Programmable Web Site:
• Support for industry standards – HTTP, XML, SOAP, XSLT
• Remote access to data and functionality
• Decoupling of data and presentation
• Creation of a platform to attract software developers
• Vehicle to unlock developer creativity
• Means to leverage our technology investment
4
What Is Amazon Web Services?
> APIs that give developers programmatic access to
Amazon’s data and technology
•
Building-block web services
•
Web-scale infrastructure
•
E-commerce capability
•
Content, data, and information
•
New business models
•
Customer-created content
5
AWS Product Family
>
Amazon Mechanical Turk
•
•
>
>
•
>
>
Top sites by Alexa traffic rank
Roll your own search engine
ECS – E-Commerce Service
•
>
Data warehouse access for web crawl data
AWSP – Alexa Web Search Platform
•
IT building block
Alexa Top Sites
AWIS – Alexa Web Information Service
•
Data warehouse access for product
pricing
SQS – Simple Queue Service
•
>
Artificial Artificial Intelligence
Paid Internet-scale workforce
AHP – Amazon Historical Pricing
•
>
Direct access to Amazon’s product catalog
Amazon S3 – Simple Storage Service
•
Storage for the Internet
6
AWS Business Models
> Free – 1 call per second, send us traffic
• E-Commerce Service
> Monthly Fee
• Amazon Historical Pricing
> Usage / Resource Based
• Amazon S3, Alexa Services, Simple Queue Service,
Amazon Mechanical Turk
7
Amazon E-Commerce Service (ECS)
Complete access to the Amazon product catalog
• Rich data schema
• Millions of products
> Shopping cart & checkout
> Wishlists
> Revenue Model: Link to Amazon Associates program
8
Successful ECS Applications
tvmojo.com
associate-o-matic.com
9
Business Model: Click Sharing
Developer’s Associate ID
Site Operator’s Associate ID
Install
on
Server
Associates Commissions
10% to Developer
90% to Site Operator
Click!
10
Alexa Web Information Service
> Alexa Web Information Service
> Access to 300 TB of Alexa crawl data
> 10 billion web pages
> 5 functions:
• Crawl
• Search
• Url Info
• Category
• Web Map
11
Alexa Web Search Platform
> Build your own search engine!
> 3 Steps:
• Specify pages to access within the 300TB archive.
• Write parallelizable application to process these pages.
• Publish results as XML feed or as web service.
> Pricing – everything costs $1:
• 50 GB of data processing
• 1 CPU Hour
• 1 GB of data downloaded
• 4000 web service requests
12
Amazon S3
Simple storage service
> Storage for the internet - web service to read and write data
> 15 cents per Gigabyte-Month to store data
> 20 cents per Gigabyte to access data
> Private and public storage
> Scalable, reliable, cost-effective, and simple!
13
Amazon S3 Properties
>
Web-scalable
•
•
>
>
Reliable – 99.99% availability
•
•
Managed replication
Self-repairing
•
•
Fast
“Take it for granted”
Cost-effective
•
•
•
>
Scale to support unlimited number of applications
Use scale to increase reliability
Pay for what you use – storage and bandwidth
Amazon datacenters with industry-leading operating costs
Expect hardware failures
Simple
•
•
Built for use in any application
Focus on innovation, not figuring out storage
14
Amazon S3 Applications
S3 Explorer
filicio.us
S3 Ajax Wiki
Jungle Disk
Backup Manager
15
Amazon S3 Applications
> Smug Mug
> Ookles
> Chicago Crime
16
Amazon
Mechanical Turk
17
Man Versus Machine
18
But Not Really
19
Amazon Mechanical Turk
Provides Web Services API for
computers to integrate human
intelligence directly into their
processing by making requests
of humans
20
Put The Human In The Loop
while (read (photo))
{
ContainsHuman = CallMechanicalTurk(photo);
if (ContainsHuman)
{
acceptPhoto(photo);
}
else
{
}
rejectPhoto(photo);
}
21
How It Works
Requester
(Developer)
www.mturk.com
Human
Intelligence
Tasks (HITs)
Worker
Qualifications
Artificial, Artificially
Completed
HITs
Intelligent Software
Workers
22
What This Means
For Software Developers
>
Solves problem of building applications that until now have not worked well
without human intelligence
For Businesses
>
Efficiently, effectively and inexpensively complete millions of tasks that
require human intelligence
>
Previously unimaginable business ideas are now feasible
For Anyone
>
Make money using the skills you already have whenever your schedule
allows
23
Sample Mechanical Turk
Business – Casting Words
24
> Podcast transcription service provider
> Transcribes audio into high-quality text
> Amazon Mechanical Turk
• Workers transcribe podcasts and index text within search
engine
> Amazon S3
• Storing the podcasts and related files
25
26
27
28
29
30
31
32
Summary Of Our Offering
> 8 powerful and unique web services
> Commercial-use licenses
> Reasonable licensing terms
> 160,000-member developer community
> Bright future
33
Questions?
34
Thanks!
> Contact Info:
• Jeff Barr
• Web Services Evangelist
• [email protected]
• http://aws.typepad.com
35