Transcript end html - DBPL 2013

FORWARD:
Declarative Ajax Web Applications
through SQL++ on a
Unified Application State
Yupeng Fu
Kian Win Ong
Yannis Papakonstantinou
UC San Diego
1
FORWARD
• Web application framework
• Single language: SQL++
• Zero frictions and automatic optimizations
– Enable IT personnel to create Ajax apps and interactive
visualizations
– Improve development productivity
• Alpha release
– http://forward.ucsd.edu
– 8 applications (commercial + academic)
2
FORWARD Website
http://forward.ucsd.edu
3
Simple Web Application
4
Simple Web Application
5
Simple Web Application
6
Mundane Low-Level Programming for Ajax Applications
Event
Side-effect
Browser
DOM (XHTML),
JavaScript (JSON)
Convert JSON
to objects
Convert objects
to JSON
Application
Server
Challenge #1: Impedance
mismatch due to different
languages and data models
Requires using multiple
languages, tedious data
modeling and coding
Java, Python
etc.(Objects)
Convert objects
to tables
Convert tables
to objects
Database
SQL (Tables)
7
Mundane Low-Level Programming for Ajax Applications
Event
Browser
DOM (XHTML),
JavaScript (JSON)
Retrieve user
input
Serialize and
parse request
Application
Server
Java, Python
etc.(Objects)
Side-effect
Check access
rights
Challenge #2: Distributed
data access and
programming due to
multiple tiers and machines
Requires partitioning code
manually, and ad-hoc
coordination across multiple
machines
Parameterize
SQL query
Database
SQL (Tables)
INSERT INTO
...
8
Mundane Low-Level Programming for Ajax Applications
Event
Browser
DOM (XHTML),
JavaScript (JSON)
Application
Server
Java, Python
etc.(Objects)
Database
SQL (Tables)
Side-effect
Incrementally rerender DOM and
JavaScript components
Instantiate partial
HTML and JSON
template
Incrementally update
old state to new state
SELECT *
FROM
...
WHERE id = ...
Challenge #3: Incrementally
modifying old page to new page
due to Ajax programming model
Requires identifying data flow
dependencies, correctly
transitioning from one consistent
state to another.
9
Incremental Updates are Error-Prone
• New requirements:
• Libraries with saved
comments are green instead
of red
• Multiple code paths to change:
• Code that loads the page
• Code that saves the
comment
• Code complexity increases with
number of events/actions
10
FORWARD Architecture
• SQL++ as single language
(inspired by Hilda and Links)
• SQL extensions for semi-structured features (nesting,
heterogeneity)
• State machine conceptual model
• No need for multiple tiers, machines, languages
Actions
Pages
Unified Application State (UAS)
11
FORWARD Architecture
• PL/SQL++
• INSERT, UPDATE,
DELETE statements
on UAS
http://localhost/libraries
• Control flow
(conditionals, loops,
functions)
Runtime Interpreter
Pages
Actions
side
effects
• Virtual database of
different data sources
• Unifying data model
read
write
next_page
Unified Application State (UAS)
12
FORWARD Architecture
• Declarative page as
rendered queries
Runtime Interpreter
• Data of page is a SQL++
query over UAS
• Rendering of page is
template markup
• Template markup for
both HTML and rich
JavaScript components
• No side-effects: can only
read from UAS
Actions
rendering
Page instance
Pages
read
Unified Application State (UAS)
13
FORWARD Architecture
• Action only saves
comment
• Action does not specify
how to replace the
textarea and Save button
on the page
Ajax (XHR)
rendering
Runtime Interpreter
Page instance
Pages
Actions
side
effects
read
write
next_page
read
Unified Application State (UAS)
User Input
14
FORWARD: Single Language, Location Transparent, Declarative
• SQL++ as single language
• Challenge #1: Impedance mismatch
Action
Specifications
Page
Specifications
Unified Application State (UAS)
Specification
• Declarative page as
rendered queries
• Challenge #3:
Incrementally modifying
old page to new page
• Virtual database
• Challenge #2: Distributed data access and programming
15
Outline
• Motivation
• Architecture
• Syntax and Semantics
–
–
–
–
Unified Application State
SQL++
Page
Action
• Future Work
16
UAS Specification
select
from
left join
where
...
db.libraries
session.notes on ...
...
Source Specification
define sql source db
options {
driver
: 'postgresql',
host
:
'book_db_host',
port
: 5432,
database : 'libraries',
user
: 'postgres',
password : '9xk8NToA'
}
• Developer provides source specification
• Framework provides capability-based source
wrappers that execute different subsets of SQL++
• SQL++ queries over UAS virtual database
• Run by FORWARD distributed query processor
• Current sources:
• SQL database
• In-memory SQL++ values
• Future sources:
• JSON databases (MongoDB, CouchDB)
• Large-scale data services (BigQuery, Redshift)
Unified Application State (UAS)
SQL
Source Wrapper
RDBMS
(Tables)
User Input
(JSON)
URL Params
(KV Pairs)
Session
(Objects)
17
SQL++ Data Model
value
=
|
|
|
• Data model designed to capture both
SQL tables and JSON for the need of
pages
• Captures subset of JSON used in
~30 JavaScript components
null
scalar
tuple
table
scalar = primitive
| type "(" primitive ")"
tuple
table
key
= "{"
( name : value "," )*
"}”
= "["
( key? tuple "," )*
"]"
= "(" primitive ")"
12.3
date('2013-08-30')
{ lat : 0.0 , lng : 12.3 , ... }
[
(1) { lat : 0.0 , lng : 12.3 } ,
(2) { lat : 0.0 , lng : 45.6 }
]
• Extended JSON syntax
with rich types and
keys
• Opposite of Math
conventions!
18
SQL++ Data Model
value
=
|
|
|
null
scalar
tuple
table
scalar = primitive
| type "(" primitive ")"
• Extensions over SQL tables:
• Optional type checking
• Root value
• Nesting
• Heterogeneity
• Optional keys
tuple
= "{"
( name : value "," )*
"}”
• Extensions over JSON
• Optional keys
• Rich scalar types (e.g. dates)
table
= "["
( key? tuple "," )*
"]"
= "(" primitive ")"
• Work in progress
• Order
• Table of scalars/tables
key
• Analogous SQL++ extensions to query
language
19
Page Instance
SQL++ Value
_1_html : {
template : '<body>...',
children : {
_2_google_map_Maps : {
markers : [
{
position : {
lat : 32.7,
lng : -117.2
},
HTML string template
with placeholders for
children
State of JavaScript
component
infowindow :
_3_html : {
template : '<div>...',
children : ...
}
}
...
]
}
}
}
20
Page Instance
Also SQL++ query
Also Page Specification
SQL++ Value
_1_html : {
template : '<body>...',
children : {
_2_google_map_Maps : {
markers : [
{
position : {
lat : 32.7,
lng : -117.2
},
Template Markup
<% html %>
<body>
..
<% unit google.map.Maps %>
{
markers : [
Syntactic sugar:
{
visual units
position : {
lat : 32.7,
lng : -117.2
},
infowindow :
_3_html : {
template : '<div>...',
children : ...
}
infowindow :
<% html %>
<div>
...
</div>
<% end html %>
}
...
]
}
...
}
}
}
Syntactic sugar:
inline HTML
]
}
<% end unit %>
</body>
<% end html %>
21
Outline
• Motivation
• Architecture
• Syntax and Semantics
–
–
–
–
Unified Application State
SQL++
Page
Action
• Future Work
22
Page Specification – Static Page
<% html %>
...
<% unit google.map.Maps %>
{
markers : [
{
position : { lat : 32.7, lng : -117.2 },
infowindow :
<% html %>
Library: Del Mar Library <br />
Comment:
<% unit html.TextArea %>
{
value : Great!
}
<% end unit %>
<br />
<button value="Save">
<% end html %>
}
...
]
}
<% end unit %>
<% end html %>
• Declarative static page using
only template markup
• HTML
• Visual unit wrappers for
JavaScript components
• Visual unit wrappers translate
between state and behavior
• Translation via renderers
and collectors
• No JavaScript code is
needed on the page
specification
23
Page Specification – Dynamic Visualization
<% with
View
libraries as
select
*
from
db.libraries
left join session.notes on ...
where
...
%>
<% unit google.map.Maps %>
Syntactic sugar:•
{
iteration
markers : [
<% for l in libraries
•
primary key (library_id)
%>
{
Substitution
position : {
lat : <%= l.lat %>
lng : <%= l.lng %>
},
...
}
<% end for %>
]
}
<% end unit %>
Declarative visualization page using only
template markup and SQL++ queries
Dynamic data of page specified by SQL++
queries over UAS
24
Page Specification – Dynamic Visualization
<% with
libraries as
select
*
from
db.libraries
left join session.notes on ...
where
...
%>
with
libraries as
select
...
from
db.libraries
left join session.notes on ...
where
...
{
<% unit google.map.Maps %>
{
markers : [
<% for l in libraries
primary key (library_id)
%>
{
position : {
lat : <%= l.lat %>
lng : <%= l.lng %>
},
...
}
<% end for %>
]
}
<% end unit %>
markers :
select
position : {
l.lat as lat,
l.lng as lng
}
from
libraries as l
primary key (library_id)
}
• Incremental updates are delegated to
FORWARD’s
• Incremental renderers
• Incremental view maintenance
25
Page Specification – User Input
<% with
libraries as
select
*
• from
Declarative
page for
db.libraries
left
join session.notes
on ...
visualization
and user input
where
...
%>
• User input is bound into the
UASgoogle.map.Maps %>
<% unit
{
: [
•markers
Events
trigger an action with its
<% for l in libraries
context
primary key (library_id)
%>
Variable in user
of
<% define comment string %>
input source
infowindow :
<% html %>
UAS
Library: <%= l.library_name %> <br />
Comment:
<% if l.comment is null %>
<% unit html.TextArea %>
Binds to variable
{
value : <% bind comment %>
}
<% end unit %> <br />
<button value="Save"
<% event click ajax /save_note %>>
{
position : {
lat : <%= l.lat %>
lng : <%= l.lng %>
},
...
<% else %>
Event / Action
<%= l.comment %> <br />
<button value="Delete"
<% event click ajax /delete_note %>>
}
<% end for %>
]
<% end if %>
<% end html %>
}
<% end unit %>
26
Action Specification
define action /save_note
function save_note;
define function save_note() as
begin
insert into session.notes
(library_ref, comment)
values (
context.l.library_ref,
context.comment
);
next_page('map');
end;
• An action is a URL mapped to a
function (procedure) that causes sideeffects
• Action can access all variables in the
event’s context
• define (mutable)
• with (immutable)
• for ... in (immutable)
•
insert, update and delete
statements writes to UAS
•
next_page() chooses the next page
to be evaluated and displayed
27
Related and Future Work in FORWARD
• Incremental view maintenance (IVM) of pages
– Incremental update of pages is delegated to FORWARD’s optimizations
– First IVM prototype by reduction to delta-based, relational IVM
• Y. Fu, K. Kowalczykowski, K. W. Ong, Y. Papakonstantinou and K. K. Zhao. Ajax-based Report Pages as
Incrementally Rendered Views. SIGMOD 2010.
– Work-in-progress based on algebraic rewrite rules that propagate predicates (as
opposed to data)
• Distributed query processor
– Virtual database is enabled by FORWARD’s distributed query processor
– Recent work led to distributor that is tuned for data of web applications: large
database and small in-memory objects
• Lightweight JavaScript components
– Visual unit wrappers for components with 50++ attributes / methods require
upfront time and effort to translate between state and behavior
– Work in progress: enable translation for subset of attributes / methods that
matter to a page
28
Cloud-based IDE
29
FORWARD Website
http://forward.ucsd.edu
30