Transcript Approaches to Application Modernization

Approaches to Application
Modernization
Don Denoncourt
[email protected]
Outline
•
•
•
•
•
•
•
What is App Modernization and Why Bother?
Modernizing Coders
Code Moves to Data, Modernizing the Database
Gradational versus Global Approach
Fear Factor
Grokking RPG
Save the Program, Save the Platform
"Modernization is our jobs. It’s what IT does.
We improve access to information for the
businesses we work for."
• Trevor Perry
Traits of a Modern App
•
•
•
•
It has maintainable and testable code
It is modular and component-based
It provides a rich user interface
It uses the Model-View-Controller (MVC)
design pattern
– Separate presentation, flow control, and business
logic
• It is responsive to new business requirements
Why Bother?
• Modernizing applications is costly
– Not modernizing is more costly
• "60 to 80 percent of all software development
is enhancements to or maintenance of existing
projects."
– Gartner Group report
• Amounts to a 10% increase in legacy code per
year
Escalating Costs
• Effort required to enhance and maintain legacy
apps escalates with time
• Fact:
– Code is read far more often than it is written.
• So:
– "What sense does it make to bloat unreadable
code at a rate of 10 percent per year?"
• Reducing maintenance costs may be the biggest
reason to modernize
Modular Applications:
• Allow you to quickly respond to business
requirements
• Improves interoperability:
– Other applications
– Languages
– Technologies
• Including SOA and REST
Modernizing Coders
Three Contexts of Modernization
1. User Interface
2. Code
3. Coders
Modern UI
• If you're not rewriting your entire app in a
new language
– Need to modularize RPG
• before adding a Web front-end
• What about… HATS and WebFacing?
– WDHT is the Botox of UI modernization
• Quick and easy
• But behind that puffed-up face
– There will always be the same old person
Application Make-over
• To make over a person (or an application)
– You need to start from the inside
• That means
– Attitude
– Knowledge
– Organization
Attitude, Knowledge, and
Organization
• Attitude should come from personal fulfillment
– But it can also be coerced from effective management
• Knowledge is acquired from experience or
education
• Organization
– micro-level
• when restructuring a program
– macro-level
• when planning an attack for modernization.
Training
• Formal education
– Most common strategy
• But the best training is mentoring
• Conferences and formal classes
– are worthwhile
• But mentoring virtually guarantees
– New techniques can transform from knowledge
to functioning business code
Mentor
• If you don’t have internal expertise
– Get a consultant
• one with mentoring skills
• To substantially decrease costs:
– Consider:
• WebEx, GoToMeeting, and alternative web
conferencing tools
– Consultant may be off-site and part-time
• Maximize your training budget's ROI
– With 2 hours mentoring a day
Master Programming Toolsets
• Promote use of a modern development tools
– WDSc/RDi
– Eclipse, NetBeans, Visual Studio
A good IDE provides tools that are necessary
in the modernization process.
Grokking the Code with WDSc
Application Diagram
Search and Replace
Outline View
Integrated Help
Streamlined edit-compile-debug
Code Moves to Data
Modernize Your Database
• 20-year-old RPG
– Code manages what should be done with db
• Table relationship management
• Foreign key constraints.
• Refactoring that code out
– Reduces requirement for code
Does your database really need
modernization?
•
•
•
•
If your files are defined with DDS
If you are not using constraints and triggers
If you are planning to implement Business Intelligence
If you are planning to provide a web front-end to your
data
• If new developers need to look at code to understand
your database
– Then. Yes. You really need to modernize your database.
Significant Performance
Improvements
• DDL-defined table page size much larger
that DDS-defined files
– DDL page size 64K
– DDS page size 8-32K
• Encoded Vector Indexes (EVI)
• SQL query engine (SQE) runs below the MI
– Versus the Classic Query Engine (CQE)
http://systeminetwork.com/article/performance-comparison-dds-defined-files-and-sql-defined-files
New Column/Field Capabilities
•
•
•
•
•
•
BLOBs and CLOBs
Constraints
User-Defined Data types (UDT)
Formula columns
Encryption and decryption
Complex view definitions (logical files)
– Expressions
– Aggregation
– Complex selection
New Column/Field Capabilities
• Long table and view names
– 128 characters
•
•
•
•
•
Identity and RowId columns
Sequences
Auto-increment of keys
Column-level triggers
Column-level privileges (authorities)
Object-relation Mapping Tools
• Generators for ORM engines
– Such as
• Java Persistence API
• Hibernate
• OJB
– Work best with a well-crafted database
• And standard SQL DDL constructs
Object Traversal
• Enabled with 1 database operation via joins
– Versus a read for each row/record
– Java:
• event.getVenue().getAddress().getStreet();
– Grails:
• event.venue.addresss.street
More reasons to modernize
• SQL is a widely used standard.
– Openness
• more and better options for third-party tools
– Books and training readily available
• Performance:
– IBM is investing money on improving database access
through SQL, not elsewhere.
• Availability of skills
– Easier find Java and SQL skills over RPG and DDS
knowledge.
• Functionality
– Some new functions require SQL.
“Code moves to data”
• Business rules
– As much as possible
• belong in DDL
• DDL-based Business Rules
– Constraints
• Primary and foreign keys
• Column-level contraints
– Triggers
• Operations to perform on
– Add, update, or delete operations.
Benefits of DDL Business Rules
• Removes duplicate code
– Decreasing complexity
– Improving maintainability
• Self-describing
• Enforces database integrity
– Rules are always followed
• Whether .Net, PHP, or Java web front-end applications
• Or other RPG
SQL's Data Definition Language (DDL)
5-minute tour
create table item (
itemid varchar(10) not null,
• PetStore DB
productid varchar(10) not null,
– Used in the ubiquitous
listprice decimal(10,2) not null,
unitcost decimal(10,2) not null,
sample web application
supplier int not null,
status varchar(2) not null,
attr1 varchar(80) not null,
attr2 varchar(80) not null,
attr3 varchar(80) not null,
attr4 varchar(80) not null,
attr5 varchar(80) not null,
constraint pk_item primary key (itemid),
constraint fk_item_1 foreign key (productid)
references product (productid),
constraint fk_item_2 foreign key (supplier)
references supplier (suppid)
);
create index itemProd on item (productid);
create table product (
productid varchar(10) not null,
category varchar(10) not null,
name varchar(80) not null,
descn varchar(255) not null,
constraint pk_product primary key (productid),
constraint fk_product_1 foreign key (category)
references category (catid)
create table supplier (
);
suppid int not null,
create table category (
name varchar(80) not null,
catid varchar(10) not null,
status varchar(2) not null,
name varchar(80) not null,
addr1 varchar(80) not null,
descn varchar(255) not null,
addr2 varchar(80) not null,
constraint pk_category primary key (catid)
city varchar(80) not null,
);
state varchar(80) not null,
zip varchar(5) not null,
phone varchar(80) not null,
constraint pk_supplier
primary key (suppid)
);
PetStore on Squirrel SQL Client
PetStore Database on
WDSc's Data Perspective
Field-level Constraints
CREATE TABLE orders (
CustNum CHAR(4),
OrderNum CHAR(4),
Quantity INTEGER CHECK (Quantity BETWEEN 1 AND 50)
);
On Delete and On Update
CREATE TABLE CORPDATA.EMPLOYEE
(EMPNO CHAR(6) NOT NULL PRIMARY KEY,
FIRSTNME VARCHAR(12) NOT NULL, MIDINIT CHAR(1) NOT NULL,
LASTNAME VARCHAR(15) NOT NULL,
WORKDEPT CHAR(3)
CONSTRAINT WORKDEPT_EXISTS
REFERENCES CORPDATA.DEPARTMENT (DEPTNO)
ON DELETE SET NULL ON UPDATE RESTRICT,
PHONENO CHAR(4), HIREDATE DATE,
JOB
CHAR(8), EDLEVEL SMALLINT NOT NULL,
SEX
CHAR(1), BIRTHDATE DATE,
SALARY DECIMAL(9,2), BONUS DECIMAL(9,2), COMM DECIMAL(9,2),
CONSTRAINT UNIQUE_LNAME_IN_DEPT UNIQUE (WORKDEPT, LASTNAME)
);
Database Reverse Engineering
Tooling
•
•
•
•
Navigator
WDSc
System i System API
Third-party tools
Navigator has a suite of SQL tools
Note: IBM is phasing out database development with Navigator
WDSc's Database Perspective
Generate DDL
on a PF called
ALLTYPES
System i API
• Generate Data Definition Language
(QSQGNDDL)
– Carsten Fleming wrote an iSeries command:
GENSQLDDL DBOBJ(yourFile) +
DBLIB(QSYS) +
TYPE(*PF) +
SRCFILE(QGPL/QDDLSRC) +
MBR(yourFile) +
MBROPT(*REPLACE)NAMING(*SQL)
http://systeminetwork.com/article/apis-example-reverse-engineering-database-files-and-objects-sqlddl-statements
Gradational versus Global
Approach
Two Strategies:
Globally or Gradually
• Global modernization
– Management backs the project with bucks
– Bring in consultants
– Purchase a modernization tool-set
• such as Databorough’s X-Analysis.
• Gradational modernization
– Refactor code on an ongoing basis during
normal business development
Global Realities
•
Management often will not back such a
concentrated effort
– Even though, in the long run,
• It is the most cost effective
•
There are two alternatives to this
1. Do nothing
• And let your code swell beyond recognition
2. Improve code with the gradational approach.
Gradational Philosophy
"Regard every bug fix or enhancement as an
opportunity to improve code."
Gradational Realities
• Consider a change request
– Estimated as a 10- to 20-hour project
• Quick-and-dirty status-quo
• Opportunity to modernize
• First time realities
– Potentially take 80–140 hours
• Time will appreciably decrease on each new
request
Gradational Benefits
• Soon you will be more receptive change
requests
• Along with
– Increased adaptability
– Greater interoperability
– Reduced code bloat
Fear Factor
Fear Factor
• Strategy a bit too radical?
• Scared to make far sweeping changes to a
program?
• Frightened because you can’t deduce the
impact of those changes?
Fear is Why Legacy Code Bloats
• Instead of correcting existing code…
– We add:
• Global variables
• Subroutines
• Or copy the program and modify the new on
• We do this so we know our new code won’t
affect old code
We Need Confidence
• Conviction that changes will work in
production
"The only way to be confident about code
changes is automated testing."
"If you can’t figure out how to test code, don’t
bother writing it. "
Refactoring
• Well known software engineering practice
– Changing code to simplify or improve
readability without changing the result.
• Automated unit testing ensures
– Code modifications cannot cause the
application to fail
RPG Unit Test Example
/free
// test encryption
encrypted = IMAXgetIFSFileAsBase64('/rxs/contract/irm/dontdelete.xml');
assertEquals(IMAXgetIFSFile('/rxs/contract/irm/dontdelete.xml'):
IMAXdecryptBase64(encrypted));
// test uploadFiles web service with 2 good files, and 2 bogus recipients
paths(1) = '/rxs/contract/irm/US0106091235211t.TXT';
paths(2) = '/rxs/contract/irm/duplicate.txt';
recipients(1) = 'Sam';
recipients(2) = 'Will';
uploadRsp = IMAXuploadFiles('TRACM01908':'rpdp01':
paths: recipients: 'RXS test request');
assertEquals('500': %char(uploadRsp.statusCode));
kar = uploadRsp.statusMsg;
assertEquals('Internal System Error - Invalid users:SAM,WILL': kar);
// test uploadFile with the same 2 good files and 1 valid recipient
reset recipients;
recipients(1) = 'TRACMTEST';
uploadRsp = IMAXuploadFiles('TRACM01908':'rpdp01':
paths: recipients: 'RXS test request');
assertEquals('500': %char(uploadRsp.statusCode));
paths(1) = '/bogus/bogus.TXT';
Gradational 4 Steps
1. Write a wide range of unit tests over existing code
–
Run them before any modifications are made.
2. Make modernization changes
•
•
Without adding bug fixes or enhancements
Rerun the tests.
3. Add tests for the change request
•
Run the tests
•
which will fail
4. Make the change
•
Iteratively rerun tests and tweak the code
•
Until the tests succeed
And consider fear no longer a factor.
Confidence with Source Control
• Source control make you bold with code
changes
– Without the ability to revert to a prior version
of code
• Coders are reluctant to make extensive changes
• Use a source-control product
– Don't rely on manual methods.
• Use open-source CVS and Subversion tools.
– or one of the various commercial products
Grokking RPG
Grokking the Code
• When a change request comes in
– Don't hack the change
– Grok the code and modernize
• Grok means:
– "Understand profoundly and intuitively"
Grokking RPG
• Rip the program apart with both your brain and
WDSc tooling until you fully comprehend it.
• If you can't Grok RPG because of a poorly named
variable or routine
– Rename it to something more meaningful
– Run unit tests to prove that your refactoring didn’t
break the code
• Discover business entities
– Those not already in record formats
– Define external structures for them.
• Refactor the code to use the external entity
structure
• Recompile and retest
Separating UI from Biz Logic
•
•
First step to modernizing
Separating business logic from display
gives three advantages:
1. You can create automated tests
2. You can reuse the modules in other
applications
3. You can more easily interface with web
technologies
Implement the Change Request
• With UI and Biz logic separate
– And perhaps readability improvements
• Now ready to implement the change request
Code Change Process
• If change request requires modification of a subroutine
– Convert it to a sub-procedure
• Remove global variables in the subprocedure
– May force you to add parameters to the subprocedure
• But enables unit testing
• Note that file parameters are available for RPG with
V6R1
– File I/O was the last vestige of requirements for global
variables in modern RPG
Code Change Process continued…
• With global variables removed
– The subprocedure can be placed in its own source
member
– And becomes a module
• Create unit tests for the subprocedure
• How did we used to test subroutines?
– We didn’t, because we couldn’t.
• Subroutines weren’t modular
– so automated atomic testing was not an option
• With subprocedures
– Unit testing should be a requirement
Save the Program,
Save the Platform
Code Inventory
and Empirical Study
• Do an inventory of your applications
– What are the assets
• Liabilities
• Do an empirical study
– What programs give you the most trouble?
• Or require the most frequent changes
– Modernize those first
Getting Started
• It costs less to modernize than to
– Put your coding heads in the sand
• And ignore code bloat
•
•
•
•
•
Deal with the fear factor
Mitigate risk with unit testing and source control
Get a mentor
Promote the use of WDSc and improved tooling
Start code reviews
RPG is One of IBM i's Greatest
Strengths
• Every RPG you rejuvenate strengthens the
platform
systeminetwork.com Articles
• "Grokking Software Development" (August 2007,
article ID 20979 at SystemiNetwork.com),
• iUnit from SystemiNetwork.com/code (select
April 2007).
• Instructions for iUnit
• "Developing Reusable RPG APIs" (February
2006, article ID 20395)
• "Testing RPG-based Stored Procedures with RPG"
(April 2007, article ID 20866).
• "RPG Source Control for Free" (August 2005,
article ID 20205),