Transcript Open Source Software - Empirical Investigation of Technology

Revealing Semantic Quality
Characteristics of
Free and Open Source Software
Department of Informatics
Technological Educational Institute of Thessaloniki, Greece
Stergios Draganidis
Kerstin V. Siakas
[email protected]
[email protected]
Outline

This paper describes the initial stage of a research
project at the department of Informatics of Alexander
Technological Education Institute of Thessaloniki in
Greece1

The research project investigates the quality of
FLOSS2 considering underlying human factors that
affect the six high level quality characteristics
according to ISO-9126
1
This work is funded by the Greek Ministry of Education (25%) and
European Union (75%) under the EPEAEK II program “Archimedes ΙΙ“
2
Free/Libre Open Source Software
Objectives

to introduce a scientific, rational and structured
instrument for FLOSS product assessment

to correlate the quality characteristics described
in ISO-9126 with measurable aspects and
underlying factors of FLOSS collaborative
development

to use the results from the initial stage for
improvement of the survey instrument to be
used on a larger scale
State-of-the-art

During the last decades a large number of
FLOSS products have been developed

The development of software through
collaborative informal networks is proposed to
produce high quality products

Several products have become competitive in a
variety of applications
 Gnome
– desktop environments
 Linux
– operating systems
 Mozilla
– web browsers
 Apache
– web servers
FLOSS
Free/Libre and Open Source Software

Combination of the terms
 “Free Software”
– Free Software Foundation
 “Open Source Software” – Open Source Initiative

Emphases the ‘libre’ meaning of the word free

Refers to software (available source code)
that gives the freedom to the user to:
 run the program for any purpose
 to study the code
 to modify the code
 to distribute the original or modified program without
any restriction or any obligation
Free Software – Open Source
Software

two ideological movements that target the same
goal:
To evolve knowledge and
develop better products

The motivations of each movement differ and many
discussions have been made to disambiguate their
nature

the terms “free software” and “open source
software” are usually used interchangeably to
describe the same thing
FLOSS Development

Collaboration and co-creation of the code by developers
(professional & amateur programmers) and users of
varying levels of programming experience

Typical FLOSS developers:

work:
• independently of any organisation
• for sake of the community projects
• isolated in their own pace, space and time

are:
• geographically dispersed
• simultaneously users of the software
Typical Developer Contribution

Start from existing or partial code

Take part in the development of a new tool or a utility

Post the evolving code to a newsgroup whose
members will try out the program and provide
feedback (programming solutions, improvements)

The enhanced code is then posted to the group, who
accepts or discards it and redistribute the final
product freely available to the wider community
One person = developer + user

The developers are themselves users of the
software  better understanding and solution of the
problems or bugs

In Proprietary Software:

the developer has difficulties to understand the
often ambiguous user requirements

Inadequate communication between the developer
and the user creates inflexible to future changes
products and products that are difficult to maintain
One person acting as developer and user simultaneously
in the development of FLOSS products seems to be a
prominent feature affecting the quality characteristics of
the software product
FLOSS Quality

The first FLOSS products were expert developers using
empirical procedures and metrics that ensure high quality
products delivery

Today, the quality of FLOSS projects is not based on a
specific method or metrics

The evaluation of quality is based on arbitrary criteria that
usually are not sufficient or adequate to represent the
quality of the product

Projects hosts provide limited metrics
 eg. sourceforge.net provides:
• projects size
• number of developers
• activity rank
• weekly download
Free/Libre and Open Software:
Survey and Study

International Institute of Infonomics University of Maastricht,
The Netherlands in 2002

First attempt for quantification and measurement of underlying
factors of quality
The Acronym FLOSS was used for first time


Delivered a large number of measurements and primary data
related to FLOSS usage and development

The aims:

To provide indicators that rationally explain the functioning of
the collaborative networks that do not have primary
dependence on money

To investigate possible impact of non monetary collaborative
development on computer industry, business and economy
* http://www.infonomics.nl/FLOSS/report/
ISO/IEC 9126 Quality Model



A framework for the evaluation of software product
quality
Applicable to every type of software
Defines six high level product quality characteristics:
 Portability
 Efficiency
 Reliability
 Functionality
 Usability
 Maintainability
Product Quality
Sub-characteristics
Attributes
Measurable entity
Not defined in standard
The ISO/IEC 9126 Standard
ISO 9126 Divisions
The standard is divided into four parts which address:
• the quality model
- ISO/IEC 9126-1 (2001)
• external metrics
- ISO/IEC 9126-2 (2003)
• internal metrics
- ISO/IEC 9126-3 (2003)
• quality in use metrics - ISO/IEC 9126-4 (2004)
Department of Informatics
Technological Educational Institute of Thessaloniki,
Greece
Research methodology :

Survey

on-line questionnaire
http://sdragan.phpnet.us/questionnaire.htm
Target group

Active persons of FLOSS projects, such as:



Project managers
Programmers
users
The research will be carried out
from April 2007 until August 2008
Questionnaire design

Php and MySql was used to implement the online
questionnaire

21 questions relating to significant attributes describing
human features and processes that have impact on
quality of FLOSS according to ISO-9126

Emphasis on:


Simplicity, user friendliness and wide acceptance

unbiased understandable questions
Demographic questions related to the responders for

clustering of the results

investigation of possible impacts to the quality
characteristics
Sample Issue Concerns

Minimum tolerated sample size for smallest sub-group
of importance?

How to ensure a representative sample?

Our target group in the initial stage:

The Greek Floss community

Results may not be representative for the wider
FLOSS community
Pilot Study: Questionnaire
moderation & promotion

The initial questionnaire was

developed in the Greek language

distributed to many individuals in charge of Greek
FLOSS communities aiming
• to evaluate the questionnaire regarding the validity and
outline
• to secure future commitment from the FLOSS
community

9 responses were received with useful comments

The feedback was used to improve the questionnaire
Questionnaire moderation &
Promotion

The moderated questionnaire was translated to
English in order to reach a greater population outside
Greece

Available on-line to enable easy completion and
analysis of the results

The on-line questionnaire has been promoted to many
FLOSS communities in order to ensure statistically
sufficient responses and produce valid results
Survey process

Acquisition of primary data that describe attributes and
processes constituting FLOSS development

Data Storage in a database system in order to simplify
appropriate transformation of the data before the analysis

Data analysis

Statistical analysis (SPSS-15)

Data mining methods will be applied to predict the
behaviour of the project
• to classify projects according to their quality measurements
• to point out quality related attributes of the FLOSS
development
Data Analysis


Considerable effort
was made to enable
the extraction of valid
results rapidly from the
primary data
A system that collects
the questionnaire
responses and
processes them
accordingly has been
developed
Expected Outcome

to reveal factors that may have impact
on the quality of FLOSS products

to increase the comprehension of
FLOSS quality attributes

to raise awareness of the dynamics that
influence the development process

to contribute to a more scientific,
rational and structured assessment of
FLOSS products & processes
Conclusions



The initial stage of a research project aiming to
determine quality level of FLOSS products was
described
Our approach evolved the research approaches that
concentrate on internal characteristics of ISO-9126
to include external characteristics and quality in
use
The correlation of quality characteristics of ISO9126 with measurable aspects and underlying
factors of FLOSS collaborative development was
expected to provide a more scientific, rational and
structured tool for FLOSS product assessment
Further Work

Stage 1:

Extending the research group
• Collaboration welcome!
• Searching for EU funding
• Preparing proposal

Improving the questionnaire

Further promotion of the questionnaire
Further Work

Stage 2:

Collecting data

Analysis of the collected data
• Statistical analysis of the data by SPSS
• Application of data mining methods

Stage 3:

Point out different perspectives and summarize them to
useful information

Dissemination
Thank you for your attention
Any questions..