Free and Open Source Software - Evaluation

Security Level:

Execution Environment:

Prototyping

Deployed version

Development Environment:
(If additional code needs to be developed)

Compliance with Mandatory Standards

Compatibility with legacy data/information formats
Ability to interface with existing informatic systems

Recommended Code Review
Yes/no, level required

Intellectual Property
Protection Issues

CF client preferences

Names

Responsibilities

Criteria

Definition

Importance Ranking In Project Context

Comments

Functionality

Does the program do what you want it to do? (REF Wheeler)
User application requirements
Match between requirements and the features offered

Cost

Initial license fees, installation costs, training costs, support/maintenance costs, license upgrade fees (usually nominal for OSS/FS programs), transition costs (such as data transition and/or transition to upgrades), and the costs of any necessary hardware. For a more balanced picture you may wish to separate one-time costs, such as those for transitions, from continuing costs.

Required Support and Maintenance

The term 'support' covers several areas: training users on how to use the product, installing it, and answering questions from users who have specific problems trying to use a working product and suggesting work-around for weaknesses in the current product. Useful programs are seldom completely static. Needs change, new uses are continuously created, and no program is perfect. In general, is there evidence that the software is under continuous development, or has work halted?

Reliability

Reliability is difficult to measure, and it depends very much on how the program is used. (REF Wheeler) GRAM, GRAS are good places to start

Quality

How well do the features you need seem to work? Do you like how they have been implemented?

Ease of Migration for Users

If moving from another software package, how hard is the migration process? Is it likely that users will have a difficult time adapting?

Performance and Scalability

The best way to measure performance is to try it on an actual case specific to your circumstance

Flexibility and Scalability

How hard is it to customize and adapt the software to your organization's needs? Will the software grow with your needs? Is it scalable?

User friendliness

Is the use of the software intuitive and obvious, given the skills of the people who will be using it? Is there a steep learning curve? Is it likely that users will have a difficult time adapting?

Developer Usability

Measure how easily programmers can use the application programmer interface (API). If the application sometimes needs to be controlled by programs, it is a significant advantage if it has a command line interface and a GUI that lets users control the engine through a familiar point-and-click interface.

Legal and License Issues

OSS/FS software licenses are important to developers, and they can affect users who may become developers, or who may pay developers to make a change, but not users who do not change the software.

Trustworthiness

Look at the code to see if the OSS/FS appears privileges; it strives for simplicity; it checks inputs carefully; to be trustworthy, i.e. it follows good practices, such as: it minimizes and source code scanning tools find few problems.

Product 1

Product 2

Product 3

Product 4

Criteria

Import- ance Ranking

Eval

Comments

Eval

Comments

Eval

Comments

Eval

Comments

Functionality

Cost

Required Support and Maintenance

Reliability

Quality

Ease of Migration for Users

Perform- ance and Scalability

Flexibility and Scalability

User friendliness

Developer Usability

Legal and License issues

Trust- worthiness