| 6TH GPCE YOUNG RESEARCHERS WORKSHOP 2004 |
|
A Common Criteria Based Approach for COTS Component Selection
Wes J. Lloyd, Colorado State University
|
 |
PDF Version |
Abstract
Component-based software engineering (CBSE) endeavors to enable software
developers to develop quality software systems with less time and resources
than traditional development approaches. Software components must be
identified and evaluated in order to determine if they provide required
functionality for systems being developed. Consideration of security
requirements for component selection is of interest. This research
considers how the Common Criteria (CC), an internationally recognized
standard for security requirements definition and security assessment
of IT systems, can be applied towards the development of component-based
systems. A CC-based COTS component selection process is proposed which
integrates activities of the CC for security requirements specification
and evaluation. Research questions are presented for the evaluation
of the process to establish its value for COTS component selection
as well as to identify areas for improvement.
1 INTRODUCTION
Rising costs and schedule constraints have forced many organizations,
including the U.S. government to use Commercial off-the-shelf (COTS)
components for developing applications with security concerns [Devanbu00].
By using preexisting components to implement security requirements,
software development organizations seek to reduce development costs
while still producing quality software in a timely manner. Components
that provide implementation of security functions such as encryption,
digital signing, access control, and authentication are often needed.
Implementing security mechanisms such as cryptographic algorithms is
complex and can result in the accidental introduction of serious flaws
into the system [Lindqvist98]. Consequently using COTS components to
implement complex security functions is an attractive option for developers.
Component selection decisions are often made in an ad-hoc manner.
Component selection processes have been proposed to improve upon the
efficiency
and effectiveness of informal methods. Existing selection processes
do not fully address the specification and evaluation of functional
and non-functional security requirements. Ruhe points out that many
existing COTS component selection processes do not provide procedures
for the evaluation of non-functional requirements [Ruhe03]. Security
vulnerabilities in components can result from development oversights
including unanticipated attacks, component integration issues, software
bugs, and unanticipated changes in COTS component product updates.
Lindqvist identifies insufficient component validation as a security
risk when using commercially available COTS components [Lindqvist98].
To support the development of secure component-based systems, component
selection processes need to address the problem of security evaluation
of COTS components. This research proposes a component selection
process, which is based on the Common Criteria (CC), an internationally
recognized
standard for security requirements definition and evaluation for
IT systems [ISO99]. The CC-based process proposes how the Common
Criteria
can be used to enhance the quality and efficiency of security specification
and evaluation of COTS components. By using the CC-based component
selection process developers can gain confidence using COTS components
to implement security functions when developing component-based
systems.
2 BACKGROUND
Selecting the appropriate COTS component to meet specified security
requirements is a security assessment problem. Myers states that written
and measurable objectives (requirements) are required in order to validate
their compliance in a software system [Myers76]. It is widely agreed
that requirements must be defined and quantifiable in order for testing
to be effective. For the assessment of COTS component security, functional
security requirements must be specified and delegated to particular
components in the software design. Existing component selection processes
specify methods to elicit software requirements in the general sense,
but they do not explicitly address how to specify security requirements.
Security requirements specification is challenging because requirements
encompass both functional and non-functional aspects and many developers
may not be familiar with the breadth of security issues needing to
be addressed.
Once security requirements are specified, available COTS components
must be evaluated to determine their suitability for use in the system
being developed. Bachman et al. identify the problem of evaluating
the security characteristics of COTS components [Bachman00]. Bertoa
proposed a quality model that defines a set of quality attributes and
metrics for evaluating COTS component quality. Bertoa identifies just
(3) security characteristics: data encryption, controllability and
auditability in [Bertoa02]. A presence metric is suggested to assess
support for these security characteristics. A presence metric as defined
only validates the existence of the function. Bertoa’s quality
model only begins to address the topic of COTS component security evaluation.
The consideration of many other functional and non-functional security
characteristics is desired. Emergent security characteristics are security
properties that are realized as a result of the system’s components
working together to provide security. Although an interesting problem,
this research focuses on security requirements specification and evaluation
at the component level to aid component selection. The evaluation of
security properties of the software system after component selection
and integration is an evaluation problem beyond the scope of this research.
A case study conducted by Bertoa discovered that of 164 COTS components
only (4%) provided information in product documentation (help-files,
manuals, demonstrations, and UML diagrams), which identified support
for security quality characteristics [Berota03]. This absence of documentation
detailing security properties of COTS components combined with the
questionable reliability of security claims in available documentation
creates the need for developers to perform their own component evaluations
to assess security.
Functional security requirements identify security mechanisms such
as user authentication, access control, encryption, data integrity,
and intrusion detection systems. The goal of these mechanisms is to
provide ample security to meet the system’s non-functional security
requirements. Non-functional security requirements identify a system’s
resilience and level of immunity to attack. In order to be highly resilient
a system needs to minimize the fallout resulting from attackers (hackers,
crackers, cyber-terrorists) exercising system vulnerabilities such
as software bugs through viruses, impersonation, sniffing, etc. In
a practical sense proving that software is secure is likened to proving
it correct. The validation of a software system’s non-functional
security is a NP-hard assessment problem which existing component selection
processes cannot address [Ruhe03]. To validate security, the software
would need to be formally proved correct against a functional specification.
This is an unrealistic problem to solve. Instead a generalized level
of security can be established using the Common Criteria for Information
Technology Evaluation (CC). The CC is a multi-part standard for specification
and evaluation of functional and non-functional security properties
for IT systems [ISO99]. The CC includes a general process model for
system security evaluation, a comprehensive set of security functional
requirements which can be used as a baseline to express security requirements
of all IT systems, and a thorough set of security assurance activities
for evaluating security provided by IT systems.
The CC defines a comprehensive set of security requirements composed
in (11) classes, which group together families of related security
requirements. Security classes include: Security Audit, Communication,
Cryptographic Support, User Data Protection, Identification and Authentication,
Security Management, Privacy, Protection of the system security functions,
Resource Utilization, System Access, and Trusted path/channels. The
security requirements identified in these classes can be used as a
basis to create security requirements specification documents.
The CC defines seven different evaluation assurance levels (EALs),
which define evaluation activities to assess security. These EAL levels
can be used to establish a generalized “level of security” provided
by a component. EAL levels include: functionally tested (EAL1), structurally
tested (EAL2), methodically tested and checked (EAL3), methodically
designed, tested and reviewed (EAL4), semi formally designed and tested
(EAL5), semi formally verified designed and tested (EAL6), and formally
verified designed and tested (EAL7). Evaluations beyond EAL3 require
intervention at the design phase of software development for security
assessment. Such evaluations may not be helpful for COTS component
evaluation since COTS components are typically preexisting. The CC
identifies (40), (83), and (110) assessment activities for levels EAL1,
EAL2, and EAL3 respectively. The CC certifies higher levels of security
assurance through applying higher levels of evaluation effort. Evaluation
effort is described as the scope, depth, and rigor of the evaluation
activities. Scope considers the percentage of the system being evaluated,
depth identifies the level of design and implementation analysis, and
rigor describes the level of formality of the assessment. No guarantee
of security exists regardless of the testing effort just as there is
no way to solve the oracle problem of software testing. We assume that
by applying higher evaluation assurance levels additional security
faults will be identified. The evaluation of the CC’s effectiveness
to establish security through evaluation is interesting and important,
but beyond the scope of this research.
3 HARNESSING THE COMMON CRITERIA
We propose a Common Criteria based COTS component selection process based
on common steps of existing component selection processes: requirements
identification, COTS component search, COTS component evaluation, and component
selection. Our process harnesses the Common Criteria (CC) for security specification
and evaluation. The process includes six steps: (1) System High Level Design,
(2) Component Requirements Definition, (3) Component Search, (4) Component
Evaluation, (5) Component Selection, and (6) Component Integration and Operation.
Fig. 1. CC Based Component Selection Process
In step (1), a high level design is produced for the component-based system.
The high level design specifies the underlying component-based architecture
and development platform(s). When selecting a component architecture, security
functions provided by the architecture need to be considered.
Security requirements for the desired COTS component are elicited in a
functional specification document known as the Security Target (ST) document
in step
(2). The (11) classes of common security requirements identified by the
CC are used to develop the ST document. The use of Protection Profiles
(PP’s),
documents that elicit reusable sets of CC requirements, can help speed
the process of creating ST documentation. A protection profile which identifies
security requirements for secure socket layer (SSL) communication components
can help speed the process of generating ST documentation for a component
search seeking SSL-based communication components.
During step (3) an initial search for candidate COTS components is conducted
using the security requirements identified in the ST document as a filter.
Product documentation (brochures, help files, manuals, etc.) is searched
to identify support for requirements identified in the ST. In the event
no components are found during the initial search, the ST document can
be refined so that COTS components can be identified which state support
for
a base level of security requirements. An alternative to revising the
ST is to abandon the search and develop custom component(s) to meet
security requirements.
In step (4) components are evaluated to eliminate inadequate candidates
from consideration. Variations of the activities in step (4) can be
considered to adapt the selection process to operate under time and
cost constraints.
A CC based evaluation is conducted on the candidate components. Initially
an EAL level 1 evaluation is performed. An EAL level 1 evaluation
identifies (40) evaluation activities. It may only be necessary to perform
a subset
of the evaluation activities. Evaluation activities are halted when
an appropriate component is identified. The following ordering of
evaluation activities
is suggested: ADV_FSP Functional Specification and documentation evaluation,
ADV_RCR evaluation of the component’s correspondence to functional
requirements, ATE_IND independent testing, AGD_ADM Administrator Guidance,
AGD_USR User Guidance, ACM_CAP CM Capabilities, and ADO_IGS Installation,
integration, and start-up. This evaluation suggests performing function
verification and testing activities first since they are most likely
to identify functional inadequacies. If multiple candidates meet all
ST security
requirements after the EAL level 1 evaluation the evaluator can choose
to modify the ST to specify more rigorous security requirements, or
to proceed
with the next EAL level of evaluation. This process of modifying the
ST or advancing to the next EAL level for evaluation is repeated until
all
candidate components except for one is eliminated from consideration.
In the event not all COTS components are eliminated, or when the evaluation
has exhausted time and financial resources a formal decision making
technique
such as the weighted sum method (WSM) or analytic hierarchy process
(AHP) can applied based to aid decision-making.
In step (5) the component that best meets the criteria stated in the
ST is selected as a result of the CC evaluation activities performed
in step
(4). Once selected the component is integrated for use into the
component-based system under development during step (6). Once a component
is in operation
previously unknown errors or vulnerabilities may surface. Required
corrections are identified forcing a revision of the COTS component.
Depending on
the cooperation of the component vendor changes may or may not be
possible. If the component is updated then the security functionality
may need
to be reevaluated. The reevaluation may require a complete revaluation
or
only
a partial evaluation of the changes.
4 RESEARCH APPROACH
By treating components as software systems as in [Orso00] we can use the
CC to specify, and evaluate the security provided by individual components.
Kahn states that all of the requirements of the CC may not be directly applicable
to individual COTS components because of their smaller functional scope
as compared to IT systems [Khan00]. We conducted a survey of COTS components
to discover the classes of CC security requirements that were supported
by commercially available products. Forty-two COTS components, which claimed
to provide security, were located using the Internet component vendor ComponentSource.com.
Components were identified through searches with security related keywords.
On average each COTS component provided security functions from two Common
Criteria classes (~2.2). Support for security requirements of eight of the
eleven CC classes were directly identified through evaluation of product
documentation.
The proposed CC-based COTS component selection process seeks
to make two significant contributions for component selection. The (11)
classes of Common
Criteria security requirements can be used as a baseline for specifying
component security requirements. Developers can develop more complete
security specifications for COTS components before beginning a product search.
The
CC can also help developers evaluate the security provided by candidate
components. By applying the evaluation activities specified by each CC
evaluation assurance level a documented and repeatable assessment of security
can be
achieved. Empirical evaluation of software development processes is generally
considered difficult. Consider evaluations which would measure the success
of various software proceses such as the waterfall model, the spiral model,
or eXtreme Programming. The next step in this research is to conduct an
exploratory investigation to apply the CC-based selection process to investigate
existing research questions. An exploratory investigation will likely
help to identify how to further direct efforts to evaluate and improve the
proposed
CC-based process. Empirical studies as well as case studies may be desired
to fully understand how software developers benefit (or do not benefit)
from using the process. These investigations will evaluate the CC-based
selection processes’ applicability for COTS component selection, as
well identify future process improvements. Investigations should seek to
quantify if the CC-based component selection process provide improvements
in component specification and security evaluation. Future research can
seek to answer: How useful is the CC-based approach in predicting suitability
of components? What difficulties are encountered: specifying security requirements,
evaluating component security? Does the CC-based selection process provide
advantages over existing processes or the ad hoc approach? Which evaluation
activities are most helpful for component selection? Which activities are
time consuming? Which provide the least/most information for component selection
decisions? What are the confusing and difficult parts of the process? What
effort is required to train developers on the use of the process?
REFERENCES
[Bachman00] Bachman F., Bass L., Buhman C., Comella-Dorda S.,
Long F., Robert J., Seacord R., Wallnau K.; Volume II, Technical
Concepts of Component-Based Software Engineering, Technical Report
CMU/SEI-2000-TR-008, Software Engineering Institute, Carnegie Mellon
University, Pittsburgh, PA., 2000.
[Bertoa02] Bertoa M.F., and Vallecillo
A.: “Quality Attributes
for COTS Components.” In Proc. of the 6th ECOOP Workshop on Quantitative
Approaches in Object-Oriented Software Engineering (QAOOSE 2002). Malaga,
Spain, June 2002.
[Bertoa03] Bertoa M.F., Troya J. and Vallecilo A.: “A Survey
on the Quality Information Provided by Software Component Vendors.” In
Proc. Of the 7th ECOOP Workshop on Quantitative Approaches in Object-Oriented
Software Engineering (QAOOSE 2003). Darmstadt, Germany, July 2003.
[Devanbu00] Devanbu P. and Stubblebine S.: Software Engineering
for Security: a Roadmap. In The Future of Software Engineering. Special
volume of the proceedings of the 22nd International Conference on Software
Engineering - ICSE 2000, June 2000.
[ISO99] ISO/IEC-15408 (1999) Common Criteria for Information
Technology Security Evaluation, v 2.1, Nat’l Inst. Standards and Technology,
Washington, DC, August 1999, http://csrc.nist.gov/cc
[Khan00] Khan K.M., Han J., Zheng Y., Characterizing User Data
Protection of Software Components. In Proceedings of the 2000 Australian
Software
Engineering Conference, Gold Coast, Queensland, Australia, April 2000.
[Lindqvist98] Lindqvist U., Jonsson E. A Map of Security Risks
Associated with Using COTS, IEEE Computer, June, 1998, pp. 60-66.
[Myers76] Myers G., Software reliability: principles and practices,
New York, John Wiley & Sons, 1976.
[Orso00] Orso A., Harrold M.J., Rosenblum D., Component Metadata
for Software Engineering Tasks, in Proceedings of EDO 2000, LNCS
Vol. 1999, Springer-Verlag, November 2000.
[Ruhe03] Ruhe G., Intelligent Support for Selection of COTS Products,
in Proceedings of the Net.Object Days 2002, Erfurt, Springer, 2003,
pp. 34-45.
About the author
|
 |
Wes J. Lloyd is presently a PHD Student in the Department
of Computer Science at Colorado State University. His research
interests in software engineering include component based software
development, empirical investigations of software engineering
practices, software processes and requirements engineering. E-Mail:
wlloyd@acm.org
|
Cite this article as follows: Wes J. Lloyd: “A Common Criteria
Based Approach for COTS Component Selection",
in Journal of Object Technology,
vol. 4, no. 3, Special Issue: 6th GPCE Young Researchers
Workshop 2004, pp. 25-32 http://www.jot.fm/issues/issue_2005_04/article4
|
