Before going live or "into production", changes to applications must go
through a series of different types of testing.
- Integration Testing – the changes are integrated into the larger
application and appropriately tested
- Functional Testing - formal test scripts (ie documents containing
test cases, or step-by-step guides to verify functional requirements) are
executed by testers
- Acceptance Testing – formal or informal testing to ensure that
functional requirements are valid
- Quality Testing – formal or informal testing to ensure that
non-functional requirements (regulatory, performance, etc) are met
- Staging Testing – verification that the software can be deployed to
an environment that matches the production environment
To ensure a distinct and efficient testing process, these tests often occur
in different application environments. Each of these environments represents the
production environment to some extent, though often with less of a hardware
Although an organization may have any number of environments, they will all
be one of the seven types:
- Development – A fairly chaotic sandbox to which all (or most)
developers have access and where development is actually done. Examples: a
shared development database; a web application in which developers edit files
in a shared directory on the server.
- Integration – An environment "owned" by development, but in which
access is controlled to a select group of developers. Its purpose is to allow
developers to verify that code runs as they expect it in an environment
similar to production.
- Testing - Often owned by development, Testing is used to execute
formal test cases (performed by analysts or testers) to ensure that the
software implements the business specifications as expected.
- User Acceptance – Occasionally owned by development, User
Acceptance is used (formally or informally) by the end-users to ensure that
the business specifications, as implemented by the software, meet the actual
- Quality Assurance – Used by QA professionals (occasionally outside
the organization) to verify that the software meets appropriate corporate,
regulatory, compliance, and performance standards.
- Staging – Owned by network operations, Staging is an almost-exact
replica of the production environment (including production data) and is used
only to verify and "practice" a deployment to production.
- Production – The final destination of the software.
In BuildMaster, there is no strict enforcement of how environments are set-up
and defined. Environments do, however, serve two key purposes.
- Workflow definition – Each application defines the workflow (ie
sequence of environments that changes must be tested in prior to production)
- Security – privileges may be defined by environment. As such,
access to things that are tied to a specific environment (production
configuration file, deployment plans, etc), may be revoked.
Environments also serve as a way to group servers into logical groups.
When setting up environments in Build Master, we do not recommend creating
separate, similarly-purposed environments for two different applications. For
example, even if two applications utilize entirely different sets of servers,
there should be a process-consistent name created (Development, Integration,
Test, Production) instead of application-specific names (Dev-App1, Dev-App2,
Even though the servers groups may be completely different, they represent
the same stages of development and, in order to keep a consistent nomenclature
throughout the application portfolio, should use the same names. Otherwise, as
the software development organization grows and changes, there will be a
veritable soup of applications names and environment names that will be
difficult to keep track of.
From a security standpoint, access to all applications' production
environments may be restricted, or only some applications' environments. This is
accomplished by adding an environmentally-scoped privilege (see "Security &
Privileges") for that specific environment.
This content has the following tags: