Progeny Package Signing Policy

John Goerzen <jgoerzen@progeny.com>

Draft 2; February, 2001

1 Introduction & Background

One weakness commonly associated with installing software off the 
Internet – or even a CD-ROM – is verification of authenticity. 
Can a system administrator really be sure that the packages he 
thinks he is downloading from a Debian or Progeny mirror are 
really official packages? Or that a CD-ROM contains the original 
packages? These days, it is not impossibly difficult to intercept 
network traffic and insert trojan packages in place of real ones, 
or to prepare a fake CD-ROM with a realistic-looking label. To 
thwart this, the ability to add GPG signatures to Debian packages 
is being implemented. These signatures add a cryptographic 
guarantee that the package is authentic in a particular way.

Each package may carry multiple signatures, and each signature 
may convey unique information. For instance, one signature might 
confirm that a legitimate Progeny developer uploaded the package; 
another, that Progeny Quality Assurance has processed the 
package; and a final one might confirm that a package is part of 
a particular release.

On each customer's computer is a file containing rules for 
authentication of these GPG signatures. The rules file describes 
which signatures are required, which are optional, and what keys 
constitute a valid signature for each signature type. Therefore, 
Progeny can ship rules that might say something like this: “For 
each package to be installed, require the Progeny corporate 
signature, a signature from the Quality Assurance team, and 
either a signature from a Progeny developer that uploaded a 
Progeny package or an automated signature indicating that the 
package came from Debian.

1.1 Goals

There are several things that we would like to accomplish with 
this system. Here is a list of many of them, an explanation of 
how we accomplish it, and why it is valuable.

1.1.1 Progeny Developer Verification

The first thing to check on a package is to verify that it was 
built by an authorized Progeny developer. To this end, we will 
have a maintainer (“maint”) signature applied to each package at 
upload time by the person that built it. This key provides 
several benefits:

• It provides a guarantee that the package was not modified by 
  anyone else within Progeny. No malicious Progeny employee could 
  modify the package on its way.

• Customers can know that a package was unmodified from the time 
  it was built.

Additionally, if Progeny ever discovers that a particular 
developer was malicious or dangerously incompetent in the 
building of packages, the packages built by that person can be 
easily identified and can be rendered uninstallable.

1.1.2 Quality Assurance Verification

Another useful point to check on a package is whether or not it 
has passed through Progeny's Quality Assurance department. By 
doing so, we can guarantee that no package bypassed the QA 
department (either by accident or maliciously), and introduce an 
additional level of verification. By having this second signature 
present on the package, we reduce the damage done if the key for 
an individual maintainer were to become compromised in some way. 
Because we have a maintainer signature already, we have already 
prevented any malicious QA employee from modifying the package, 
but if Progeny does at some time find cause to suspect a QA 
person of handling packages in a dangerous way, the key can be 
removed and the packages rendered uninstallable in the same 
manner as with the Debian maintainer signature.

1.1.3 General Malicious Employee Protection

Because Progeny's business is so highly dependent upon secure and 
stable software, if a malicious employee were to be ever hired, 
serious damage could be done. Package signatures and 
compartmentalization can be used together to provide an effective 
weapon to significantly lessen the potential harm that could be 
caused by such a person. In general, the worst harm that could be 
inflicted upon Progeny by a malicious employee would be the 
insertion of trojan or otherwise malicious code into the packages 
that we distribute to customers and use ourselves. The mechanisms 
outlined above can help to combat this threat in several ways.

• A malicious system administrator with full access to the 
  machines on which the archive is served from would ordinarily 
  be able to modify the packages on those machines and insert 
  trojan code virtually undetected. Since the packages had 
  already gone through QA, and probably were already installed on 
  Progeny systems, the change would probably be noticed by 
  customers first. With this system, the maintainer signature is 
  applied before the package is even uploaded to the archive in 
  the first place, so any such change would be immediately 
  noticeable.

• A malicious developer, often also with full access to the 
  archive, could bypass QA to get malicious code into the 
  archive. With this system, no package would be installable 
  unless it bears a signature from a valid QA employee.

• A malicious QA employee could modify the package after it has 
  been received from the developers but before it is placed onto 
  the archive. With this proposal, such a change would be 
  immediately noticed because the maintainer signature check 
  would fail.

1.1.4 General Malicious Outsider Protection

Another category of problems that we have to worry about concern 
malicious outsiders. There are several ways that such a person 
could cause Progeny customers to unknowingly install and run code 
that poses a security risk to them. We can deal with most of 
them.

• Someone could intercept the traffic between a customer's 
  computer and the Progeny archive, inserting data containing 
  trojan code without the customer's knowledge. This is already 
  possible and has been done. By placing signatures on the 
  packages, if they are modified any way during download, the 
  customer would immediately be aware.

• By “social engineering”, that is, convincing someone through 
  non-technical means to install a package, a malicious person 
  could convince one of our customers to install a package that 
  contains a security problem. However, with signature 
  verification, such a package would fail to install.

• An outsider could replace the data in a current package with 
  that from a previous version of a package, perhaps one with 
  known security vulnerabilities. With our archive and release 
  signatures, we can dramatically reduce the potential pool of 
  packages from which such a person could draw a signed Progeny 
  package containing a known security problem.

• A cracker or someone else that gains unauthorized access to 
  Progeny archives could modify the .debs that customers download 
  in-place, giving them code that could pose a security risk. 
  With the mechanisms in this proposal, such a modification would 
  cause multiple signatures to fail, providing an instant way to 
  know that the package had been modified.

1.2 Unaddressed Issues

While this proposal can deal with a great many potential problems 
that could occur, there are still several problems that cannot be 
dealt with solely within the scope of this system. Here are some 
of them that we should be aware of.

• A cracker obtaining root maliciously on a customer's machine 
  could bypass all of the checks that are in place here. Such a 
  person could install fake versions of the verifier that could 
  report false positives. In general, if root is compromised, we 
  can no longer make any assumptions about the security of the 
  customer's machine.

• A cracker or internal person that has obtained root on a 
  Progeny developer's machine could alter files such that 
  packages built by that developer would contain trojan code that 
  the developer may not be aware of. Hopefully, this would be 
  detected during the QA process, but there is no guarantee of 
  that. Such a cracker could also replace GPG with a program that 
  could surreptitiously gather the developer's secret key and 
  passphrase, causing a compromise of the developer's secret key. 
  Again, once someone has obtained root, we can no longer make 
  any assumptions about the security of the machine.

• We must have a secure way to get the initial keyring and 
  software to the customer. This is a question of how far we go 
  to make the guarantee. Some options include placing the 
  software behind a HTTPS site that bears a signature from a 
  well-known certifying authority. However, if someone's web 
  browser has already been modified by a malicious person, even 
  this would not guarantee success. Another option is to 
  distribute the initial software and keys via CD-ROM shipped 
  directly from our office, though once again, a malicious person 
  could prepare a fake CD-ROM bearing a realistic-looking label 
  and ship it to a customer. These are questions that we have to 
  deal with outside the framework of the package signing software 
  itself, and basically rely upon striking an acceptable balance 
  between paranoia and ease of use.

2 Criteria For Verification

The verification process is that which is performed on a 
customer's machine when a package is to be installed. 
Verification software has configurable criteria to indicate what 
signatures are necessary to consider a package to be valid. While 
the verification policy is up to each individual system 
administrator, we must supply a default. A package from Progeny 
will consider to pass the tests if all of these criteria are met:

• The package must bear an “origin” signature made by a 
  designated origin key.

• The package must bear a quality assurance (“qa”) signature made 
  by a member of the Progeny QA team.

• The package must have a maintainer (“maint”) signature from 
  either:


  – A member of the Progeny development team; or

  – An automated signature applied to packages imported 
    unmodified from Debian.

Possible modifications that we could consider making would be to 
change some of the logic from “and” to “or”, thus permitting 
installation even if one signature is missing or somehow invalid. 
This could make dealing with situations involving lost keys, 
changing employees, and upgrades easier, but comes at the expense 
of opening up some potential malicious employee attacks.

3 Procedures For Signing

Keys can be added to packages during the development cycle. At 
each step, we should ensure that all existing keys on the package 
are valid. The first signature on a package will be added by 
either the Progeny developer responsible for it or the system 
that is installing the package from Debian. The next signature 
will be added by the person in QA that approves the package. 
After that, the origin signature should be added, and finally, 
when the package is placed into the archive, an archive signature 
should be added. All signatures except automated ones are to be 
made by personal keys. Progeny will maintain keyrings of valid 
developer and QA keys. Below, the meaning and procedures for each 
Progeny signature are described.

3.1 Maintainer Signature

The maintainer signature is named “maint”. It is to be applied to 
any package uploaded into the archive by a Progeny developer. The 
maintainer signature is made with that developer's own GPG key. 
Its purpose is to let the customer verify that the package was 
indeed worked on by an authorized Progeny employee.

Many packages (a majority, in fact) appear in the Progeny 
distribution directly from Debian with no modification or 
intervention from Progeny employees. There is no real Progeny 
developer or uploader in charge of these packages. For these 
situations, a special key should be generated. A signature from 
that key should be automatically added to packages that come 
through from Debian. The username and description for that key 
should make clear that it does not originate from a particular 
Progeny developer.

3.2 Quality Assurance

As packages move into the distribution, we anticipate that they 
will be screened by QA. To that end, a “qa” signature will be 
made for the package. It will be made using the GPG key of the 
specific QA employee that approves the package. Any package 
bearing a valid QA signature can thus be assumed to have passed 
their testing for the distribution. There should be no case in 
which this key is added automatically. An exception may need to 
be made for initial bootstrapping; that is, dealing with the 
thousands of packages already in the distribution prior to QA 
procedures.

3.3 Origin

The only key that must be present in the debsigs system is “
origin”. This key certifies the originator of the package (in 
this case, Progeny Linux Systems) and lets the signature verifier 
load the appropriate ruleset for the verification of signatures. 
This key will probably be added automatically, using our 
corporate key or some designated origin key that bears its 
signature.

4 Safeguarding Keys

The task of safeguarding individual keys is already well-known, 
understood, and easily manageable. However, safeguarding keys 
that are used by many people or to apply automated signatures 
gets more difficult. In some cases, the security of these is not 
as important – for instance, an archive key will not inflict much 
damange if compromised. However, in other cases, the security is 
still quite important.

Safeguarding of individual keys is a well-known problem with 
well-documented answers. To provide a comprehensive system, there 
are also other cases that we need to consider, which are spelled 
out below.

5 Special Cases & Procedures

While the policy laid out here will work smoothly for day-to-day 
operation, things can go wrong or get confused and planning for 
them is a must. In this section, I describe some special cases 
that can face us, what they mean for the package signing system, 
and how we can prepare for them ahead of time and deal with them 
when they occur.

5.1 Employee Leaves

If a Progeny employee leaves the company for some reason, and 
this person was someone whose key was used to create signatures 
for packages that are used by customers, then Progeny will need 
to decide, on a case-by-case basis, several things:

• Do we have any reason to believe that the packages in the 
  archive now bearing this person's signature are in any way 
  damaged or tampered with?

• Do we have any reason to believe that this person may, in the 
  future, intend to inflict harm on Progeny through the use of 
  the key used to apply those signatures?

If the answer to the first question is “yes”, then we would 
immediately need to remove the key of this employee from all 
keyrings and rebuild the effected packages. If the answer to the 
second question is “yes”, then we would need to immediately 
remove this person's key from all internal keyrings (such that 
the maintainer signature check made by QA would fail if somehow 
the person uploads a package after leaving the company), but 
removing the key from the external keyrings is not a priority and 
can be done at a more leisurely pace. If both answers are “no”, 
then Progeny should, as a precaution, rebuild packages signed by 
that user eventually, but there is no need to do so before the 
next release.

5.2 People Join Progeny

When a new person is hired for a development or QA position, this 
person will need to generate a personal key and have it inserted 
onto the keyring. Customers will not be able to verify a 
signature made by this new person until they have upgraded their 
Progeny keyring package. Once that is done, packages will be able 
to verify with no difficulty. We may need to ensure that the new 
keyring package gets installed early in any upgrade procedure, 
but beyond that, there is little that we must do specially in 
this case. Note, though, that an existing employee must be the 
one to sign the new keyring package.

5.3 Many Employees Leave

In the event that enough employees from a particular department 
leave the company such that there no longer exists anyone with a 
key that can make a valid signature for a particular category, we 
need to have a way to allow customers to continue receiving 
updates in a secure fashion. For this purpose, I suggest that 
Progeny create a generic key for both the QA and maintainer 
categories, store it in some physically secure area (such as a 
safe or deposit box) on removable media in a fashion that only 
authorized company officers could get the key in an emergency. 
The public key would be inserted on the distributed keyring 
files, but would never be used unless absolutely necessary.

5.4 Key Compromises

If a key becomes compromised (that is, an intruder obtains the 
secret key and passphrase, enabling unauthorized signatures), 
then we must take steps to rectify the problem immediately. 
Depending on the type of key that was compromised and the 
circumstances, we might opt to immediately remove the compromised 
key from all keyrings and rebuild any effected packages. In all 
cases, the victim of the compromise should issue a revocation 
certificate, discard the old key, and generate a new one for 
future use.

5.5 Compartmentalization

One of the benefits of having multiple signatures on a package is 
to reduce the damage that can be inflicted if one of those keys 
becomes untrusted in some fashion. For this reason, we should 
make sure that we do not have overlap between those that make 
signatures for different parts of the operation. Specifically, 
nobody should be both on the maintainer and the QA keyring.

6 Future Options

In the course of preparing this policy, some ideas were discussed 
but discarded for the moment. This section will explain them as 
possible options for future use.

6.1 Goal: Release & Dating of Packages

One potential problem that could arise for a customer is that a 
malicious person could present an old package as if it were 
current. If this old package contains problems, such as known 
security vulnerabilities, doing so could be an avenue for attack. 
Preventing this problem entirely is not really feasible; however, 
we can significantly lessen the potential vulnerability by 
providing a limited “window” of time for which we consider a 
particular package valid.

For this situation, we must consider CD-ROM and online 
distribution separately. For online distribution, we can apply an 
“archive” signature, automatically, to each package in the 
archive. Because all signatures contain a timestamp indicating 
when they were made, the signature verifier on the customer's 
machine can ensure that the signature was placed on the package 
within a certain timeframe – for example, two weeks. On our side, 
we would automatically re-sign all packages with the archive key 
periodically – in this example, perhaps once a week. In this way, 
a customer installing a package knows that it is no more than two 
weeks out of date. Packages distributed via CD-ROM will either 
not have the archive signature or will have it ignored.

This solution won't work for a CD-ROM because they are frequently 
used more than two weeks from the release and because there is no 
way to propagate signature updates to people using them. 
Therefore, the best we can do is guarantee the customer that a 
given package is indeed part of the release that is expected – 
that is, if Progeny 2.1 is current, that a given package was 
released as part of Progeny 2.1 and does not date back to version 
1.1. Another benefit of the release signature is that it can be 
virtually uncompromisable. The release signature can be applied 
to the packages at release time, and then the key behind it can 
be destroyed. However, as a practical matter, we will need to 
publish at least one future release key in the current keyring so 
that people upgrading will be able to do so in a secure fashion.

6.1.1 Archive

The “archive” signature is to be automatically applied by 
dinstall when a package is installed into Progeny's 
publically-accessible archive, regardless of the particular area 
that it is installed into. A valid signature can be used to 
assure that a package was indeed posted by Progeny on a specific 
date.

6.1.2 Release

The release key should be kept in a secure, safe place accessible 
only by authorized company officers prior to use. After it has 
been used, it should be permanently destroyed. Release keys will 
need to be generated at least one release ahead of use to provide 
a suitable upgrade path for users.

