aws-encryption-sdk¶
The AWS Encryption SDK for Python provides a fully compliant, native Python implementation of the AWS Encryption SDK.
The latest full documentation can be found at Read the Docs.
Find us on GitHub.
Getting Started¶
Required Prerequisites¶
- Python 2.7+ or 3.4+
- cryptography >= 2.5.0
- boto3
- attrs
Installation¶
Note
If you have not already installed cryptography, you might need to install additional prerequisites as detailed in the cryptography installation guide for your operating system.
$ pip install aws-encryption-sdk
Concepts¶
There are four main concepts that you need to understand to use this library:
Cryptographic Materials Managers¶
Cryptographic materials managers (CMMs) are resources that collect cryptographic materials and prepare them for use by the Encryption SDK core logic.
An example of a CMM is the default CMM, which is automatically generated anywhere a caller provides a master key provider. The default CMM collects encrypted data keys from all master keys referenced by the master key provider.
An example of a more advanced CMM is the caching CMM, which caches cryptographic materials provided by another CMM.
Master Key Providers¶
Master key providers are resources that provide master keys. An example of a master key provider is AWS KMS.
To encrypt data in this client, a MasterKeyProvider
object must contain at least one MasterKey
object.
MasterKeyProvider
objects can also contain other MasterKeyProvider
objects.
Master Keys¶
Master keys generate, encrypt, and decrypt data keys. An example of a master key is a KMS customer master key (CMK).
Data Keys¶
Data keys are the encryption keys that are used to encrypt your data. If your algorithm suite uses a key derivation function, the data key is used to generate the key that directly encrypts the data.
Usage¶
EncryptionSDKClient¶
To use this module, you (the caller) must first create an instance of the EncryptionSDKClient
class.
The constructor to this class accepts an optional keyword argument, commitment_policy
, that controls
which algorithm suites can be used for encryption and decryption. If no value
is provided for this argument, a default value of REQUIRE_ENCRYPT_REQUIRE_DECRYPT
is used. Unless
you have specialized performance requirements or are in the process of migrating from an older
version of the AWS Encryption SDK, we recommend using the default value.
import aws_encryption_sdk
from aws_encryption_sdk.identifiers import CommitmentPolicy
client = aws_encryption_sdk.EncryptionSDKClient(
commitment_policy=CommitmentPolicy.REQUIRE_ENCRYPT_REQUIRE_DECRYPT
)
You must then create an instance of either a master key provider or a CMM. The examples in this
readme use the StrictAwsKmsMasterKeyProvider
class.
StrictAwsKmsMasterKeyProvider¶
A StrictAwsKmsMasterKeyProvider
is configured with an explicit list of AWS KMS
CMKs with which to encrypt and decrypt data. On encryption, it encrypts the plaintext with all
configured CMKs. On decryption, it only attempts to decrypt ciphertexts that have been wrapped
with one of the configured CMKs.
Because the StrictAwsKmsMasterKeyProvider
uses the boto3 SDK to interact with AWS KMS,
it requires AWS Credentials.
To provide these credentials, use the standard means by which boto3 locates credentials or provide a
pre-existing instance of a botocore session
to the StrictAwsKmsMasterKeyProvider
.
This latter option can be useful if you have an alternate way to store your AWS credentials or
you want to reuse an existing instance of a botocore session in order to decrease startup costs.
To create a StrictAwsKmsMasterKeyProvider
you must provide one or more CMKs. For providers that will only
be used for encryption, you can use any valid KMS key identifier. For providers that will be used for decryption, you
must use the key ARN; key ids, alias names, and alias ARNs are not supported.
If you configure the the StrictAwsKmsMasterKeyProvider
with multiple CMKs, the final message
will include a copy of the data key encrypted by each configured CMK.
import aws_encryption_sdk
kms_key_provider = aws_encryption_sdk.StrictAwsKmsMasterKeyProvider(key_ids=[
'arn:aws:kms:us-east-1:2222222222222:key/22222222-2222-2222-2222-222222222222',
'arn:aws:kms:us-east-1:3333333333333:key/33333333-3333-3333-3333-333333333333'
])
You can add CMKs from multiple regions to the StrictAwsKmsMasterKeyProvider
.
import aws_encryption_sdk
kms_key_provider = aws_encryption_sdk.StrictAwsKmsMasterKeyProvider(key_ids=[
'arn:aws:kms:us-east-1:2222222222222:key/22222222-2222-2222-2222-222222222222',
'arn:aws:kms:us-west-2:3333333333333:key/33333333-3333-3333-3333-333333333333',
'arn:aws:kms:ap-northeast-1:4444444444444:key/44444444-4444-4444-4444-444444444444'
])
DiscoveryAwsKmsMasterKeyProvider¶
We recommend using a StrictAwsKmsMasterKeyProvider
in order to ensure that you can only
encrypt and decrypt data using the AWS KMS CMKs you expect. However, if you are unable to
explicitly identify the AWS KMS CMKs that should be used for decryption, you can instead
use a DiscoveryAwsKmsMasterKeyProvider
for decryption operations. This provider
attempts decryption of any ciphertexts as long as they match a DiscoveryFilter
that
you configure. A DiscoveryFilter
consists of a list of AWS account ids and an AWS
partition.
import aws_encryption_sdk
from aws_encryption_sdk.key_providers.kms import DiscoveryFilter
discovery_filter = DiscoveryFilter(
account_ids=['222222222222', '333333333333'],
partition='aws'
)
kms_key_provider = aws_encryption_sdk.DiscoveryAwsKmsMasterKeyProvider(
discovery_filter=discovery_filter
)
If you do not want to filter the set of allowed accounts, you can also omit the discovery_filter
argument.
Note that a DiscoveryAwsKmsMasterKeyProvider
cannot be used for encryption operations.
Encryption and Decryption¶
After you create an instance of an EncryptionSDKClient
and a MasterKeyProvider
, you can use either of
the client’s two encrypt
/decrypt
functions to encrypt and decrypt your data.
import aws_encryption_sdk
from aws_encryption_sdk.identifiers import CommitmentPolicy
client = aws_encryption_sdk.EncryptionSDKClient(
commitment_policy=CommitmentPolicy.FORBID_ENCRYPT_ALLOW_DECRYPT
)
kms_key_provider = aws_encryption_sdk.StrictAwsKmsMasterKeyProvider(key_ids=[
'arn:aws:kms:us-east-1:2222222222222:key/22222222-2222-2222-2222-222222222222',
'arn:aws:kms:us-east-1:3333333333333:key/33333333-3333-3333-3333-333333333333'
])
my_plaintext = b'This is some super secret data! Yup, sure is!'
my_ciphertext, encryptor_header = client.encrypt(
source=my_plaintext,
key_provider=kms_key_provider
)
decrypted_plaintext, decryptor_header = client.decrypt(
source=my_ciphertext,
key_provider=kms_key_provider
)
assert my_plaintext == decrypted_plaintext
assert encryptor_header.encryption_context == decryptor_header.encryption_context
You can provide an encryption context: a form of additional authenticating information.
import aws_encryption_sdk
from aws_encryption_sdk.identifiers import CommitmentPolicy
client = aws_encryption_sdk.EncryptionSDKClient(
commitment_policy=CommitmentPolicy.FORBID_ENCRYPT_ALLOW_DECRYPT
)
kms_key_provider = aws_encryption_sdk.StrictAwsKmsMasterKeyProvider(key_ids=[
'arn:aws:kms:us-east-1:2222222222222:key/22222222-2222-2222-2222-222222222222',
'arn:aws:kms:us-east-1:3333333333333:key/33333333-3333-3333-3333-333333333333'
])
my_plaintext = b'This is some super secret data! Yup, sure is!'
my_ciphertext, encryptor_header = client.encrypt(
source=my_plaintext,
key_provider=kms_key_provider,
encryption_context={
'not really': 'a secret',
'but adds': 'some authentication'
}
)
decrypted_plaintext, decryptor_header = client.decrypt(
source=my_ciphertext,
key_provider=kms_key_provider
)
assert my_plaintext == decrypted_plaintext
assert encryptor_header.encryption_context == decryptor_header.encryption_context
Streaming¶
If you are handling large files or simply do not want to put the entire plaintext or ciphertext in memory at once, you can use this library’s streaming clients directly. The streaming clients are file-like objects, and behave exactly as you would expect a Python file object to behave, offering context manager and iteration support.
import aws_encryption_sdk
from aws_encryption_sdk.identifiers import CommitmentPolicy
import filecmp
client = aws_encryption_sdk.EncryptionSDKClient(
commitment_policy=CommitmentPolicy.FORBID_ENCRYPT_ALLOW_DECRYPT
)
kms_key_provider = aws_encryption_sdk.StrictAwsKmsMasterKeyProvider(key_ids=[
'arn:aws:kms:us-east-1:2222222222222:key/22222222-2222-2222-2222-222222222222',
'arn:aws:kms:us-east-1:3333333333333:key/33333333-3333-3333-3333-333333333333'
])
plaintext_filename = 'my-secret-data.dat'
ciphertext_filename = 'my-encrypted-data.ct'
with open(plaintext_filename, 'rb') as pt_file, open(ciphertext_filename, 'wb') as ct_file:
with client.stream(
mode='e',
source=pt_file,
key_provider=kms_key_provider
) as encryptor:
for chunk in encryptor:
ct_file.write(chunk)
new_plaintext_filename = 'my-decrypted-data.dat'
with open(ciphertext_filename, 'rb') as ct_file, open(new_plaintext_filename, 'wb') as pt_file:
with client.stream(
mode='d',
source=ct_file,
key_provider=kms_key_provider
) as decryptor:
for chunk in decryptor:
pt_file.write(chunk)
assert filecmp.cmp(plaintext_filename, new_plaintext_filename)
assert encryptor.header.encryption_context == decryptor.header.encryption_context
Performance Considerations¶
Adjusting the frame size can significantly improve the performance of encrypt/decrypt operations with this library.
Processing each frame in a framed message involves a certain amount of overhead. If you are encrypting a large file, increasing the frame size can offer potentially significant performance gains. We recommend that you tune these values to your use-case in order to obtain peak performance.
Modules¶
aws_encryption_sdk |
High level AWS Encryption SDK client functions. |
aws_encryption_sdk.exceptions |
Contains exception classes for AWS Encryption SDK. |
aws_encryption_sdk.identifiers |
AWS Encryption SDK native data structures for defining implementation-specific characteristics. |
aws_encryption_sdk.caches |
Common functions and structures for use in cryptographic materials caches. |
aws_encryption_sdk.caches.base |
Base class interface for caches for use with caching crypto material managers. |
aws_encryption_sdk.caches.local |
Local, in-memory, LRU, cryptographic materials cache for use with caching cryptographic materials providers. |
aws_encryption_sdk.caches.null |
Null cache: a cache which does not cache. |
aws_encryption_sdk.key_providers.base |
Base class interface for Master Key Providers. |
aws_encryption_sdk.key_providers.kms |
Master Key Providers for use with AWS KMS |
aws_encryption_sdk.key_providers.raw |
Resources required for Raw Master Keys. |
aws_encryption_sdk.materials_managers |
Primitive structures for use when interacting with crypto material managers. |
aws_encryption_sdk.materials_managers.base |
Base class interface for crypto material managers. |
aws_encryption_sdk.materials_managers.caching |
Caching crypto material manager. |
aws_encryption_sdk.materials_managers.default |
Default crypto material manager class. |
aws_encryption_sdk.streaming_client |
High level AWS Encryption SDK client for streaming objects. |
aws_encryption_sdk.structures |
Public data structures for aws_encryption_sdk. |
aws_encryption_sdk.internal |
Internal Implementation Details |
aws_encryption_sdk.internal.crypto.authentication |
Contains authentication primitives. |
aws_encryption_sdk.internal.crypto.data_keys |
Contains data key helper functions. |
aws_encryption_sdk.internal.crypto.elliptic_curve |
Contains elliptic curve functionality. |
aws_encryption_sdk.internal.crypto.encryption |
Contains encryption primitives and helper functions. |
aws_encryption_sdk.internal.crypto.iv |
Helper functions used for generating deterministic initialization vectors (IVs). |
aws_encryption_sdk.internal.crypto.wrapping_keys |
Contains wrapping key primitives. |
aws_encryption_sdk.internal.defaults |
Default values for AWS Encryption SDK. |
aws_encryption_sdk.internal.formatting |
Formatting functions for aws_encryption_sdk. |
aws_encryption_sdk.internal.formatting.deserialize |
Components for handling AWS Encryption SDK message deserialization. |
aws_encryption_sdk.internal.formatting.encryption_context |
Components for handling serialization and deserialization of encryption context data in AWS Encryption SDK messages. |
aws_encryption_sdk.internal.formatting.serialize |
Components for handling AWS Encryption SDK message serialization. |
aws_encryption_sdk.internal.str_ops |
Helper functions for consistently obtaining str and bytes objects in both Python2 and Python3. |
aws_encryption_sdk.internal.structures |
Public data structures for aws_encryption_sdk. |
aws_encryption_sdk.internal.utils |
Helper utility functions for AWS Encryption SDK. |
Changelog¶
2.0.0 – 2020-09-24¶
Features¶
- Updates to the AWS Encryption SDK. 73cce71
Breaking Changes¶
KMSMasterKeyProvider
is removed. Customers must useStrictAwsKmsMasterKeyProvider
with explicit key ids, orDiscoveryAwsKmsMasterKeyProvider
to allow decryption of any ciphertext to which the application has access.- The
encrypt
,decrypt
, andstream
methods in theaws_encryption_sdk
module are removed, replaced by identically named methods on the newEncryptionSDKClient
class. - Key committing algorithm suites are now default.
See Migration guide for more details.
1.7.0 – 2020-09-24¶
Features¶
- Updates to the AWS Encryption SDK. ef90351
Deprecations¶
KMSMasterKeyProvider
is deprecated. Customers should move toStrictAwsKmsMasterKeyProvider
with explicit key ids, orDiscoveryAwsKmsMasterKeyProvider
to allow decryption of any ciphertext to which the application has access.- The
encrypt
,decrypt
, andstream
methods in theaws_encryption_sdk
module are deprecated. Customers should move to the identically named methods on the newEncryptionSDKClient
class.
See Migration guide for more details.
1.4.1 – 2019-09-20¶
1.4.0 – 2019-05-23¶
Minor¶
- Remove dependence on all
source_stream
APIs except forread()
. #103
Potentially Backwards Incompatible¶
- Encryption streams no longer close the
source_stream
when they themselves close. If you are using context managers for all of your stream handling, this change will not affect you. However, if you have been relying on theStreamDecryptor
orStreamEncryptor
to close yoursource_stream
for you, you will now need to close those streams yourself. StreamDecryptor.body_start
andStreamDecryptor.body_end
, deprecated in a prior release, have now been removed.
1.3.8 – 2018-11-15¶
1.3.7 – 2018-09-20¶
1.3.6 – 2018-09-04¶
1.3.5 – 2018-08-01¶
- Move the
aws-encryption-sdk-python
repository fromawslabs
toaws
.
1.3.4 – 2018-04-12¶
1.3.3 – 2017-12-05¶
Maintenance¶
- Blacklisted pytest 3.3.0 #32 pytest-dev/pytest#2957
1.3.1 – 2017-09-12¶
Reorganization¶
- Moved source into
src
. - Moved examples into
examples
. - Broke out
internal.crypto
into smaller, feature-oriented, modules.
Tooling¶
Maintenance¶
- Updated
internal.crypto.authentication.Verifier
to usePrehashed
. - Addressed docstring issue #7.
- Addressed docstring issue #8.
- Addressed logging issue #10.
- Addressed assorted linting issues to bring source, tests, examples, and docs up to configured linting standards.
1.3.0 – 2017-08-04¶
Major¶
- Added cryptographic materials managers as a concept
- Added data key caching
- Moved to deterministic IV generation
Minor¶
- Added changelog
- Fixed attrs usage to provide consistent behavior with 16.3.0 and 17.x
- Fixed performance bug which caused KDF calculations to be performed too frequently
- Removed
line_length
as a configurable parameter ofEncryptingStream
andDecryptingStream
objects to simplify class APIs after it was found in further testing to have no measurable impact on performance - Added deterministic length eliptic curve signature generation
- Added support for calculating ciphertext message length from header
- Migrated README from md to rst
1.2.2 – 2017-05-23¶
- Fixed
attrs
version to 16.3.0 to avoid breaking changes in attrs 17.1.0
1.2.0 – 2017-03-21¶
- Initial public release