aws-encryption-sdk¶

https://travis-ci.org/aws/aws-encryption-sdk-python.svg?branch=master

https://ci.appveyor.com/api/projects/status/p3e2e63gsnp3cwd8/branch/master?svg=true

The AWS Encryption SDK for Python provides a fully compliant, native Python implementation of the AWS Encryption SDK.

The latest full documentation can be found at Read the Docs.

Find us on GitHub.

Security issue notifications

Getting Started¶

Required Prerequisites¶

Python 2.7+ or 3.4+
cryptography >= 1.8.1
boto3
attrs

Installation¶

Note

If you have not already installed cryptography, you might need to install additional prerequisites as detailed in the cryptography installation guide for your operating system.

$ pip install aws-encryption-sdk

Concepts¶

There are four main concepts that you need to understand to use this library:

Cryptographic Materials Managers¶

Cryptographic materials managers (CMMs) are resources that collect cryptographic materials and prepare them for use by the Encryption SDK core logic.

An example of a CMM is the default CMM, which is automatically generated anywhere a caller provides a master key provider. The default CMM collects encrypted data keys from all master keys referenced by the master key provider.

An example of a more advanced CMM is the caching CMM, which caches cryptographic materials provided by another CMM.

Master Key Providers¶

Master key providers are resources that provide master keys. An example of a master key provider is AWS KMS.

To encrypt data in this client, a MasterKeyProvider object must contain at least one MasterKey object.

MasterKeyProvider objects can also contain other MasterKeyProvider objects.

Master Keys¶

Master keys generate, encrypt, and decrypt data keys. An example of a master key is a KMS customer master key (CMK).

Data Keys¶

Data keys are the encryption keys that are used to encrypt your data. If your algorithm suite uses a key derivation function, the data key is used to generate the key that directly encrypts the data.

Usage¶

To use this client, you (the caller) must provide an instance of either a master key provider or a CMM. The examples in this readme use the KMSMasterKeyProvider class.

KMSMasterKeyProvider¶

Because the KMSMasterKeyProvider uses the boto3 SDK to interact with AWS KMS, it requires AWS Credentials. To provide these credentials, use the standard means by which boto3 locates credentials or provide a pre-existing instance of a botocore session to the KMSMasterKeyProvider. This latter option can be useful if you have an alternate way to store your AWS credentials or you want to reuse an existing instance of a botocore session in order to decrease startup costs.

import aws_encryption_sdk
import botocore.session

kms_key_provider = aws_encryption_sdk.KMSMasterKeyProvider()

existing_botocore_session = botocore.session.Session()
kms_key_provider = aws_encryption_sdk.KMSMasterKeyProvider(botocore_session=existing_botocore_session)

You can pre-load the KMSMasterKeyProvider with one or more CMKs. To encrypt data, you must configure the KMSMasterKeyProvider with as least one CMK. If you configure the the KMSMasterKeyProvider with multiple CMKs, the final message will include a copy of the data key encrypted by each configured CMK.

import aws_encryption_sdk

kms_key_provider = aws_encryption_sdk.KMSMasterKeyProvider(key_ids=[
    'arn:aws:kms:us-east-1:2222222222222:key/22222222-2222-2222-2222-222222222222',
    'arn:aws:kms:us-east-1:3333333333333:key/33333333-3333-3333-3333-333333333333'
])

You can add CMKs from multiple regions to the KMSMasterKeyProvider.

import aws_encryption_sdk

kms_key_provider = aws_encryption_sdk.KMSMasterKeyProvider(key_ids=[
    'arn:aws:kms:us-east-1:2222222222222:key/22222222-2222-2222-2222-222222222222',
    'arn:aws:kms:us-west-2:3333333333333:key/33333333-3333-3333-3333-333333333333',
    'arn:aws:kms:ap-northeast-1:4444444444444:key/44444444-4444-4444-4444-444444444444'
])

Encryption and Decryption¶

After you create an instance of a MasterKeyProvider, you can use either of the two high-level encrypt/decrypt functions to encrypt and decrypt your data.

import aws_encryption_sdk

kms_key_provider = aws_encryption_sdk.KMSMasterKeyProvider(key_ids=[
    'arn:aws:kms:us-east-1:2222222222222:key/22222222-2222-2222-2222-222222222222',
    'arn:aws:kms:us-east-1:3333333333333:key/33333333-3333-3333-3333-333333333333'
])
my_plaintext = b'This is some super secret data!  Yup, sure is!'

my_ciphertext, encryptor_header = aws_encryption_sdk.encrypt(
    source=my_plaintext,
    key_provider=kms_key_provider
)

decrypted_plaintext, decryptor_header = aws_encryption_sdk.decrypt(
    source=my_ciphertext,
    key_provider=kms_key_provider
)

assert my_plaintext == decrypted_plaintext
assert encryptor_header.encryption_context == decryptor_header.encryption_context

You can provide an encryption context: a form of additional authenticating information.

import aws_encryption_sdk

kms_key_provider = aws_encryption_sdk.KMSMasterKeyProvider(key_ids=[
    'arn:aws:kms:us-east-1:2222222222222:key/22222222-2222-2222-2222-222222222222',
    'arn:aws:kms:us-east-1:3333333333333:key/33333333-3333-3333-3333-333333333333'
])
my_plaintext = b'This is some super secret data!  Yup, sure is!'

my_ciphertext, encryptor_header = aws_encryption_sdk.encrypt(
    source=my_plaintext,
    key_provider=kms_key_provider,
    encryption_context={
        'not really': 'a secret',
        'but adds': 'some authentication'
    }
)

decrypted_plaintext, decryptor_header = aws_encryption_sdk.decrypt(
    source=my_ciphertext,
    key_provider=kms_key_provider
)

assert my_plaintext == decrypted_plaintext
assert encryptor_header.encryption_context == decryptor_header.encryption_context

Streaming¶

If you are handling large files or simply do not want to put the entire plaintext or ciphertext in memory at once, you can use this library’s streaming clients directly. The streaming clients are file-like objects, and behave exactly as you would expect a Python file object to behave, offering context manager and iteration support.

import aws_encryption_sdk
import filecmp

kms_key_provider = aws_encryption_sdk.KMSMasterKeyProvider(key_ids=[
    'arn:aws:kms:us-east-1:2222222222222:key/22222222-2222-2222-2222-222222222222',
    'arn:aws:kms:us-east-1:3333333333333:key/33333333-3333-3333-3333-333333333333'
])
plaintext_filename = 'my-secret-data.dat'
ciphertext_filename = 'my-encrypted-data.ct'

with open(plaintext_filename, 'rb') as pt_file, open(ciphertext_filename, 'wb') as ct_file:
    with aws_encryption_sdk.stream(
        mode='e',
        source=pt_file,
        key_provider=kms_key_provider
    ) as encryptor:
        for chunk in encryptor:
            ct_file.write(chunk)

new_plaintext_filename = 'my-decrypted-data.dat'

with open(ciphertext_filename, 'rb') as ct_file, open(new_plaintext_filename, 'wb') as pt_file:
    with aws_encryption_sdk.stream(
        mode='d',
        source=ct_file,
        key_provider=kms_key_provider
    ) as decryptor:
        for chunk in decryptor:
            pt_file.write(chunk)

assert filecmp.cmp(plaintext_filename, new_plaintext_filename)
assert encryptor.header.encryption_context == decryptor.header.encryption_context

Performance Considerations¶

Adjusting the frame size can significantly improve the performance of encrypt/decrypt operations with this library.

Processing each frame in a framed message involves a certain amount of overhead. If you are encrypting a large file, increasing the frame size can offer potentially significant performance gains. We recommend that you tune these values to your use-case in order to obtain peak performance.

Modules¶

`aws_encryption_sdk`	High level AWS Encryption SDK client functions.
`aws_encryption_sdk.exceptions`	Contains exception classes for AWS Encryption SDK.
`aws_encryption_sdk.identifiers`	AWS Encryption SDK native data structures for defining implementation-specific characteristics.
`aws_encryption_sdk.caches`	Common functions and structures for use in cryptographic materials caches.
`aws_encryption_sdk.caches.base`	Base class interface for caches for use with caching crypto material managers.
`aws_encryption_sdk.caches.local`	Local, in-memory, LRU, cryptographic materials cache for use with caching cryptographic materials providers.
`aws_encryption_sdk.caches.null`	Null cache: a cache which does not cache.
`aws_encryption_sdk.key_providers.base`	Base class interface for Master Key Providers.
`aws_encryption_sdk.key_providers.kms`	Master Key Providers for use with AWS KMS
`aws_encryption_sdk.key_providers.raw`	Resources required for Raw Master Keys.
`aws_encryption_sdk.materials_managers`	Primitive structures for use when interacting with crypto material managers.
`aws_encryption_sdk.materials_managers.base`	Base class interface for crypto material managers.
`aws_encryption_sdk.materials_managers.caching`	Caching crypto material manager.
`aws_encryption_sdk.materials_managers.default`	Default crypto material manager class.
`aws_encryption_sdk.streaming_client`	High level AWS Encryption SDK client for streaming objects.
`aws_encryption_sdk.structures`	Public data structures for aws_encryption_sdk.
`aws_encryption_sdk.internal`	Internal Implementation Details
`aws_encryption_sdk.internal.crypto.authentication`	Contains authentication primitives.
`aws_encryption_sdk.internal.crypto.data_keys`	Contains data key helper functions.
`aws_encryption_sdk.internal.crypto.elliptic_curve`	Contains elliptic curve functionality.
`aws_encryption_sdk.internal.crypto.encryption`	Contains encryption primitives and helper functions.
`aws_encryption_sdk.internal.crypto.iv`	Helper functions used for generating deterministic initialization vectors (IVs).
`aws_encryption_sdk.internal.crypto.wrapping_keys`	Contains wrapping key primitives.
`aws_encryption_sdk.internal.defaults`	Default values for AWS Encryption SDK.
`aws_encryption_sdk.internal.formatting`	Formatting functions for aws_encryption_sdk.
`aws_encryption_sdk.internal.formatting.deserialize`	Components for handling AWS Encryption SDK message deserialization.
`aws_encryption_sdk.internal.formatting.encryption_context`	Components for handling serialization and deserialization of encryption context data in AWS Encryption SDK messages.
`aws_encryption_sdk.internal.formatting.serialize`	Components for handling AWS Encryption SDK message serialization.
`aws_encryption_sdk.internal.str_ops`	Helper functions for consistently obtaining str and bytes objects in both Python2 and Python3.
`aws_encryption_sdk.internal.structures`	Public data structures for aws_encryption_sdk.
`aws_encryption_sdk.internal.utils`	Helper utility functions for AWS Encryption SDK.

Changelog¶

1.4.1 – 2019-09-20¶

Bugfixes¶

Fix region configuration override in botocore sessions. #190 #193

Minor¶

Caching CMM must require that max age configuration value is greater than 0. #147 #172

1.4.0 – 2019-05-23¶

Minor¶

Remove dependence on all source_stream APIs except for read(). #103

Potentially Backwards Incompatible¶

Encryption streams no longer close the source_stream when they themselves close. If you are using context managers for all of your stream handling, this change will not affect you. However, if you have been relying on the StreamDecryptor or StreamEncryptor to close your source_stream for you, you will now need to close those streams yourself.
StreamDecryptor.body_start and StreamDecryptor.body_end, deprecated in a prior release, have now been removed.

Maintenance¶

Move all remaining unittest tests to pytest. #99

Bugfixes¶

Fix MasterKeyprovider.decrypt_data_key_from_list error handling. #150

1.3.8 – 2018-11-15¶

Bugfixes¶

Remove debug logging that may contain input data when encrypting non-default unframed messages. #105

Minor¶

Add support to remove clients from KMSMasterKeyProvider client cache if they fail to connect to endpoint. #86
Add support for SHA384 and SHA512 for use with RSA OAEP wrapping algorithms. #56
Fix streaming_client classes to properly interpret short reads in source streams. #24

1.3.7 – 2018-09-20¶

Bugfixes¶

Fix KMSMasterKeyProvider to determine the default region before trying to create the requested master keys. #83

1.3.6 – 2018-09-04¶

Bugfixes¶

StreamEncryptor and StreamDecryptor should always report as readable if they are open. #73
Allow duck-typing of source streams. #75

1.3.5 – 2018-08-01¶

Move the aws-encryption-sdk-python repository from awslabs to aws.

1.3.4 – 2018-04-12¶

Bugfixes¶

AWS KMS master key/provider user agent extension fixed. #47

Maintenance¶

New minimum pytest version 3.3.1 to avoid bugs in 3.3.0 #32
New minimum attrs version 17.4.0 to allow use of converter rather than convert #39
Algorithm Suites are modeled as collections of sub-suites now #36
Selecting test suites is more sane now, with pytest markers. #41

1.3.3 – 2017-12-05¶

Bugfixes¶

Remove use of attrs functionality deprecated in 17.3.0 #29

Maintenance¶

Blacklisted pytest 3.3.0 #32 pytest-dev/pytest#2957

1.3.2 – 2017-09-28¶

Addressed issue #13 to properly handle non-seekable source streams.

1.3.1 – 2017-09-12¶

Reorganization¶

Moved source into src.
Moved examples into examples.
Broke out internal.crypto into smaller, feature-oriented, modules.

Tooling¶

Added tox configuration to support automation and development tooling.
Added pylint, flake8, and doc8 configuration to enforce style rules.

Maintenance¶

Updated internal.crypto.authentication.Verifier to use Prehashed.
Addressed docstring issue #7.
Addressed docstring issue #8.
Addressed logging issue #10.
Addressed assorted linting issues to bring source, tests, examples, and docs up to configured linting standards.

1.3.0 – 2017-08-04¶

Major¶

Added cryptographic materials managers as a concept
Added data key caching
Moved to deterministic IV generation

Minor¶

Added changelog
Fixed attrs usage to provide consistent behavior with 16.3.0 and 17.x
Fixed performance bug which caused KDF calculations to be performed too frequently
Removed line_length as a configurable parameter of EncryptingStream and DecryptingStream objects to simplify class APIs after it was found in further testing to have no measurable impact on performance
Added deterministic length eliptic curve signature generation
Added support for calculating ciphertext message length from header
Migrated README from md to rst

1.2.2 – 2017-05-23¶

Fixed attrs version to 16.3.0 to avoid breaking changes in attrs 17.1.0

1.2.0 – 2017-03-21¶

Initial public release