Technical
Python Plus AWS SDK: The Boto3 Patterns I Actually Use
Boto3 is the AWS SDK for Python. It is also a big surface area with a lot of ways to shoot yourself in the foot. After six months of daily boto3 use in production, I have settled on a short list of patterns that work. Everything else I avoid.
Use Resources, Not Clients (Mostly)
Boto3 has two abstractions: clients (low-level, one method per API call) and resources (higher-level, object-oriented). I default to resources for the basics and reach for clients when I need control:
import boto3
table = boto3.resource('dynamodb').Table('posts')
table.put_item(Item=payload) # clean
ddb = boto3.client('dynamodb')
ddb.transact_write_items(TransactItems=[...]) # need client for thisResources handle pagination, serialization, and most of the tedious bits. Clients are needed for features the resource layer does not expose, like transactions or batch operations at scale.
Single Client per Process
Creating a boto3 client is expensive. Each one opens connection pools and loads configuration. I create clients at module load, never inside request handlers:
# module load
DDB = boto3.resource('dynamodb')
POSTS = DDB.Table('posts')
def handler(event):
return POSTS.get_item(Key={'slug': event['slug']})In Lambda that means the client is reused across warm invocations. Cold starts pay once, warm invocations are free.
Always Paginate
Almost every list or scan operation returns a page, not the full result set. The paginator abstraction handles that cleanly:
paginator = ddb_client.get_paginator('scan')
for page in paginator.paginate(TableName='posts'):
for item in page['Items']:
process(item)Forgetting to paginate is how you get silent data truncation at 1MB. Always use the paginator.
Handle ClientError Specifically
Boto3 raises ClientError for most AWS failures. The error code lives in a nested dict:
from botocore.exceptions import ClientError
try:
table.put_item(Item=payload, ConditionExpression='attribute_not_exists(slug)')
except ClientError as e:
if e.response['Error']['Code'] == 'ConditionalCheckFailedException':
return already_exists()
raiseCatching ClientError broadly and branching on the code is how you get clean business logic without losing visibility into which specific failure happened.
Read the boto3 documentation for the full surface area.
RELATED READING
The Consulting Shift I Am Making In Year Two
After a year of writing and building, my consulting practice is changing shape. Shorter engagements. Sharper outcomes.
ReadThe Frontend Shift: Shipping Less JavaScript In Year Two
A year ago I reached for Next.js for everything. This year I often reach for nothing.
ReadThe Serverless Lesson I Would Write On A Sticky Note
After a year of shipping serverless projects, one rule explains most of the wins and all of the losses.
Read