Experience
Recent Professional Experience Since 2020
Recent Professional Experience Since 2020
Many engagements and projects have been done repeatedly and indicated accordingly.
Client had a 6-person legal team managing multiple law firms engaged in legal action. Each law firm has unique contracts and submits electronic legal bills requiring compliance verification. The legal team manually spot-checked only 5% of bills due to bandwidth limitations. Client sought to leverage Generative AI to automatically review bills in real-time.
Configured an instance of Anthropic Claude 3.5 Sonnet in AWS Bedrock. Legal contracts were exported to PDF and stored in S3. Multiple Lambda functions in Python handled converting PDF contracts to text using AWS Textract and storing in S3, loading contract text and deriving billing rules via Bedrock inference with results stored in DynamoDB, converting LEDES-formatted bills to JSON in DynamoDB, loading bills and rules and providing to Bedrock with predefined prompts for noncompliance detection in JSON format, and sending email notifications for flagged bills.
Model evaluation ensured correct billing rule extraction and accurate identification of noncompliant line items without false positives. A web application using Python and Flask provided a UI for legal team review. Terraform configured S3 buckets, Lambda functions with CI/CD pipelines, AWS Step Functions state machines, and hosting infrastructure. Multiple bills verified in parallel through isolated workflow executions.
The system now automatically reviews 100% of the incoming legal bills in real-time with immediate cost savings and freed legal team capacity.
Client required a complete inventory of AWS resources across all accounts and regions, refreshed daily, for compliance determination and automatic remediation of noncompliant resources.
A Python application scanned all AWS accounts and regions, checked resources against compliance rules, performed automatic remediations, and generated reports. Deployed as AWS Step Functions State Machine with Python executing in Scan, Evaluation, and Remediation states. Parallel tasks enabled concurrent scanning across accounts and regions. Inventoried resources persisted in ElastiCache (Redis).
Remediations included enabling missing VPC flow logs and Transit Gateway flow logs, enabling Route 53 query logging for hosted zones, applying missing EC2 instance tags, enabling AWS Backup for unprotected resources, and ensuring S3 bucket versioning and encryption.
Architecture was extensible through custom plugins. New remediations required only writing a new Python class automatically discovered by the application. Reports generated in Markdown, text, CSV, and HTML formats. Extensive documentation (100+ pages) detailed design and operation. Several weeks spent mentoring engineers on upgrades and deployments.
The system runs every night in under 1 hour generating comprehensive inventory reports sent to management with automatic remediation of noncompliant resources.
Client had implemented an 80-step workflow using Lambda functions, containers, and EMR jobs coordinated through S3 events, SQS queues, and SNS topics. Recommended redesign using Step Functions for improved maintainability, extensibility, and reliability.
Worked with Data Science team and engineers to identify logical workflow step groupings, implemented in Step Functions as States containing multiple Tasks. Steps executable in parallel were identified to improve overall execution time. Robust error handling included exponential retry and detailed logging. All SQS queues and SNS topics eliminated.
Terraform configuration created a dozen modules deploying the State Machine, all States and Tasks. Configuration used remote state from existing Terraform to import Lambda ARNs and resource identifiers. LucidChart diagrams documented all States and Tasks. Training provided to Data Science team and engineers on Step Functions for future workflow modifications.
The result was a vastly more maintainable and extensible workflow where failed steps could be retried individually rather than requiring full workflow re-execution, with automatic team notification when workflow stalls.
AWS requested Migration Acceleration Program (MAP) assessment for existing customer to determine cloud readiness and outline multi-year migration plan. Initial state consisted of EKS cluster, Aurora Postgres cluster, SQS queues, Lambda functions, and on-premises EKS cluster.
Phase 1: Implemented multi-account AWS best practices in AWS Organizations and Control Tower, including Landing Zone creation, standard OUs, and account restructuring.
Phase 2: Eliminated FIFO queue bottleneck enqueuing multiple identical messages by leveraging WebSockets and API Gateway.
Phase 3: Extended API Gateway use to eliminate direct Internet SQS access, improving security.
Phase 4: Replaced regional SQS queues with DynamoDB global tables and streams, improving scalability and enabling higher availability across new regions.
Phase 5: Migrated from expensive Aurora Postgres cluster to DynamoDB global tables (schema supported this), improving performance, reducing costs, and enabling better scalability and availability.
Phase 6: Enabled 2nd region for all DynamoDB global tables, duplicating infrastructure providing highly-available infrastructure spanning 2 regions with CloudFront distributions before API Gateway and Route 53 latency-based routing with health checks.
Expected monthly cost after Phase 6 was less than the customer's current AWS bill through serverless and capacity-based resources.
Client needed rapid backup and isolation of 2 petabytes of data across 600+ S3 buckets in 100+ AWS accounts and 14 regions supporting urgent business continuity requirements.
All work automated due to repetitive nature of applying identical changes across many S3 buckets. Python scripts performed inventorying all S3 buckets across accounts and regions, applying versioning and encryption to source buckets, creating S3 Batch Operations to encrypt existing unencrypted objects, creating destination buckets in isolated AWS account outside primary Organization in unused region, and applying versioning and encryption to destination buckets.
Additional automation included lifecycle policies moving destination bucket objects to Glacier Deep Storage after 180 days, Object Lock with Compliance enabled on destination buckets with 90-day retention period, S3 replication enabled from all source to destination buckets, and S3 Batch Replication jobs created for every source bucket to replicate existing objects.
All data was successfully replicated over 50 hours, incurring a total client-approved cost of over $100,000 inclusive of data transfer, object lock costs, and AWS charges.
Implemented AWS best practices in multi-account management across six client engagements. AWS Organizations enabled with OU structure designed per AWS best practices accounting for client-specific requirements. Control Tower Landing Zone established with all existing accounts enrolled, resolving enrollment errors.
Clients evaluated AWS Conformance Packs for HIPAA, GDPR, GxP, NIST, PCI-DSS compliance standards, selecting guardrail subsets. Python scripts inventoried AWS resources across accounts and regions predetermining proposed guardrail compliance before selective application to compliant accounts.
Third-party IdP (Azure Active Directory, Okta) federated to AWS SSO (now IAM Identity Center). Terraform configurations managed AWS SSO: creating groups mapping users, creating permission sets and mapping to groups/accounts, creating and applying Service Control Policies to OUs.
Delegated administrator accounts configured for selected AWS services. Tagging policies applied for cost allocation and ownership tracing. Account and resource naming conventions documented with extensive OU structure, guardrail, and compliance requirement documentation provided to executive leadership with final compliance audits.
Implemented AWS best practices in centralized logging across five client engagements. Organization CloudTrail logging enabled to S3 in Log Archive account. Terraform configurations created regional S3 buckets in Log Archive account for additional logs (ALB, CloudFront, etc.) with appropriate bucket policies. Kinesis Firehose resources created for logs requiring them: API Gateway, CloudWatch log forwarding.
S3 events per bucket configured to send messages to SQS queues monitored by SIEM with bucket policies including SIEM permissions for object retrieval. Existing Terraform modules and configurations refactored to send logs to appropriate S3 buckets in Log Archive account per region.
Teams across organizations configured existing infrastructure to forward logs to appropriate S3 buckets. LucidChart diagrams and documentation provided for all buckets and log delivery flows.
Implemented automated self-service AWS account provisioning using Terraform with Jira integration across two client engagements. Custom Issue Type created in Jira environment with fields capturing account name, OU, region, VPC CIDR (when no AWS IPAM available), and optional features.
Custom workflow created with multiple states and transitions allowing Issues approval with state updates as accounts progress through creation, enrollment, and infrastructure provisioning. Jira automations interact with AWS through SNS to start/restart account provisioning.
Multiple Lambda functions in Python: HTTP callbacks to Jira webhooks updating Issue states, receiving account details from Jira Issues starting account creation/enrollment, writing account details to Parameter Store for provisioning workflow steps, and provisioning infrastructure automation IAM roles in newly provisioned accounts.
Containerized Terraform version created fetching configurations from GitHub and applying to new accounts. Terraform configuration deployed Lambda functions with CI/CD pipelines, AWS Step Functions state machines, EventBridge resources, ECS infrastructure, Service Catalog resources, CloudWatch log groups/alarms, and numerous IAM roles.
Created data pipelines populating security-oriented data lakes enabling real-time analytics and alerts across two client engagements. Event data from CloudTrail and S3 ingested and aggregated into S3 buckets preserving original data. Ingestion triggered Lambda functions transforming native structures into Parquet format stored in analytics data lakes.
Real-time data analysis revealed patterns in user S3 object access and AWS resource access with suspicious activity triggering Security team alerts. All Lambda functions in various pipelines written in Python. Terraform configurations provisioned all infrastructure including Lambda functions with associated CI/CD.
Designed and deployed real-time financial data ingestion, transformation, model training, and inference. Worked with quantitative analysts to identify all relevant financial data sources and map them. Feature stores implemented selecting features for predictive models focused on minimizing features to improve performance.
Streaming infrastructure (Kinesis) designed ingesting data from Bloomberg API and other sources with WebSocket subscriptions facilitating real-time ingestion over persistent low-latency connections updating online feature stores in real-time. Sentiment analysis (AWS Comprehend, various FMs) performed in real-time from textual data from web scraping, PDFs, press releases, and news articles.
Model training performed with analysts in SageMaker running on NVIDIA A100 compute resources. Batch training with historical data and online training (fine-tuning) using online feature store data as embeddings with continual online training time reduced to approximately 60 seconds.
Inference endpoint exposed as WebSocket (socket.io) with continual prediction streams allowing subscribed client applications displaying up-to-date trend predictions in real-time. All event-driven architecture resources designed and deployed with Terraform configurations provisioning all infrastructure.
Deployed AWS Rekognition pipeline to flag offensive content. Worked with engineering team to create multiple .NET Lambda functions: copying files from source S3 bucket to scratch bucket, preprocessing files into multiple files (multi-page PDFs processed to one image per page), making API calls to AWS Rekognition determining image offensiveness, and flagging offensive files.
Terraform configuration provisioned source and scratch S3 buckets, Lambda functions, Lambda function CI/CD pipelines, and AWS Step Functions State Machine orchestrating workflow step execution. LucidChart diagram provided showing infrastructure and State Machine states.
Implemented cost-effective multi-account, multi-region networking following AWS best practices across three client engagements. Shared Services account and environment-specific Networking accounts created under Infrastructure OU. Current IP space usage reviewed and documented with new CIDR ranges proposed per region, account, and OU.
Terraform configuration created IPAM pools in Shared Services account. Environment-specific Networking account configurations included Transit Gateway and flow logs, egress VPC, and associated routes per region. Regional Transit Gateway peering configured with other Transit Gateways as needed.
Baseline account configurations for all other accounts created VPCs, networking resources, and attached VPCs to TGWs in environment-specific Networking accounts. Design leveraged egress VPCs to avoid excess NAT gateway provisioning, saving significant costs. Detailed LucidChart diagrams illustrating network design and traffic flows provided.
Containerized existing web services and provisioned associated infrastructure in ECS across more than twenty client engagements. Worked with engineering teams to select base images and create Dockerfiles for web service containerization. Terraform configurations provisioned ECS clusters, services for each web service, task definitions, Application Load Balancers with target groups, listeners, rules, SSL certificates, CloudWatch log groups, and critical metric alarms.
Terraform configurations provisioned ECR repositories and CI/CD pipelines building container images, pushing to ECR repositories, and updating associated ECS services. Engineering teams mentored on Docker and building future applications in containers.
Deployed existing load-balanced web services behind API Gateway and CloudFront across three client engagements. Worked with engineering teams to identify web service endpoints and map them to API Gateway routes. Terraform configurations provisioned APIs in API Gateway including all routes and custom domains.
Terraform configurations provisioned passthrough CloudFront distributions for all domains using API Gateway as origins with CloudFront used purely for reducing API latency with no caching enabled. Extensive documentation and diagrams provided for final infrastructure.
Migrated existing web services via lift and shift to new AWS Workloads accounts across four client engagements. Worked with engineering teams to identify all infrastructure resources associated with migrating services. Terraform configurations provisioned all resources in new Workloads accounts.
Terraform configurations provisioned CI/CD pipelines for building and deploying services in new accounts. Engineering teams worked to migrate production to new services and decommission old infrastructure.
Recreated entire organizational infrastructure using Terraform. All existing infrastructure reviewed and documented using LucidChart diagrams with specific resource attributes documented in Confluence, logically grouped by application and service. Over 900 resources identified including S3 buckets and policies, SQS queues and triggers, SNS topics and subscriptions, EC2 instances, load balancers, Lambda functions, ECS Fargate containers, DynamoDB tables, Amazon EMR workloads, Kinesis data streams, and CloudWatch logs.
CloudOps team led in creating numerous Terraform configurations and modules reproducing existing infrastructure. All configurations deployed and old infrastructure decommissioned post-migration.
Migrated from monolithic Terraform to modern Terraservices model. All application and service-specific resources documented with common infrastructure identified for Terraform module provisioning. CloudOps team created Terraform modules for common infrastructure and created configurations for each application/service leveraging new modules.
New infrastructure (over 3,000 resources) deployed side-by-side with existing infrastructure with migration achieved by changing DNS entries in Route 53 using Terraform. Replaced infrastructure decommissioning managed. CloudOps team mentored on writing Terraform following Terraservices model.
Refactored and centralized CI/CD pipelines from multiple third parties (CircleCI, Jenkins, GitLab) into common, environment-specific CI/CD solution built on AWS services. Separate AWS accounts created under Deployments OU per environment. Terraform configuration deployed required resources in each Deployments account including IAM roles, cross-account policies, S3 artifact buckets, CloudWatch log groups and alarms.
Terraform modules created for provisioning CodePipeline, CodeDeploy, CodeBuild resources with associated IAM roles with cross-account permissions. Numerous build scripts from CircleCI, Jenkins, and GitLab refactored into corresponding workflows using AWS resources. Existing applications and services updated to use new CI/CD pipelines in Deployments accounts.
LucidChart diagrams and documentation provided for new CI/CD infrastructure with developer mentoring on new CI/CD processes.