Description
To support the migration of legacy SAS-based ETL workloads to a modern Python-based data platform leveraging technologies such as Parquet and Spark, we are building an AI-assisted, agent-driven migration factory that supports the analysis, transformation, validation, and generation of Python artefacts throughout the migration lifecycle.
We are looking for a Senior Python Data Engineer with proven experience in both Python data engineering and AI-assisted software development. The consultant will contribute to the delivery of migration solutions while helping to optimize and evolve the agentic workflows, orchestration mechanisms, and validation processes that underpin the migration factory.
The role combines deep Python and data engineering expertise with practical experience in AI-assisted development, agent orchestration and human-in-the-loop validation. The consultant will work closely with data engineers, architects and SAS specialists to improve the quality, scalability and effectiveness of the migration process.
Core Domains of Expertise
1. Python Data Engineering
- Design, develop and validate Python-based data transformation solutions.
- Review and optimize AI-generated Python code.
- Ensure code quality, maintainability, performance and operational robustness.
- Support complex migration cases where automated generation requires refinement.
- Contribute to data quality, schema management and Parquet optimization.
- Support PySpark-based implementations where appropriate.
2. AI-Assisted Development & Agent Harness Engineering
- Act as a power user and contributor to the AI-assisted migration pipeline.
- Help evolve the agent harness responsible for orchestrating specialized agents.
- Improve interactions between agents responsible for: SAS analysis,Functional analysis generation,Technical analysis generation,Python code generation,Validation and testing
- Improve workflow reliability, traceability and throughput.
- Enhance human-in-the-loop validation mechanisms and feedback loops.
- Identify opportunities to increase automation while maintaining quality and control.
Required Skills
Python & Data Engineering
- 5+ years of hands-on Python development.
- Strong knowledge of: Pandas,Polars,PyArrow,Parquet,Testing frameworks
- Experience with:
- Data engineering,ETL / ELT processes,Data quality controls,Schema management
- Experience with PySpark is a strong asset.
AI-Assisted Development & Agentic Systems
- Experience with AI-assisted software development (GitHub Copilot and similar tools).
- Experience with:
- Agentic workflows,Multi-agent systems,Agent harness engineering,Workflow orchestration,Human-in-the-loop validation
SAS (Asset)
- Experience with SAS or the ability to understand legacy SAS programs is a strong asset.
- Understanding of:
- DATA steps,PROC SQL, SAS macros,ETL processing patterns
- Ability to laborate effectively with SAS analysts and subject-matter experts during migration and validation activities.
Ideal Candidate
A senior Python Data Engineer who combines strong software and data engineering capabilities with hands-on experience in AI-assisted development, agentic workflows and agent harness engineering, and who can help industrialize an AI-driven migration factory while remaining actively involved in delivery and solving complex migration challenges.
