Lead Software Engineer

About the role

About The Company

J.P. Morgan is one of the oldest and most prestigious financial institutions globally, with a rich history spanning over 200 years. As a leader in the financial services industry, J.P. Morgan offers innovative solutions across investment banking, consumer and small business banking, commercial banking, financial transaction processing, and asset management. The company serves millions of consumers, small businesses, and some of the world's most prominent corporate, institutional, and government clients under the J.P. Morgan and Chase brands. Its longstanding reputation is built on a commitment to excellence, integrity, and delivering value to its clients through cutting-edge financial products and services.

About The Role

We have an exciting opportunity for a Lead Software Engineer to join the Corporate Technology division at J.P. Morgan. In this pivotal role, you will be an integral part of an agile team dedicated to enhancing, building, and delivering trusted, market-leading technology products. Your responsibilities will encompass designing scalable data architectures, implementing resilient data ingestion pipelines, and driving operational excellence across the firm's data platforms. You will lead efforts to develop and maintain a modern lakehouse architecture in AWS, leveraging Databricks and Delta Lake technologies. Your expertise will help ensure the delivery of secure, stable, and high-performance data solutions that support the firm's strategic objectives. As a leader, you will mentor engineering teams, establish best practices, and collaborate closely with stakeholders to translate business needs into innovative technical solutions.

Qualifications

The ideal candidate will possess formal training or certifications in software engineering and have over five years of hands-on experience across the Software Development Life Cycle. Strong expertise in data engineering, including designing and leading multi-team data platform architectures, is essential. You should have practical experience building and operating Databricks Lakehouses hosted on AWS, with a deep understanding of Delta Lake features such as ACID transactions, schema evolution, and partitioning. Proven skills in Spark performance tuning, cluster management, and optimizing batch and streaming pipelines are required. Familiarity with AWS fundamentals, including S3, IAM, KMS, and networking, is necessary. Experience implementing data governance and security controls within Databricks, such as Unity Catalog and permissions, is also important. Additionally, you should demonstrate proficiency in AI-assisted development tools, responsible AI practices, and ownership of system reliability, including monitoring, incident response, and RCA processes.

Responsibilities

Architect and design the lakehouse architecture, including bronze, silver, and gold data layers, and develop domain-specific data products.

Implement scalable data ingestion pipelines from AWS sources into Databricks, supporting batch and streaming data, including Change Data Capture (CDC) where applicable.

Create maintainable and modular data pipelines using Delta Live Tables and standard jobs, ensuring comprehensive testing and documentation.

Operationalize workloads in production environments through Databricks workflows, ensuring robustness with retries, checkpointing, idempotency, and safe reruns.

Enforce data governance by design, including data classification, auditing, lineage, and controlled sharing, adhering to security best practices.

Drive team adoption of AI-assisted engineering practices to improve code quality, delivery speed, and operational outcomes, including code review, refactoring, and testing standards.

Leverage tools within the SDLC, including AI-powered automation, to enhance development efficiency and quality.

Optimize performance and manage costs by tuning Spark and Delta workloads, right-sizing clusters, and managing storage layouts and job expenses.

Lead and mentor engineering teams, set engineering standards, conduct design reviews, and promote best practices in Spark and Databricks development.

Collaborate with cross-functional teams to translate stakeholder requirements into technical solutions, ensuring alignment with security and platform standards.

Implement CI/CD pipelines and Infrastructure as Code (IaC) using Terraform for Databricks and AWS resources, facilitating seamless promotion across environments.

Develop comprehensive testing strategies, including unit, integration, data quality, and backfill procedures, with disciplined version control and documentation.

Benefits

J.P. Morgan offers a competitive total rewards package, including a base salary commensurate with experience, skill set, and location. Eligible employees may also receive performance-based incentives, including cash bonuses and equity awards, recognizing individual contributions. The firm provides a comprehensive suite of benefits designed to support employee well-being and development, such as health care coverage, on-site wellness centers, retirement savings plans, backup childcare, tuition reimbursement, mental health resources, and financial coaching. Additionally, J.P. Morgan emphasizes work-life balance and professional growth through various programs and initiatives. Details regarding compensation and benefits will be shared during the hiring process.

Equal Opportunity

J.P. Morgan is committed to fostering an inclusive and diverse workplace. We are an equal opportunity employer and do not discriminate based on race, religion, color, national origin, gender, sexual orientation, gender identity, gender expression, age, marital or veteran status, pregnancy, disability, or any other protected attribute under applicable law. We also provide reasonable accommodations for applicants and employees to support their religious practices, mental health, or physical disabilities. Our commitment to diversity and inclusion is fundamental to our success, and we welcome candidates from all backgrounds to join our team.