Lead Software Engineer

About the role

About The Company

J.P. Morgan, one of the most established and renowned financial institutions globally, has a rich history spanning over 200 years. As a leader in the financial services industry, J.P. Morgan offers innovative solutions across investment banking, consumer and small business banking, commercial banking, financial transaction processing, and asset management. Serving millions of consumers, small businesses, and some of the world's most prominent corporate, institutional, and government clients, the firm is committed to delivering excellence, stability, and security in all its operations. With a focus on technological innovation and responsible banking practices, J.P. Morgan continuously invests in its digital infrastructure to maintain its competitive edge and meet the evolving needs of its clients and stakeholders.

About The Role

We have an exciting opportunity for a Lead Software Engineer to join the Corporate Technology team at J.P. Morgan. In this role, you will be an integral part of an agile development environment, responsible for designing, building, and maintaining cutting-edge data platforms that support the firm's strategic objectives. You will lead efforts to develop scalable, secure, and reliable data solutions within the cloud ecosystem, primarily focusing on the Databricks Lakehouse architecture hosted on AWS. Your expertise will drive the implementation of resilient data ingestion pipelines, data governance frameworks, and operational excellence initiatives, ensuring the delivery of trusted data products that enable informed decision-making across the organization. As a senior technical leader, you will also mentor junior engineers, uphold engineering standards, and collaborate with cross-functional teams to translate complex business requirements into effective technical solutions.

Qualifications

The ideal candidate will possess formal training or certification in software engineering, coupled with over five years of hands-on experience in the full Software Development Life Cycle. Demonstrated expertise in data engineering, particularly in designing and leading multi-team data platform initiatives, is essential. Candidates should have practical experience building and operating Databricks Lakehouse environments within AWS, with a deep understanding of Delta Lake features such as ACID transactions, schema evolution, and partitioning. Strong proficiency in Spark performance tuning, cluster management, and optimizing batch and streaming pipelines is required. Familiarity with AWS fundamentals including S3, IAM, KMS, and networking is necessary, along with experience implementing data governance and security controls like Unity Catalog. Leadership skills in AI-assisted software development tools, responsible AI practices, and reliable incident management are highly valued. Certifications in cloud services, data engineering, or related fields will be advantageous.

Responsibilities

Architect the lake house platform by designing bronze, silver, and gold data layers, as well as domain-specific data products to facilitate scalable analytics and data science initiatives.

Implement resilient data ingestion pipelines at scale, utilizing batch and streaming methods within AWS sources into Databricks, including change data capture (CDC) where applicable.

Develop maintainable, modular data pipelines using Delta Live Tables or standard jobs, ensuring comprehensive testing, documentation, and adherence to best practices.

Operationalize workloads with Databricks workflows and jobs, ensuring robustness through retries, checkpointing, idempotency, and safe re-execution strategies.

Enforce data governance by design, including data classification, access controls, auditing, lineage, and metadata management, maintaining compliance with security standards.

Drive the adoption of AI-assisted engineering practices within the team to enhance code quality, accelerate delivery, and improve operational outcomes. Promote secure coding, peer reviews, automated testing, and reuse of effective patterns.

Leverage tools within the Software Development Life Cycle, including enterprise AI-assisted development and automation capabilities, to maximize automation benefits.

Optimize performance and cost efficiency by tuning Spark/Delta workloads, right-sizing clusters, and managing storage layouts and job expenses.

Lead and mentor engineering teams by establishing standards, conducting design reviews, and upskilling team members on best practices in Spark and Databricks.

Translate stakeholder requirements into technical plans, communicate trade-offs effectively, and collaborate with security and platform teams for seamless integration.

Implement CI/CD pipelines using Terraform for Databricks and AWS resources, ensuring smooth promotion across environments.

Develop comprehensive testing strategies, including unit and integration tests, data quality checks, contract testing, and backfill procedures, along with maintaining detailed documentation and runbooks.

Benefits

J.P. Morgan offers a competitive total rewards package that reflects the importance of our employees. Compensation is determined based on role, experience, skill set, and location, with opportunities for performance-based bonuses and discretionary incentives paid in cash or equity. Our benefits program includes comprehensive health care coverage, on-site wellness centers, a retirement savings plan, backup childcare, tuition reimbursement, mental health support, and financial coaching. We are committed to supporting our employees' well-being and professional growth through ongoing development programs and a supportive work environment. Additional benefits and details will be discussed during the hiring process, emphasizing our dedication to fostering a rewarding and inclusive workplace.

Equal Opportunity

J.P. Morgan is an equal opportunity employer that values diversity and inclusion. We do not discriminate based on race, religion, color, national origin, gender, sexual orientation, gender identity, gender expression, age, marital or veteran status, pregnancy, disability, or any other protected attribute under applicable law. We are committed to providing reasonable accommodations for applicants and employees to support their religious practices, mental health, or physical disabilities.