Machine Learning Engineer - Distributed Training & Research Infrastructure

  • Fintal Partners
  • New York, New York
  • 11/21/2026
Full time Information Technology Telecommunications

Job Description

Job Description

We're partnering with a leading global trading firm to hire a Machine Learning Engineer to build the systems their researchers use to train models at scale. You'll own the infrastructure that turns a research idea into a trained, tested model, and you'll be measured on how fast and how cheaply that happens.


What you'll do

  • Build and scale distributed training pipelines for large models and complex, high-volume datasets
  • Design the parallelism strategy, data loading and checkpointing for multi-node GPU training
  • Automate the research loop with quantitative researchers: experiment orchestration, hyperparameter tuning and model retraining
  • Work with HPC specialists to raise training throughput and GPU utilization, and to cut compute cost
  • Extend the internals of PyTorch and other open-source ML tools when off-the-shelf isn't fast enough
  • Evaluate third-party tools and decide when to build versus buy


What you'll bring

  • 3+ years in machine learning, focused on training systems
  • Hands-on experience scaling training across many GPUs and nodes with PyTorch, JAX or TensorFlow (e.g. FSDP, DeepSpeed, Megatron, DDP)
  • A strong grasp of ML fundamentals, including transformer and attention internals, beyond high-level APIs
  • Production-quality Python, with testing discipline and ownership of the systems you ship
  • A habit of measuring before optimizing: profiling, benchmarking and finding the real bottleneck
  • Clear communication, so you can explain your technical decisions to researchers and adapt when the evidence says to


Nice to have

  • Experience with HPC schedulers (Slurm) and cloud or orchestration platforms (Kubernetes, Ray)
  • Contributions to open-source ML or distributed-systems projects
  • Familiarity with NCCL, collective communication or interconnect tuning


No trading experience is required. What matters is depth in large-scale training systems and the ability to move fast among very smart people.