Sean Segal

I'm currently a research scientist at Uber ATG R&D where I work on deep learning research for the development of self-driving cars. I'm also a PhD student in Computer Science at the University of Toronto supervised by Professor Raquel Urtasun.

I obtained my Masters in Computer Science at University of Toronto in 2020, supervised by Professor Raquel Urtasun. Prior to that, I did my Bachelor's at Brown University studying Computer Science and Economics.

Email  /  GitHub  /  Google Scholar  /  LinkedIn  /  Twitter

profile photo


project image

Universal Embeddings for Spatio-Temporal Tagging of Self-Driving Logs

Sean Segal, Eric Kee, Wenjie Luo, Abbas Sadat, Ersin Yumer, Raquel Urtasun
Conference on Robot Learning (CoRL), 2020 (Oral, Best Paper Presentation Finalist)
arXiv / pdf / video / live presentation

Spatio-temporal tagging of self-driving scenes from raw sensor data.

project image

End-to-end Contextual Perception and Prediction with Interaction Transformer

Lingyun (Luke) Li, Bin Yang, Ming Liang, Wenyuan Zeng, Mengye Ren, Sean Segal, Raquel Urtasun
International Conference on Intelligent Robots and Systems (IROS), 2020 (Oral)
arXiv / pdf

Adapting the Transformer to model multi-agent interactions in trajectory prediction.

project image

Discrete Residual Flow for Probabilistic Pedestrian Behavior Prediction

Ajay Jain, Sergio Casas, Renjie Liao, Yuwen Xiong, Song Feng, Sean Segal, Raquel Urtasun
Conference on Robot Learning (CoRL), 2019
arXiv / pdf / video

Multimodal, long-range pedestrian prediction through probabilistic occupancy maps.


Here are some projects I've worked on over the years. These include coursework, side projects, and work I've done during internships.

project image

Carla RL Gym


OpenAI Gym environment for the CARLA self-driving simulator with implementations of multiple common RL baselines (e.g., A2C, ACKTR, PPO). Originally written as part of a final project for Prof. Jimmy Ba’s deep reinforcement learning course. Now has 50+ stars on Github!

project image

Javascript Imports LSP Server


An LSP server for Javascript imports, providing Diagnostics, Code Actions and Autocompletion for Javascript Imports in any LSP-compatible IDE. Written during my time at Facebook on the Nuclide team. Most of the code is open source and can be found here.

project image

T.I.N.C is not Catan

website / GitHub

Fully functional Settlers of Catan emulator online. Also offers advanced economic features including decimal resource counts and dynamic port exchange rates based on resource supply & demand. Play Online Now!

project image

Simplex Solver


Solves Linear Programs in Python using the Simplex Method. Originally written as an assignment for Optimization Methods for Finance course at Brown University.


Brown University

Spring 2018 - Head TA for CS1420 (Machine Learning) with Prof. Michael Littman

Fall 2017 - TA for CS1470 (Deep Learning) with Prof. Eugene Charniak

Spring 2017 - TA for CS0320 (Software Engineering) with Prof. John Jannoti

Fall 2016 - TA for CS0330 (Computer Systems) with Prof. Tom Doeppner

Spring 2016 - TA for CS0004 (Scientific Computing) with Prof. Dan Potter

Design and source code from Jon Barron's website