I want to make systems that learn visual features from all the data (video/3D/hands/etc) and model the world and actions. I think there are untapped sneaky self-supervised signals out there that will let us represent scenes more compositionally, beyond just next token prediction. In the past, I have used satellite imagery of the sun to better estimate the solar magnetic field.


SuperSynthIA: Physics-Ready Full-Disk Vector Magnetograms from HMI, Hinode, and Machine Learning
Ruoyu Wang, David Fouhey, Richard E.L. Higgins, Spiro K. Antiochos, Graham Barnes, J. Todd Hoeksema, K.D. Leka, Yang Liu, Peter W. Schuck, Tamas I. Gombosi
Astrophysical Journal, Accepted 2024 bibtex

‣ A new method for combining multiple instruments to produce magnetograms useful for downstream systems.

paper / site / github
Towards A Richer 2D Understanding of Hands at Scale
Tianyi Cheng*, Dandan Shan*, Ayda Sultan, Richard E.L. Higgins, David Fouhey
NeurIPS 2023, bibtex

‣ A new dataset, tasks, and model for understanding more complex hand interactions, including bimanual manipulation and tool use.

MOVES: Manipulated Objects in Video Enable Segmentation
Richard E.L. Higgins, David Fouhey
CVPR 2023, bibtex

‣ We use disagreement from a background motion model as a pseudolabel to train hand and held-object grouping and association.

paper / site / github
EPIC-KITCHENS VISOR Benchmark: VIdeo Segmentations and Object Relations
Ahmad Darkhalil, Dandan Shan, Bin Zhu, Jian Ma, Amlan Kar, Richard E.L. Higgins, Sanja Fidler, David Fouhey, Dima Damen
NeurIPS 2022, bibtex

‣ We made EPIC-KITCHENS VISOR, a new dataset of annotations for segmenting hands and active kitchen objects in egocentric video.

paper / site
On Identifying and Mitigating Bias in Inferred Measurements for Solar Vector Magnetic-Field Data
K.D. Leka, Eric L. Wagner, Ana Belén Griñón-Marín, Véronique Bommier, Richard E.L. Higgins
Solar Physics, July 2022 bibtex

‣ I trained a neural network to produce synthetic magnetic field inversions and we used the outputs to fix (ongoing) biases in the satellite processing pipeline.

paper
COHESIV: Contrastive Object and Hand Embeddings for Segmentation In Video
Richard E.L. Higgins*, Dandan Shan*, and David F. Fouhey
NeurIPS 2021, bibtex

‣ We built a system that predicts segmentation masks for objects held by hands. The system trains from person, object, and background pseudolabels made by subtracting detected people from optical flow.

paper
SynthIA: A Synthetic Inversion Approximation for the Stokes Vector Fusing SDO and Hinode into a Virtual Observatory
Richard E.L. Higgins, David F. Fouhey, Spiro K. Antiochos, Graham Barnes, Mark C.M. Cheung, J. Todd Hoeksema, KD Leka, Yang Liu, Peter W. Schuck, Tamas I. Gombosi
Astrophysical Journal Supplement Series, March 2022 bibtex
SDO Science Seminar, Invited Talk 2021

‣ Hinode's SOT-SP measures small areas of the sun in high spatial and spectral resolution to predict the magnetic field. SDO/HMI measures the full-disk in lower resolution, both spatial and spectral.
‣ By training a neural network to accurately predict Hinode's estimated field using only HMI's input, we created a virtual observatory that melds the best parts of both instruments.

paper
Fast and Accurate Emulation of the SDO/HMI Stokes Inversion with Uncertainty Quantification
Richard E.L. Higgins, David F. Fouhey, Dichang Zhang, Spiro K. Antiochos, Graham Barnes, Todd Hoeksema, KD Leka, Yang Liu, Peter W. Schuck, Tamas I. Gombosi
Astrophysical Journal, April 2021 bibtex
AGU, ML in Space Weather, Poster 2020
COSPAR2021, Workshop on ML for Space Sciences, Talk 2021

‣ I trained a UNet to predict magnetic field parameters on the sun using polarized light (IQUV's) recorded from the Solar Dynamics Observatory's HMI sensor.

paper / site / github / talk / poster
Network Reconstruction Reveals that Valproic Acid Activates Neurogenic Transcriptional Programs in Adult Brain Following Traumatic Injury
Gerald A. Higgins, Patrick Georgoff, Vahagn Nikolian, Ari Allyn-Feuer, Brian Pauls, Richard Higgins, Brian D. Athey, and Hasan E. Alam
Pharmaceutical Research, August 2017 bibtex

‣ I constructed topologically associated domains and analyzed RNA-seq data to identify differential gene expression using bioinformatics libraries in R and Python.

Matrix Metalloproteinase-9 Regulates Neuronal Circuit Development and Excitability
Sachiko Murase, Crystal Lantz, Eunyoung Kim, Nitin Gupta, Richard Higgins, Mark Stopfer, Dax A. Hoffman, and Elizabeth M. Quinlan
Journal of Molecular Neurobiology, July 2016 bibtex

‣ I wrote MATLAB functions that classified windows of mouse EEG recordings as seizure/not seizure with max-margin unsupervised learning.

Visualizing What Cichlids with Different Cone Opsins See
Richard Higgins, Karen Carleton
Undergrad Resarch Project, 2012

‣ I wrote a Java application that changed underwater images into false-color analogs for different cone opsins, to understand fish conspicuity.
‣ The idea was that these bright fish all have very different color cones and might actually be disguised in the eyes of predators.

Off-Policy RL that Prioritizes Value Function Improvement for Speedup
Ethan Brooks, Idris Hanafi, Jake DeGasperis, and Richard Higgins
Parallel Computer Architecture Project, Winter 2019

‣ By allocating more of fixed resources to value function improvement, we were able to train a reinforcement learning model to converge more quickly than a baseline.
‣ The idea is that policy and value networks might need very different batch sizes/GPU usage for stability in different environments and that this can be learnt.

github
Augmenting Physical Safety by Modifying Vehicular Networking Communication Methods
Eric Newberry, Hsun-Wei Cho, and Richard Higgins
Advanced Networking Project, Winter 2019

‣ We evaluated the frequency of vehicle safety messages to identify what adjustments could augment vehicle safety and reduce potential network congestion for self-driving cars.

github
Journal of Useless Information
Richard Higgins
Joke Project, Winter 2019

‣ I made a joke website for the statistically significant but insignificant results of the world.

site / github
GTA Perception
Richard Higgins, Parth Chopra, Chris Rockwell, Sahib Dhanjal, Ung Hee Lee.
Self-Driving Cars Project, Fall 2018

‣ We finetuned a Squeeze and Excitation ResNet to classify objects appearing in road-scene images. Finished in the top 10 for the class.

github
Applying CNN Architectures to Sentence Generation
Harmanpreet Kaur, Heeryung Choi, Aahlad Chandrabhatta, and Richard Higgins
Natural Language Processing Project, Fall 2017

‣ We trained a DenseNet language model. We explored how residual connectivity can compare favorably to RNNs.

github
Keras for X-Ray Radiology
Alexander Zaitzeff and Richard Higgins
Advanced Computer Vision Project, Fall 2017

‣ We built a (briefly) state-of-the-art multilabel x-ray disease classifer.

presentation / github
Nerve Sense
Richard Higgins
Summer Project, San Leandro, Summer 2016

‣ I built a stacked hourglass model with residual connections for nerve segmentation.
‣ I released a keras residual unit for use as a configurable building block on github.

project github / residual unit github
Dreambook
Shariq Hashme, Matt Favero, and Richard Higgins
Summer Project, Mountain View, Summer 2015

‣ We created a service for deep dreaming your Facebook profile picture with convolutional neural networks.

github
God Game
Shariq Hashme, Matt Favero, and Richard Higgins
Hackathon Project, Bitcamp, April 2015

‣ We created a Unity and Microsoft Kinect game in virtual reality using superpowers triggered by gesture and gaze.

devpost / github
Spot
Fuad Balashov and Richard Higgins
Android Project, March 2015

‣ We created a mobile app for sharing photos and videos by location with a map interface. Like Snapmap.

Cyberspace
Winter Project, January 2015

‣ I created a virtual reality browser for exploring the internet as if it were a 3D city with websites as buildings.
‣ The idea here is that similar and inter-linked websites should be located nearby one another.

github
Clickopedia
Hackathon Project, MHacks, Ann Arbor, January 2015

‣ I made an Android app that matchmakes people and lets two players race from one wikipedia page to another by only clicking links.

devpost / video
National Data Science Bowl
Winter Project, College Park, January 2015

‣ I trained a convolutional neural network in Caffe to do plankton image classification.

github
Autostock
Summer Project, Fremont, Summer 2014

‣ I made a neural network in PyBrain and later PyTorch for stock forecasting using convolutional neural networks and policy gradients.

github
Air Quality / UV / Pollen App
Ted Smith, David Ho, and Richard Higgins
Human Computer Interaction, Spring 2013

‣ We made an Android app for Air Quality, UV, and Pollen Count tracking.

video / video demo / github
Anjoyd
Dan Gillespie, Jeff Hilnbrand, Brent Bovenzi, and Richard Higgins
Hackathon Project, HackMIT, 2013
Finalist

‣ We made a robot which was steered by an Android phone and sent video to a Google Glass.

devpost / video / github
Paybaq
Alejandro Newell, Jonathan Lockwood, Wajahat Siddiqui, and Richard Higgins
Hackathon Project, MHacks, Ann Arbor, 2013

‣ We made an Android app for sending payment reminders to "friends".

devpost / video / github
Hotdrop
Dan Gillespie, Jeff Hilnbrand, Evan, and Richard Higgins
Hackathon Project, PennApps, 2013

‣ We made an Android location-based messaging app.

devpost / video
Arcade Glass
Dan Gillespie and Richard Higgins
Summer Project, Startup Shell, College Park, Summer 2013

‣ We made Android arcade games for Google Glass, controlled through head motions.

Bio-Inspired Robotics
Alejandro Newell, Matthew Smith, and Richard Higgins
Mechanical Engineering Course, Winter 2013

‣ We modeled a horse robot in CAD, tested gaits in a simulator, 3D printed it, and surpassed distance requirements in the course.

video
New York University
Visiting Academic
Courant Institute of Mathematical Sciences, New York University
New York, September 2023 - Present

‣ Doing research towards finishing my PhD!

cs.nyu.edu
Meta
Computer Vision Research Scientist Intern
FAIR, Meta
Menlo Park, May 2023 - November 2023

‣ Working on 3D hand understanding as a computer vision research intern at FAIR.

meta.com
University of Michigan
Graduate Student Instructor
Computer Science Department, University of Michigan
Ann Arbor, December 2018 - May 2019

‣ I led discussions, created assignments with Numpy and Pytorch in Python, graded projects, and hosted office hours for ~150 upper-level CS students.

web.eecs.umich.edu/~fouhey/teaching/EECS442_W19/
Voxel 51
Computer Vision Engineering Intern
Ann Arbor, November 2018 - March 2019

‣ I incorporated and trained various object detection neural networks as part of a video analysis platform to identify objects in dashcam footage.

voxel51.com
Gigster
Software Engineering Consultant
San Francisco, August 2016 - July 2018

‣ I built a style-transfer service on AWS that used to process millions of images/day.
‣ I built a GAN that performs face attribute transformation for a social media company.
‣ I built a CNN backend to provide object recognition in a Fortune 500 company iOS app.
‣ I designed many CNN computer vision systems for Fortune 500 clients across industries.

gigster.com
Unscan
Founder
New York, August 2015 - May 2016

‣ We built an automated document extraction service on AWS, with custom LSTMs for OCR.

minimill.co/unscan
Redspread
First Engineer
San Francisco, March 2015 - August 2015

‣ I developed machine learning tools to automatically scale Kubernetes pods based on networks requests, CPU, and memory usage.

ycombinator.com/companies/redspread
Mammalian Physiology
Teaching Assistant
Biology Department, University of Maryland
College Park, January 2014 - June 2014

‣ I instructed multi-hour discussions on cardiac function, renal system, nervous system, pharmacology, digestion, and more.

science.umd.edu/classroom/bsci440
Cooperative Housing University of Maryland
Housing Chair, Finance Manager
College Park, August 2011 - June 2013

‣ I was the primary contact with landlords, handled house finances, and organized housing for the next school year.

chum.coop
Woods Hole Yacht Club
Sailing Instructor
Woods Hole, June - August, 2007 - 2009

‣ I taught children how to sail and not crash into expensive boats.

woodsholeyachtclub.org