News

    During my PhD, I worked on representation learning in images. Most recently, I've worked on verb compositionality in image editing. Prior to that, I trained neural nets to segment scenes from pseudolabels (ascribing motion to either camera or hands). I also used satellite imagery of the sun to improve estimates of the solar magnetic field.


    SELDOM: Scene Editing via Latent Diffusion with Object-centric Modifications
    Richard E.L. Higgins, Ekdeep S. Lubana, David F. Fouhey
    In preparation for submission to the International Conference on Computer Vision, February 2025 bibtex

    ‣ A method using a combination of visual and textual prompts to condition latent diffusion for image editing.

    SuperSynthIA: Physics-Ready Full-Disk Vector Magnetograms from HMI, Hinode, and Machine Learning
    Ruoyu Wang, David Fouhey, Richard E.L. Higgins, Spiro K. Antiochos, Graham Barnes, J. Todd Hoeksema, K.D. Leka, Yang Liu, Peter W. Schuck, Tamas I. Gombosi
    Astrophysical Journal, July 2024 bibtex

    ‣ A new method for combining multiple instruments to produce magnetograms useful for downstream systems.

    paper / site / github
    Towards A Richer 2D Understanding of Hands at Scale
    Tianyi Cheng*, Dandan Shan*, Ayda Sultan, Richard E.L. Higgins, David Fouhey
    NeurIPS 2023, bibtex

    ‣ A new dataset, tasks, and model for understanding more complex hand interactions, including bimanual manipulation and tool use.

    MOVES: Manipulated Objects in Video Enable Segmentation
    Richard E.L. Higgins, David Fouhey
    CVPR 2023, bibtex

    ‣ We use disagreement from a background motion model as a pseudolabel to train hand and held-object grouping and association.

    paper / site / github
    EPIC-KITCHENS VISOR Benchmark: VIdeo Segmentations and Object Relations
    Ahmad Darkhalil, Dandan Shan, Bin Zhu, Jian Ma, Amlan Kar, Richard E.L. Higgins, Sanja Fidler, David Fouhey, Dima Damen
    NeurIPS 2022, bibtex

    ‣ We made EPIC-KITCHENS VISOR, a new dataset of annotations for segmenting hands and active kitchen objects in egocentric video.

    paper / site
    On Identifying and Mitigating Bias in Inferred Measurements for Solar Vector Magnetic-Field Data
    K.D. Leka, Eric L. Wagner, Ana Belén Griñón-Marín, Véronique Bommier, Richard E.L. Higgins
    Solar Physics, July 2022 bibtex

    ‣ I trained a neural network to produce synthetic magnetic field inversions and we used the outputs to fix (ongoing) biases in the satellite processing pipeline.

    paper
    COHESIV: Contrastive Object and Hand Embeddings for Segmentation In Video
    Richard E.L. Higgins*, Dandan Shan*, and David F. Fouhey
    NeurIPS 2021, bibtex

    ‣ We built a system that predicts segmentation masks for objects held by hands. The system trains from person, object, and background pseudolabels made by subtracting detected people from optical flow.

    paper
    SynthIA: A Synthetic Inversion Approximation for the Stokes Vector Fusing SDO and Hinode into a Virtual Observatory
    Richard E.L. Higgins, David F. Fouhey, Spiro K. Antiochos, Graham Barnes, Mark C.M. Cheung, J. Todd Hoeksema, KD Leka, Yang Liu, Peter W. Schuck, Tamas I. Gombosi
    Astrophysical Journal Supplement Series, March 2022 bibtex
    SDO Science Seminar, Invited Talk 2021

    ‣ Hinode's SOT-SP measures small areas of the sun in high spatial and spectral resolution to predict the magnetic field. SDO/HMI measures the full-disk in lower resolution, both spatial and spectral.
    ‣ By training a neural network to accurately predict Hinode's estimated field using only HMI's input, we created a virtual observatory that melds the best parts of both instruments.

    paper
    Fast and Accurate Emulation of the SDO/HMI Stokes Inversion with Uncertainty Quantification
    Richard E.L. Higgins, David F. Fouhey, Dichang Zhang, Spiro K. Antiochos, Graham Barnes, Todd Hoeksema, KD Leka, Yang Liu, Peter W. Schuck, Tamas I. Gombosi
    Astrophysical Journal, April 2021 bibtex
    AGU, ML in Space Weather, Poster 2020
    COSPAR2021, Workshop on ML for Space Sciences, Talk 2021

    ‣ I trained a UNet to predict magnetic field parameters on the sun using polarized light (IQUV's) recorded from the Solar Dynamics Observatory's HMI sensor.

    paper / site / github / talk / poster
    Network Reconstruction Reveals that Valproic Acid Activates Neurogenic Transcriptional Programs in Adult Brain Following Traumatic Injury
    Gerald A. Higgins, Patrick Georgoff, Vahagn Nikolian, Ari Allyn-Feuer, Brian Pauls, Richard Higgins, Brian D. Athey, and Hasan E. Alam
    Pharmaceutical Research, August 2017 bibtex

    ‣ I constructed topologically associated domains and analyzed RNA-seq data to identify differential gene expression using bioinformatics libraries in R and Python.

    Matrix Metalloproteinase-9 Regulates Neuronal Circuit Development and Excitability
    Sachiko Murase, Crystal Lantz, Eunyoung Kim, Nitin Gupta, Richard Higgins, Mark Stopfer, Dax A. Hoffman, and Elizabeth M. Quinlan
    Journal of Molecular Neurobiology, July 2016 bibtex

    ‣ I wrote MATLAB functions that classified windows of mouse EEG recordings as seizure/not seizure with max-margin unsupervised learning.

    Visualizing What Cichlids with Different Cone Opsins See
    Richard Higgins, Karen Carleton
    Undergrad Resarch Project, 2012

    ‣ I wrote a Java application that changed underwater images into false-color analogs for different cone opsins, to understand fish conspicuity.
    ‣ The idea was that these bright fish all have very different color cones and might actually be disguised in the eyes of predators.

    Off-Policy RL that Prioritizes Value Function Improvement for Speedup
    Ethan Brooks, Idris Hanafi, Jake DeGasperis, and Richard Higgins
    Parallel Computer Architecture Project, Winter 2019

    ‣ By allocating more of fixed resources to value function improvement, we were able to train a reinforcement learning model to converge more quickly than a baseline.
    ‣ The idea is that policy and value networks might need very different batch sizes/GPU usage for stability in different environments and that this can be learnt.

    github
    Augmenting Physical Safety by Modifying Vehicular Networking Communication Methods
    Eric Newberry, Hsun-Wei Cho, and Richard Higgins
    Advanced Networking Project, Winter 2019

    ‣ We evaluated the frequency of vehicle safety messages to identify what adjustments could augment vehicle safety and reduce potential network congestion for self-driving cars.

    github
    Journal of Useless Information
    Richard Higgins
    Joke Project, Winter 2019

    ‣ I made a joke website for the statistically significant but insignificant results of the world.

    site / github
    GTA Perception
    Richard Higgins, Parth Chopra, Chris Rockwell, Sahib Dhanjal, Ung Hee Lee.
    Self-Driving Cars Project, Fall 2018

    ‣ We finetuned a Squeeze and Excitation ResNet to classify objects appearing in road-scene images. Finished in the top 10 for the class.

    github
    Applying CNN Architectures to Sentence Generation
    Harmanpreet Kaur, Heeryung Choi, Aahlad Chandrabhatta, and Richard Higgins
    Natural Language Processing Project, Fall 2017

    ‣ We trained a DenseNet language model. We explored how residual connectivity can compare favorably to RNNs.

    github
    Keras for X-Ray Radiology
    Alexander Zaitzeff and Richard Higgins
    Advanced Computer Vision Project, Fall 2017

    ‣ We built a (briefly) state-of-the-art multilabel x-ray disease classifer.

    presentation / github
    Nerve Sense
    Richard Higgins
    Summer Project, San Leandro, Summer 2016

    ‣ I built a stacked hourglass model with residual connections for nerve segmentation.
    ‣ I released a keras residual unit for use as a configurable building block on github.

    project github / residual unit github
    Dreambook
    Shariq Hashme, Matt Favero, and Richard Higgins
    Summer Project, Mountain View, Summer 2015

    ‣ We created a service for deep dreaming your Facebook profile picture with convolutional neural networks.

    github
    God Game
    Shariq Hashme, Matt Favero, and Richard Higgins
    Hackathon Project, Bitcamp, April 2015

    ‣ We created a Unity and Microsoft Kinect game in virtual reality using superpowers triggered by gesture and gaze.

    devpost / github
    Spot
    Fuad Balashov and Richard Higgins
    Android Project, March 2015

    ‣ We created a mobile app for sharing photos and videos by location with a map interface. Like Snapmap.

    Cyberspace
    Winter Project, January 2015

    ‣ I created a virtual reality browser for exploring the internet as if it were a 3D city with websites as buildings.
    ‣ The idea here is that similar and inter-linked websites should be located nearby one another.

    github
    Clickopedia
    Hackathon Project, MHacks, Ann Arbor, January 2015

    ‣ I made an Android app that matchmakes people and lets two players race from one wikipedia page to another by only clicking links.

    devpost / video
    National Data Science Bowl
    Winter Project, College Park, January 2015

    ‣ I trained a convolutional neural network in Caffe to do plankton image classification.

    github
    Autostock
    Summer Project, Fremont, Summer 2014

    ‣ I made a neural network in PyBrain and later PyTorch for stock forecasting using convolutional neural networks and policy gradients.

    github
    Air Quality / UV / Pollen App
    Ted Smith, David Ho, and Richard Higgins
    Human Computer Interaction, Spring 2013

    ‣ We made an Android app for Air Quality, UV, and Pollen Count tracking.

    video / video demo / github
    Anjoyd
    Dan Gillespie, Jeff Hilnbrand, Brent Bovenzi, and Richard Higgins
    Hackathon Project, HackMIT, 2013
    Finalist

    ‣ We made a robot which was steered by an Android phone and sent video to a Google Glass.

    devpost / video / github
    Paybaq
    Alejandro Newell, Jonathan Lockwood, Wajahat Siddiqui, and Richard Higgins
    Hackathon Project, MHacks, Ann Arbor, 2013

    ‣ We made an Android app for sending payment reminders to "friends".

    devpost / video / github
    Hotdrop
    Dan Gillespie, Jeff Hilnbrand, Evan, and Richard Higgins
    Hackathon Project, PennApps, 2013

    ‣ We made an Android location-based messaging app.

    devpost / video
    Arcade Glass
    Dan Gillespie and Richard Higgins
    Summer Project, Startup Shell, College Park, Summer 2013

    ‣ We made Android arcade games for Google Glass, controlled through head motions.

    Bio-Inspired Robotics
    Alejandro Newell, Matthew Smith, and Richard Higgins
    Mechanical Engineering Course, Winter 2013

    ‣ We modeled a horse robot in CAD, tested gaits in a simulator, 3D printed it, and surpassed distance requirements in the course.

    video
    New York University
    Visiting Academic
    Courant Institute of Mathematical Sciences, New York University
    New York, September 2023 - March 2025

    ‣ I did work on hand, 3D, sun, and image editing projects, while mentoring students and finishing my PhD.

    cs.nyu.edu
    Meta
    Computer Vision Research Scientist Intern
    FAIR, Meta
    Menlo Park, May 2023 - November 2023

    ‣ I developed a system for estimating 4D hand pose as a computer vision research intern on the Ego How-To team.

    meta.com
    University of Michigan
    Graduate Student Instructor
    Computer Science Department, University of Michigan
    Ann Arbor, December 2018 - May 2019

    ‣ I led discussions, created assignments with Numpy and Pytorch in Python, graded projects, and hosted office hours for ~150 upper-level CS students.

    web.eecs.umich.edu/~fouhey/teaching/EECS442_W19/
    Voxel 51
    Computer Vision Engineering Intern
    Ann Arbor, November 2018 - March 2019

    ‣ I incorporated and trained various object detection neural networks as part of a video analysis platform to identify objects in dashcam footage.

    voxel51.com
    Gigster
    Software Engineering Consultant
    San Francisco, August 2016 - July 2018

    ‣ I built a style-transfer service on AWS that used to process millions of images/day.
    ‣ I built a GAN that performs face attribute transformation for a social media company.
    ‣ I built a CNN backend to provide object recognition in a Fortune 500 company iOS app.
    ‣ I designed many CNN computer vision systems for Fortune 500 clients across industries.

    gigster.com
    Unscan
    Founder
    New York, August 2015 - May 2016

    ‣ We built an automated document extraction service on AWS, with custom LSTMs for OCR.

    minimill.co/unscan
    Redspread
    First Engineer
    San Francisco, March 2015 - August 2015

    ‣ I developed machine learning tools to automatically scale Kubernetes pods based on networks requests, CPU, and memory usage.

    ycombinator.com/companies/redspread
    Mammalian Physiology
    Teaching Assistant
    Biology Department, University of Maryland
    College Park, January 2014 - June 2014

    ‣ I instructed multi-hour discussions on cardiac function, renal system, nervous system, pharmacology, digestion, and more.

    science.umd.edu/classroom/bsci440
    Cooperative Housing University of Maryland
    Housing Chair, Finance Manager
    College Park, August 2011 - June 2013

    ‣ I was the primary contact with landlords, handled house finances, and organized housing for the next school year.

    chum.coop
    Woods Hole Yacht Club
    Sailing Instructor
    Woods Hole, June - August, 2007 - 2009

    ‣ I taught children how to sail and not crash into expensive boats.

    woodsholeyachtclub.org