Kun He

何坤 • Researcher at Meta Reality Labs • FirstnameLastname @ meta.com

kun16.jpg

Meta Reality Labs

Redmond, WA, USA

I am a research scientist at Meta Reality Labs, working on the future of interactions for Virtual Reality and Augmented Reality, using computer vision and machine learning techniques. Here’s my CV.

My current research interests include: human body/hand pose estimation and tracking, domain adaptation, and active learning.

I graduated from Boston University in 2018 with PhD and MSc degrees in computer science. My advisor was Professor Stan Sclaroff. My PhD dissertation focused on learning vector embeddings for visual data, which finds application in image retrieval and other related computer vision tasks. During grad school, I did research internships at Disney Research and Honda Research Institute, and I was a visiting student at Brown University. I hold a Bachelor’s degree in computer science from Zhejiang University, China.

news

Jun 2023 AssemblyHands dataset will be featured in the 7th HANDS workshop at ICCV 2023: Task 1 - Egocentric 3D Hand Pose Estimation.
Feb 2023 Paper accepted to CVPR 2023 (with Takehiko Ohkawa et al.): AssemblyHands: Towards Egocentric Activity Understanding via 3D Hand Pose Estimation.
Oct 2022 Paper accepted to WACV 2023 (with Qi Feng et al.): Rethinking the Data Annotation Process for Multi-view 3D Pose Estimation with Active Learning and Self-Training.
Mar 2022 Paper accepted at CVPR 2022 (with Fadime Sener et al.): Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activities
Sep 2019 Check out hand tracking on Oculus Quest!

publications

conference papers

  1. AssemblyHands: Towards Egocentric Activity Understanding via 3D Hand Pose Estimation
    Takehiko Ohkawa,  Kun He, Fadime Sener, Tomáš Hodaň, Luan Tran, and Cem Keskin
    In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2023
    Website
  2. Rethinking the Data Annotation Process for Multi-view 3D Pose Estimation with Active Learning and Self-Training
    Qi Feng,  Kun He, He Wen, Cem Keskin, and Yuting Ye
    In Proc. IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) 2023
    PDF Code
  3. Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activities
    Fadime Sener, Dibyadip Chatterjee, Daniel Shelepov,  Kun He, Dipika Singhania, Robert Wang, and Angela Yao
    In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2022
    PDF Website
  4. Deep Metric Learning to Rank
    Fatih Çakir*, Kun He*, Xide Xia, Brian Kulis, and Stan Sclaroff  (* equal contribution)
    In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2019
    PDF Code Poster
  5. Generalized Majorization-Minimization
    Sobhan Naderi Parizi,  Kun He, Reza Aghajani, Stan Sclaroff, and Pedro Felzenszwalb
    In Proc. International Conference on Machine Learning (ICML) 2019
    PDF
  6. Multilevel Language and Vision Integration for Text-to-Clip Retrieval
    Huijuan Xu,  Kun He, Bryan A. Plummer, Leonid Sigal, Stan Sclaroff, and Kate Saenko
    In Proc. The Thirty-Third AAAI Conference on Artificial Intelligence 2019
    PDF Code
  7. Hashing with Binary Matrix Pursuit
    Fatih Çakir,  Kun He, and Stan Sclaroff
    In Proc. European Conference on Computer Vision (ECCV) 2018
    PDF Code
  8. Local Descriptors Optimized for Average Precision
    Kun He, Yan Lu, and Stan Sclaroff
    In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2018
    PDF Poster Website
  9. Hashing as Tie-Aware Learning to Rank
    Kun He, Fatih Çakir, Sarah Adel Bargal, and Stan Sclaroff
    In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2018
    PDF Code Poster
  10. MIHash: Online Hashing With Mutual Information
    Fatih Çakir*, Kun He*, Sarah Adel Bargal, and Stan Sclaroff  (* equal contribution)
    In Proc. IEEE International Conference on Computer Vision (ICCV) 2017
    PDF Code Poster
  11. Parameterizing Object Detectors in the Continuous Pose Space
    Kun He, Leonid Sigal, and Stan Sclaroff
    In Proc. European Conference on Computer Vision (ECCV) 2014
    PDF Supp Poster
  12. Scale Resilient, Rotation Invariant Articulated Object Matching
    Hao Jiang, Tai-Peng Tian,  Kun He, and Stan Sclaroff
    In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2012
    PDF

journal papers

  1. Hashing with Mutual Information
    Fatih Çakir*, Kun He*, Sarah Adel Bargal, and Stan Sclaroff  (* equal contribution)
    IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 2019
    PDF Code
  2. Predicting Foreground Object Ambiguity And Efficiently Crowdsourcing the Segmentation(s)
    Danna Gurari,  Kun He, Bo Xiong, Jianming Zhang, Mehrnoosh Sameki, Suyog Dutt Jain, Stan Sclaroff, Margrit Betke, and Kristen Grauman
    International Journal of Computer Vision (IJCV) 2018
    PDF

theses

  1. Learning Deep Embeddings by Learning to Rank
    Kun He
    Ph.D. Dissertation, Boston University, 2018
    PDF
  2. Stochastic Functional Descent for Learning Support Vector Machines
    Kun He
    M.Sc. Thesis, Boston University, 2013
    PDF