Here is Jinhui (叶劲辉), an incoming PhD student at HKUST and supervised by Prof. Jiaya Jia. My research interests lie at the intersection of action perception, manipulation, and language-based human-robot interaction.
Previously, I obtained my degree from South China University of Technology and pursued an MPhil at The Hong Kong University of Science and Technology (Guangzhou) under the guidance of Prof. Hui Xiong and Junwei Liang.
I have also had the opportunity to intern at several prestigious institutions:
- At Tencent AI Lab, I worked on sign language translation in collaboration with Xing Wang and Wenxiang Jiao.
- At Stanford Vision and Learning Lab, I collaborated with Prof Manling Li, Prof. Jiajun Wu and Prof. Li Fei-Fei.
- At CMU LIT, I worked with Prof. Yonatan Bisk on Embodied-RAG.
Recently, I have been interning at Shanghai AI Lab with Dr. Yilun Chen, where our efforts are focused on aligning cognition and action within a Vision-Language-Action (VLA) framework.
For my PhD, I plan to further explore this topic by developing a unified VLA model—a concept encapsulated in the Chinese phrase “知行合一” (the unity of knowledge and action, where a robot “does what it knows and knows what it is doing”). I am deeply passionate about this area and believe it holds great promise for both advancing research and enriching my personal philosophy. I welcome discussions and collaboration on these topics, so please feel free to reach out via email if you’re interested.
News
• 0928: Excited to announce that our two projects clinched 1st and 2nd places in the Human-Machine Interaction track of the Pazhou Algorithm Contest! We’ve been awarded a total of 120,000 RMB and look forward to a big celebratory dinner. The relative works will be documented in upcoming papers. link
• Before 08/30/2023: […]
Publications
LongvideoHaystack: Re-thinking Temporal Search for Long-Form Video Understandin
Jinhui Ye, Zihan Wang, Haosen Sun, Keshigeyan Chandrasegaran, Zane Durante, Cristobal Eyzaguirre, Yonatan Bisk, Juan Carlos Niebles, Ehsan Adeli, Li Fei-Fei, Jiajun Wu, Manling Li
CVPR 2025 . [Project Page] [Dataset] Paper & Code coming soon
SePer: Measure Retrieval Utility Through The Lens Of Semantic Perplexity Reduction
Lu Dai, Yijie Xu, Jinhui Ye, Hao Liu, Hui Xiong
ICLR 2025(spotlight) . [Openreview]
Improving Gloss-free Sign Language Translation by Reducing Representation Density
Jinhui Ye, Xing Wang, Wenxiang Jiao, Junwei Liang, Hui Xiong
NeurIPS 2024. [arXiv] [code]
Cross-modality Data Augmentation for End-to-End Sign Language Translation
Jinhui Ye, Wenxiang Jiao, Xing Wang, Zhaopeng Tu, Hui Xiong
EMNLP 2023. [arXiv] [code]
Scaling Back-Translation with Domain Text Generation for Sign Language Gloss Translation
Jinhui Ye, Wenxiang Jiao, Xing Wang, Zhaopeng Tu
EACL 2023. [paper] [code]
Aspect-Opinion Correlation Aware and Knowledge-Expansion Few Shot Cross-Domain Sentiment Classification
Haopeng Ren, Yi Cai, Yushi Zeng, Jinhui Ye, Ho-fung Leung, Qing Li
Transactions on Affective Computing 2022. [paper] [code]
On Processing Papers
GeoDeformer: Geometric Deformable Transformer for Action Recognition
Jinhui Ye, Jiaming Zhou, Hui Xiong, Junwei Liang
Preprint 2023. [arXiv]
Spatial-Temporal Alignment Network for Action Recognition
Jinhui Ye, Junwei Liang
Preprint 2023. [arXiv]
Internship
• CMU LTI | Visiting Student | Sep. 2024 ‑ Present
• Stanford AI Lab | Intern | Dec. 2023 ‑ Oct. 2024
• Tencent AI Lab | Summer Intern | Aug. 2021 ‑ Aug. 2022
Awards
• Outstanding Undergraduate Thesis in SCUT
• National Scholarship in China