Yang Yebin, a graduate of the Zhiyuan Engineering Honors Program (class of 2025), has proposed a novel scaling method for large language models that achieves approximately 35% training cost savings.

As first author, Yang introduced JTok (Joint-Token), an efficient capacity-scaling module developed in collaboration with the SJTU-Xiaohongshu Joint Lab, under the supervision of Professor Yan Junchi from SJTU's AI School. The work presents a third scaling path beyond traditional Dense and Mixture-of-Experts (MoE) models.

The Core Innovation

JTok assigns each token a set of learnable identity vectors. During computation, tokens retrieve corresponding vectors through a lookup table and apply modulation to layer outputs—injecting substantial token-specific knowledge with minimal overhead and no matrix multiplication. The enhanced version, JTok-M, enables context-aware modulation by dynamically selecting vectors based on contextual environment.

Controlled experiments demonstrate that JTok-M achieves equivalent performance with approximately 35% less training compute, with its own predictable scaling potential. 

A Tribute to SJTU's 130th Anniversary

The name JTok carries special significance—"Joint" resonates with "Jiao Tong" in Chinese, reflecting a shared identity among Jiaotong University members as the institution approaches its 130th anniversary.

Yang credits his Zhiyuan education, particularly Professor Yu Yong's rigorous data structures course and academic mentor Professor Wang Lei's continuous guidance, for shaping his research trajectory.