Working on using beam search to guide the retrieval-augmented system for efficient searching and better performance.
Design, develop, and deploy a multi-hop retrieval-augmented transformer.
Inspected, profiled, and sped up the inverse cloze task and pre-training by 10 times and 8 times respectively.
Researched on substituting memory in model’s parameters with plugged-in external memory, expanding the power and expressiveness of the model s.t. the model could be more parameter-efficient.
Worked on designing, implementing, and tuning an reinforcement-learning/decision transformer-based agent that performs heavily-constrained chip placement efficiently without supervision.