Artificial Intelligence & AI Convergence Network 사업단에서 공동으로
개최하는 Colloquium을 4월 29일(월) 오후 3시에 개최하오니 많은 참여 부탁드립니다.
When : 2024년 4월 29일(월) 오후 3시
Where : 팔달관 407호 Speaker : 강병곤 교수(SUNY Korea) Title : On the efficiency of pre-trained word embeddings on Transformers
Abstract : In this talk, I will discuss a counter-intuitive phenomenon where state-of-the-art pretrained word embedding vectors perform poorly compared to randomly initialized vectors. More specifically, such a tendency is observed almost exclusively in Transformer architectures, making it a problem to most modern NLP systems that are heavily dependent on Transformers.
I also describe a very simple remedy that somewhat alleviates this shortcoming, as well as numerous empirical results to back this claim.
If time permits, I will also share a failure case of another popular deep learning system (autograd), leading us to the discussion of whether we should take prevalent technologies for granted in this field.
Bio : Byungkon Kang got his PhD at KAIST in 2013 before joining Samsung Advanced Institute of Technology. After that, he spent a few years at Ajou University as a research professor.He is currently an assistant professor at SUNY Korea, working on various AI/ML topics.
