NAVER : CLOVA Proves AI Research Capabilities with Record 12 Papers Accepted by ICLR

NAVER CLOVA Proves AI Research Capabilities with Record 12 Papers Accepted by ICLR

- AI studies by NAVER's AI division were accepted by ICLR, the world's top artificial intelligence and machine learning conference which is recognized by Google Scholar - NAVER is the first Korean company to present double-digit papers at top global machine learning conferences - The company's close collaborations in AI with top academic institutions also show promising results.

2022-01-26

NAVER(KRX: 035420) announced that its AI division NAVER CLOVA has demonstrated its global research capabilities as 12 of its research papers have been accepted by the top machine learning conference, the International Conference on Learning Representations (ICLR) 2022. A total of 17 were accepted by the ICLR including submissions by other NAVER affiliates such as NAVER Labs Europe (NLE) and LINE.

Now in its tenth edition, ICLR is globally renowned for presenting and publishing cutting-edge research on all aspects of deep learning used in the fields of artificial intelligence, statistics and data science. ICLR is listed as Google Scholar's top publication in artificial intelligence(AI) and machine learning(ML) publication since 2021, with the highest h5-index amongst 20 leading journals.

*h5-index: Google Scholar's metric for evaluating the cumulative impact of publications for both productivity and citation impact in the last five complete years.

This year was the first time that NAVER has had double-digit papers selected by a global machine learning conference. NAVER is the first among Korean companies to feature at major machine learning conferences and is the country's second research entity to have double-digit papers published at ICLR after Korea Advanced Institute of Science and Technology (KAIST).

The company earlier had double-digit papers published last year at ICASSP, Interspeech, ICCV, EMNLP and other voice recognition/synthesis, Computer Vision, Natural Language Processing conferences, demonstrating its global AI technology R&D capabilities. Along with this year's recognition at ICLR, NAVER is reaffirming its global leadership and capabilities in core AI technology and applied AI science.

The selected papers include, 'Learning Features with Parameter-Free Layers' which introduces a new design paradigm in rethinking network architecture to build Convolutional Neural Networks (CNN), making it possible to improve data processing ability using parameter-free operations; 'Which Shortcut Cues Will DNNs Choose? A Study from the Parameter-Space Perspective' which documents an experimental analysis of how deep neural networks (DNN) often rely on easy-to-learn discriminatory features, or cues, that are not necessarily essential to the problem at hand; and 'Contrastive Fine-grained Class Clustering via Generative Adversarial Networks' which focuses on learning feature representations that encourage data to form distinct cluster boundaries in the embedding space.

NAVER's close collaboration on AI research with academic institutions in Korea is also garnering significant attention. The ICLR 2022 also selected papers from NAVER's research partnerships with Korea's most prestigious universities including Seoul National University (SNU) and KAIST. Together with Professor Chun Byung-Gon and his team from the SNU-NAVER Hyperscale AI Center, NAVER proposed 'SUMNAS: Supernet with Unbiased Meta-Features for Neural Architecture Search', which effectively automates artificial neural network building. In addition, 'Generating Videos with Dynamics-aware Implicit Generative Adversarial Networks' by Professor Shin Jin-woo of KAIST-NAVER Hypercreative AI Center was also accepted by the conference. The paper focuses on a dynamics-aware implicit generative adversarial network (DIGAN), a novel generative adversarial network for long video generation.

"NAVER's accomplishments at ICLR 2022 will be firmly etched in Korean AI history. Not only has NAVER CLOVA AI's research capabilities been recognized on a global level, but NAVER's research collaborations with SNU and KAIST have also delivered meaningful results," said Jeong Seok-keun, Head of NAVER CLOVA. "Although there are not many research achievements in the field of machine learning compared to applied AI in Korea, NAVER CLOVA is significantly contributing to increasing national competitiveness in the field of AI."

NAVER also announced that seven more of its research papers on CLOVA's voice recognition and voice synthesis technology-based services will be presented at ICASSP 2022, the world's largest and most comprehensive technical conference focused on signal processing and its applications. ICASSP has accepted over 10 papers, from NAVER affiliates including LINE, marking another milestone in NAVER's AI R&D journey.

-END-

[Reference 1] NAVER CLOVA papers accepted by ICLR 2022

1. Learning Features with Parameter-Free Layers

https://openreview.net/forum?id=bCrdi4iVvv

Han Dong-yoon(NAVER AI Lab), Yoo Young-joon(NAVER Clova), Kim Beom-young (NAVER Clova), Heo Byeong-ho(NAVER AI Lab)

Proposes that simple built-in parameter-free operations can be a favorable alternative to the efficient trainable layers replacing spatial operations in a network architecture.

2. SUMNAS: Supernet with Unbiased Meta-Features for Neural Architecture Search (SNU-NAVER Hyperscale AI Center)

https://openreview.net/forum?id=Z8FzvVU6_Kj

Ha Hyeon-min(SNU), Kim Ji-Hoon(NAVER CLOVA), Park Se-min(Google), Chun Byung-Gon(SNU)

Proposes a supernet learning strategy for Neural Architecture Search based on meta-learning to tackle the knowledge forgetting issue.

3. ViDT: An Efficient and Effective Fully Transformer-based Object Detector

https://openreview.net/forum?id=w4cXZDDib1H

Song Hwa-jun(NAVER AI Lab), Deqing Sun(Google Research), Chun Sang-hyuk (NAVER AI Lab), Varun Jampani(Google Research), Han Dong-yoon(NAVER AI Lab), Heo Byeong-ho(NAVER AI Lab), Kim Won-jae(NAVER AI Lab), Ming-Hsuan Yang(Google Research, UC Merced)

Proposes fully transformer-based object detectors which are more effective than existing DETR.

4. Coherence-based Label Propagation over Time Series for Accelerated Active Learning

https://openreview.net/forum?id=gjNcH0hj0LM

Shin Yoo-ju(KAIST), Yoon Su-sik(UIUC), Kim Sun-dong(IBS), Song Hwan-jun(NAVER AI Lab), Lee Jae-Gil(KAIST), Lee Byung-Suk(Univ. of Vermont)

Proposes a method to automatically assign a queried label to the data points within an accurately estimated time-series segment.

5. Which Shortcut Cues Will DNNs Choose? A Study from the Parameter-Space Perspective

https://openreview.net/forum?id=qRDQi3ocgR3

Luca Scimeca(NAVER AI Lab), Oh Seong-Joon(NAVER AI Lab), Chun Sang-hyuk(NAVER AI Lab), Michael Poli(NAVER AI Lab), Yun Sang-doo(NAVER AI Lab)

Experimental study on why deep neural networks (DNNs) often take learning shortcuts.

6. Know Your Action Set: Learning Action Relations for Reinforcement Learning

https://openreview.net/forum?id=MljXVdp4A3N

Ayush Jain*(USC), Norio Kosaka*(NAVER CLOVA), Kim Kyung-Min(NAVER CLOVA), Joseph J Lim(KAIST, NAVER AI Lab)

Posits that learning the interdependence between actions is crucial for RL agents acting under a varying action set.

7. Skill-based Meta-Reinforcement Learning

https://openreview.net/forum?id=jeLW-Fh9bV

Nam Tae-wook(KAIST), Shao-Hua Sun(USC), Karl Pertsch(USC), Hwang Sung-Ju(KAIST), Joseph J Kim(KAIST, NAVER AI Lab)

Proposes a method that enables meta-learning on long-horizon, sparse-reward tasks, allowing unseen target tasks to be solved with orders of magnitude fewer environment interactions.

8. Task-Induced Representation Learning

Jun Yamada (Oxford), Karl Pertsch(USC), Anisha Gunjal(USC), Joseph J Lim(KAIST, NAVER AI Lab)

https://openreview.net/forum?id=OzyXtIZAzFv

Formalizes the problem of task-induced representation learning (TARP), which aims to leverage such task information in offline experience from prior tasks for learning compact representations that focus on modeling only task-relevant aspects.

9. Encoding Weights of Irregular Sparsity for Fixed-to-Fixed Model Compression

https://openreview.net/forum?id=Vs5NK44aP9P

Park Bae-Seong*(NAVER CLOVA), Kwon Se-Jung(NAVER CLOVA), Oh Dae-hwan(Samsung Research), Kim Hyeong-wook(NAVER CLOVA), Lee Dong-soo(NAVER CLOVA)

Studies fixed-to-fixed (lossless) encoding architecture/algorithm to support fine-grained pruning methods such that sparse neural networks can be stored in a highly regular structure.

10. Online Continual Learning on Class Incremental Blurry Task Configuration with Anytime Inference

https://openreview.net/forum?id=nrGGfMbY_qK

Koh Hyun-seo(GIST, NAVER AI Lab), Kim Da-huyn(GIST, NAVER AI Lab), Ha Jung-Woo(NAVER AI Lab), Choi Jong-hyun Choi(GIST, NAVER AI Lab)

Proposes a novel continual learning setup that is online, task-free, class-incremental, of blurry task boundaries and subject to inference queries at any moment. Additionally proposes a new metric to better measure the performance of the continual learning methods subject to inference queries at any moment.

11. Generating Videos with Dynamics-aware Implicit Generative Adversarial Networks (KAIST-NAVER Hypercreative AI Center)

https://openreview.net/forum?id=Czsdv-S4-w9

Yu Si-hyun(KAIST), Tack Ji-hoon(KAIST), Mo Sang-woo(KAIST), Kim Hyun-su Kim(NAVER AI Lab), Kim Jun-ho(NAVER AI Lab), Ha Jung-Woo(NAVER AI Lab), Shin Jin-woo(KAIST)

Proposes dynamics-aware implicit generative adversarial network (DIGAN), a novel generative adversarial network for long video generation.

12. Contrastive Fine-grained Class Clustering via Generative Adversarial Networks

https://openreview.net/forum?id=XWODe7ZLn8f

Kim Yun-ji(NAVER AI Lab), Ha Jung-Woo(NAVER AI Lab)

Introduces C3-GAN, a method for unsupervised fine-grained class clustering that leverages the categorical inference power of InfoGAN with contrastive learning.

[Reference 2] NAVER CLOVA papers accepted by ICASSP

1. Phase Continuity: Learning Derivatives of Phase Spectrum for Speech Enhancement.

Kim Do-yeon(Yonsei Univ.), Han Hye-won(Yonsei Univ.), Shin Hyeon-Kyeong(Yonsei Univ., NAVER CLOVA), Chung Soo-Whan (NAVER CLOVA), Kang Hong-Goo Kang(Yonsei Univ.)

To emphasize the continuity of speech spectrum in time and frequency, phase spectrum was studied in kernel unit and based on this data, total speech enhancement was achieved.

2. Multi-Scale Speaker Embedding-based Graph Attention Networks for Speaker Diarisation

https://arxiv.org/abs/2110.03361

Kwon Young-ki (NAVE CLOVA), Heo Hee-Soo(NAVER CLOVA), Jung Jee-weon(NAVER CLOVA), Kim You-jin Kim(NAVER CLOVA), Lee Bong-Jin(NAVER CLOVA), Chung Joon-son(KAIST)

Improving multi-scale speaker diarisation through graph attention networks based on affinity matrixes from multi-scale embeddings.

3. AASIST: Audio Anti-Spoofing using Integrated Spectro-Temporal Graph Attention Networks

https://arxiv.org/abs/2110.01200

Jung Jee-weon(NAVER CLOVA), Heo Hee-soo(NAVER CLOVA), Hemlata Tak(EURECOM), Shim Hye-jin(University of Seoul), Chung Joon-son(KAIST), Lee Bong-jin(NAVER CLOVA),Yu Ha-jin(University of Seoul), Nicholas Evans(EURECOM)

Proposes a novel heterogeneous stacking graph attention layer that models artefacts spanning heterogeneous temporal and spectral domains with a heterogeneous attention mechanism and a stack node.

4. Attentive Max Feature Map and Joint Training for Acoustic Scene Classification

https://arxiv.org/abs/2104.07213

Shim Hye-Jinm(University of Seoul), Jung Jee-weon(NAVER CLOVA), Kim Ju-ho Kim(University of Seoul), Yu Ha-jin(University of Seoul)

Proposes an attentive max feature map that combines two effective techniques, attention and a max feature map, to further elaborate the attention mechanism and mitigate the excessive discarding of potentially valuable information.

5. Spell My Name: Keyword Boosted Speech Recognition

https://arxiv.org/abs/2110.02791

Nam Nam-kyu (NAVER CLOVA), Kim Geon-min(NAVER CLOVA), Chung Joon-Son (KAIST)

Improving keyword recognition performance through automatic speech recognition systems to better identify uncommon keywords and enable greater readability of results.

6. Integration of Pre-trained Networks with Continuous Token Interface for End-to-End Spoken Language Understanding

https://arxiv.org/abs/2104.07253

Seo Seung-hyun*(NAVER CLOVA), Kwak Dong-hyun*(NAVER CLOVA), Lee Bo-won(Inha University)

Proposes a simple and robust integration method for the E2E SLU network with novel Interface, Continuous Token Interface (CTI), the junctional representation of the ASR and NLU networks when both networks are pre-trained with the same vocabulary.

7. Multi-Domain Unsupervised Image-to-Image Translation with Appearance Adaptive Convolution

Jung So-mi(NAVER LABS), Lee Ji-young(NAVER AI Lab), Son Kwang-hoon(Yonsei University)

Proposes a novel multi-domain unsupervised image-to-image translation (MDUIT) framework that leverages the decomposed content feature and appearance adaptive convolution to translate an image into a target appearance while preserving the given geometric content.