Full Name
Meir Feder
Job Title
Jokel Professor, School of Electrical Engineering
Company
Tel Aviv University
Speaker Bio
Meir Feder is a Professor in the School of Electrical Engineering in Tel Aviv University and the holder of the Jokel chair in Information Theory. He is also the head of Tel-Aviv University center for AI and Data Science. He was also a Visiting Professor at the department of EECS at MIT. His research interests are information theory and its applications in signal processing, communication and recently machine learning.
He received several academic and professional awards including the IEEE Information Theory society best paper award for his work on universal prediction, the “creative thinking” award of the Israeli Defense Forces, and the Research Prize of the Israeli Electronic Industry, awarded by the president of Israel. In parallel to his academic career, he is closely involved in the high-tech industry and founded several companies, among them Peach Networks (Acq: MSFT) and Amimon (Acq:LON.VTC). With his renewed interest in machine learning and AI, he co-founded Run:ai (Acq:NVDA), a virtualization and acceleration platform for AI infrastructure, which was acquired just last week by Nvidia. The technology he invented developed in Amimon of joint source-channel coding over MIMO was awarded the scientific and engineering award of the academy of motion pictures arts and science (OSCAR) and he was announced as the principal inventor for the 73rd EMMY engineering award.
He received several academic and professional awards including the IEEE Information Theory society best paper award for his work on universal prediction, the “creative thinking” award of the Israeli Defense Forces, and the Research Prize of the Israeli Electronic Industry, awarded by the president of Israel. In parallel to his academic career, he is closely involved in the high-tech industry and founded several companies, among them Peach Networks (Acq: MSFT) and Amimon (Acq:LON.VTC). With his renewed interest in machine learning and AI, he co-founded Run:ai (Acq:NVDA), a virtualization and acceleration platform for AI infrastructure, which was acquired just last week by Nvidia. The technology he invented developed in Amimon of joint source-channel coding over MIMO was awarded the scientific and engineering award of the academy of motion pictures arts and science (OSCAR) and he was announced as the principal inventor for the 73rd EMMY engineering award.
Speaking At
Abstract
Universal coding, prediction and learning usually consider the case where the data generating mechanism is unknown or non-existent, and the goal of the universal scheme is to compete with the best hypothesis from a given hypothesis class, either on the average or in the worst-case. Multiple universality considers the case where the relevant hypothesis class is also unknown: there is a set of hypotheses classes, with possibly different complexities. Sometime, but not necessarily, simpler classes are nested within more complex classes. A main challenge is to correctly define the universality criterion so that the extra “regret” for not knowing the relevant class is monitored. We propose several possible definitions and derive their min-max optimal solutions, including the suggestion of an hierarchy of such sets. Interestingly, the proposed approach can be used to obtain Elias codes for universal representation of the integers. Further, we suggest a multiple universality approach for general linear models, including linear regression, logistic regression and Perceptrons. Finally, we present how multiple universality, with its non-uniform convergence and regret bounds, can be applied to explain and design learning schemes for general, large or even huge, “over-parameterized” model classes such as deep neural networks, transformers and so on.
