Optimization

The list below contains various faster and practical optimizers that can improve numerical efficiency.

  • A Momentum-Guided Frank-Wolfe Algorithm
    B. Li, Mario Coutino, G. B. Giannakis, and G. Leus
    IEEE Transactions on Signal Processing, vol. 69, pp. 3597-3611, June 2021.
    TLDR: Nesterov's momentum improves convergence rate of FW on a class of problems.
    [code]

  • Almost Tune-Free Variance Reduction
    B. Li, L. Wang, and G. B. Giannakis
    Proc. ICML 2020. (acceptance rate 21.8%)
    TLDR: Practical SVRG and SARAH that eliminate manually tuning parameters.

  • Frank Wolfe with Averaged Gradients
    Y. Zhang, B. Li, and G. B. Giannakis
    Proc. ICASSP 2021.
    TLDR: A few variants of FW are developed.

  • Revisit of Estimate Sequence for Accelerated Gradient Method
    B. Li, M. Coutino, and G. B. Giannakis
    Proc. ICASSP 2020.
    TLDR: Deeper understaning and more insights for Nesterov's momentum.

Distributed systems

Works focusing on distributed training and systems are summarized here.

  • TWICE: Two-Way Innovation Compression for Communication Efficient Federated Learning
    J. Sun, B. Li, T. Chen, Q. Yang, Z. Yang, and G. B. Giannakis
    submitted to IEEE Transactions on Cybernetics, July 2020.
    TLDR: We propose a novel communication efficient method for federated learning.

  • Real-Time Energy Management with Improved Cost-Capacity Tradeoff
    B. Li, T. Chen, X. Wang, and G. B. Giannakis
    Proc. GlobalSIP 2017.
    TLDR: Short version of the above TSG paper.
    [slides]

Learning under uncertainty

Here goes papers to harness statistical efficiency when learning under uncertainties.

  • Bandit Online Learning with Unknown Delays
    B. Li, T. Chen, and G. B. Giannakis
    Proc. AISTATS 2019. (acceptance rate 32.4%)
    TLDR: We study adversarial bandit problem under delayed feedback, where the delay is unknown. This is useful for certain scenarios in recommender systems.
    [poster]

  • Near-Optimal Algorithms for Piecewise-Stationary Cascading Bandits
    L. Wang, H. Zhou, B. Li, L.R. Varshney, and Z. Zhao
    Proc. ICASSP 2021.
    TLDR: Cascading bandits in piecewise-stationary environment.

  • Secure Edge Computing in IoT via Online Learning
    B. Li, T. Chen, and G. B. Giannakis
    Proc. Asilomar 2018.
    TLDR: A short version of the TSP paper above.