Create accountLogin

Recent searches

No recent searches

Search options

Only available when logged in.
veganism.social is one of the many independent Mastodon servers you can use to participate in the fediverse.
Veganism Social is a welcoming space on the internet for vegans to connect and engage with the broader decentralized social media community.

Administered by:

Server stats:

302
active users

veganism.social: AboutStatusPrivacy policy

Mastodon: AboutGet the appKeyboard shortcutsView source codev4.3.0-alpha.5+vegan

#with

9 posts7 participants0 posts today
Public
Hacker News

Optimizing ML Training with Metagradient Descent

arxiv.org/abs/2503.13751

arXiv logo
arXiv.orgOptimizing ML Training with Metagradient DescentA major challenge in training large-scale machine learning models is configuring the training process to maximize model performance, i.e., finding the best training setup from a vast design space. In this work, we unlock a gradient-based approach to this problem. We first introduce an algorithm for efficiently calculating metagradients -- gradients through model training -- at scale. We then introduce a "smooth model training" framework that enables effective optimization using metagradients. With metagradient descent (MGD), we greatly improve on existing dataset selection methods, outperform accuracy-degrading data poisoning attacks by an order of magnitude, and automatically find competitive learning rate schedules.
#HackerNews#Optimizing#ML
VegansExploreLive feeds
About