The Other Half of the Memory Wall: Serving 35B MoEs from SSD with Trained Routing Prediction
Edge0 enables efficient mixture-of-experts inference on consumer hardware by predicting expert routing ahead of time and using recovery adapters to maintain quality with minimal active memory.
The Other Half of the Memory Wall: Serving 35B MoEs from SSD with Trained Routing Prediction
Edge0 enables efficient mixture-of-experts inference on consumer hardware by predicting expert routing ahead of time and using recovery adapters to maintain quality with minimal active memory.

