Reducing cost, latency, and manual efforts in hyperparameter tuning

Building machine/deep learning models is a tough task; it involves multiple sub-tasks such as data preparation, model architecture creation, hyperparameter tuning, and extensive experimentation. Each set of hyperparameters creates a new experiment and if not done in an efficient manner, can be highly compute intensive, resulting in higher compute costs and time. Ray Tune gives us out-of-the-box features to manage compute resources efficiently. Its scheduling algorithms give us the opportunity to cut down the compute required by pruning bad trials and exploit the same resources to schedule new trials. Also, with hyperparameter tuning and extensive experimentation comes the problem of tracking, logging all these experiments, and saving the best model. Integration of Ray Tune with tools like MLflow and Weights & Biases saves the day. We will talk about how we have used Ray Tune to reduce manual efforts, compute costs, and logging hassle while training our models.