Compute utilization at Mila¶
Compute powers Mila’s success¶
Compute is central to most research projects at Mila and a key driver of breakthroughs. Making the most of available capacity is vital to the success of Mila and its researchers.
Yet, some of Mila's internal compute remains inefficiently used (e.g. idle GPUs from inefficient jobs) which slows research progress. Raising awareness of efficient compute usage helps everyone run more experiments and get results faster.
What is expected of researchers¶
As a researcher, you are expected to request the right amount of compute for your experiments and use allocated resources efficiently. This ensures that your experiments do not negatively impact fellow researchers who also need compute resources.
The “Safety net” system¶
To ensure everyone benefits from the compute power made available to the scientific community, the IDT (Innovation, Development and Technology) team has introduced an awareness system designed to prevent unused GPUs by giving users adequate tools to identify and prevent compute waste.
IDT monitor cluster performance metrics, focusing strictly on hardware telemetry, never private data or source code. This provides usage reports and notifications, and helps you eliminate compute waste.
| Tool | Purpose | What’s in it for you |
|---|---|---|
| Reports & Dashboard | ⏱️ Continuous tracking | Access your data via the public dashboard. Monthly reports sent via Slack to help you track your usage. |
| Waste notification | 🔔 Early warning | Stay informed in case of significantly low usage, receive tips, and get in touch whenever needed. |
| IDT Interventions | 👨💻 Expert coaching | Benefit from 1-on-1 coaching with an expert to analyze your scripts and optimize job efficiency. |
How IDT may help you¶
It can be difficult to estimate how much compute your experiments actually need. IDT is here to support you with:
- One-on-one expert guidance during Office Hours or via Slack.
- Support with tracking and reporting tools to help you understand your compute usage.
- Standard resources and guides to help you diagnose and improve.
This system has been co-created with the Mila Research Compute Committee and will be improved over time. Thank you in advance for sharing your feedback and suggestions.
Mila looks forward to seeing you run your projects on its infrastructure.
Let’s put those clusters to work!