Machine Learning Hub
A dynamic platform for the Tapis Framework, designed to streamline the machine learning workflow for researchers utilizing TACC's HPC systems.
Web Application
Full-Stack
Technologies used:
- Flask
- FastAPI
- Typescript
- React
- Docker
- Kubernetes
- Figma
- Hugging Face Hub API
- Langchain API
- Tapis API
Authors: Dhanny Indrakusuma, Nathan Freeman, Joe Stubbs
Machine learning plays a pivotal role in research by enabling the extraction of meaningful insights from vast and complex datasets, accelerating data analysis and pattern recognition. Its ability to uncover hidden relationships and trends empowers cross-disciplinary decision-making, predictive modeling, and new discoveries. Nonetheless, users lacking technical proficiency may grapple with the complexities inherent in machine learning models, leading to challenges in both implementation and interpretation. This underscores the need for user-friendly interfaces and tools that can effectively bridge this gap.
Within the Cloud and Interactive Computing group (CIC) at the Texas Advanced Computing Center (TACC), we are actively developing a Machine Learning Hub Portal (ML Hub), that integrates seamlessly into the Tapis Framework. Designed as a user-friendly platform, the ML Hub seeks to amplify the experiences of developers, scientists, and researchers engaged in machine learning research. Currently, Hugging Face offers a valuable resource by providing open-source pre-trained machine learning models and libraries, enabling developers to leverage powerful AI capabilities without building models from scratch. Through the integration of Hugging Face's API into our application, we aim to harness their state-of-the-art models to enhance ML Hub’s offerings and functionality.
Presently, our team has developed the Models Overview and Models Download features within ML Hub, offering non- technical users a unified avenue for accessing and viewing information on available machine learning models. In the upcoming months, our focus encompasses the implementation of the Inference Client and Training Engine, alongside the seamless integration of ML Hub's functionalities into the Tapis user interface.
Core features of ML Hub:
- Models Overview: The models page on the portal displays an immediate snapshot of the most downloaded Hugging Face models and allows users to filter models by author. Furthermore, users can access detailed specifics regarding a model.
- Models Download: Through the portal, users can download a specific model. The application fetches the latest version of the model in a version-aware manner, and generates download links for the model.
- Inference Client: The inference client enables users to spin up or spin down a server on TACC’s HPC systems for machine learning model inference, and display model results and metadata to user. This service is a fast way to get started, test different models, and prototype AI products.
- Training Engine: The ML Hub Portal provides users with the ability to fine-tune machine learning models with their dataset and send jobs to TACC’s HPC systems. Users can quickly showcase their fine-tuned models without concerning themselves with the complexities of the underlying technologies.
In conclusion, the ML Hub Portal aims to democratize the power of machine learning by providing user-friendly access to Hugging Face's cutting-edge models and addressing the challenges faced by non-technical users. This portal not only enhances their engagement but also unlocks new horizons for collaboration and innovation, ultimately shaping a more inclusive and impactful future in machine learning research.
An illustration of the Machine Learning Hub architecture: