GPU batch job queue with AWS spot instances | glowing telegram - Episode 136

Channel:
United States saebynVODs
Subscribers:
542
Published on ● Video Link: https://www.youtube.com/watch?v=c4X2K3JWSiU


Duration: 0:00
226 views
3

In today's video, I focus on refining and enhancing our Rust/TypeScript web app, "Glowing Telegram," by transitioning specific parts to a serverless architecture with Pulumi and Python. This episode addresses the deployment of a GPU batch job queue—a crucial step in optimizing our application's performance on AWS using spot instances for cost-effectiveness.

We dive deeply into job definitions and container properties, investigating how to manage environment variables and connect to ECS roles. This session stands out by troubleshooting roles and compute environments, which is essential to improve our deployment strategy. By the end of the stream, we've successfully configured a managed compute environment and laid the groundwork for automatic job triggering using AWS services.

This detailed exploration brings us closer to a seamless transition to serverless, bridging the functions and operations of our backend with an evolving new UI for Glowing Telegram that we're setting up for future integration.

🔗 Check out my Twitch channel for more streams: https://www.twitch.tv/saebyn
GitHub: https://github.com/saebyn/glowing-telegram
Discord: https://discord.gg/N7xfy7PyHs



Other Videos By saebynVODs


2024-12-01Improving a Video Editor with TypeScript w/ Multiple Files | glowing telegram project - Episode 146
2024-11-30Thinking about video segments in Typescript/React Video Editor | glowing telegram - Episode 145
2024-11-29Troubleshooting typescript/react video editor components | glowing telegram - Episode 144
2024-11-28Building a new Video Editor Frontend in TypeScript | glowing telegram - Episode 143
2024-11-27More deploying AWS Step Functions | glowing telegram - Episode 142
2024-11-26More Lambda Functions and Step Functions with AWS and Pulumi | glowing telegram - Episode 141
2024-11-25Streamlining AWS Lambda with Rust and Python | glowing telegram - Episode 140
2024-11-23Testing our Docker image + exploring EC2 spot instance cost | glowing telegram - Episode 139
2024-11-21Streamlining Audio Model Downloads and Video Processing with AWS | glowing telegram - Episode 138
2024-11-19Troubleshooting AWS Batch Job Setup with Whisper AI | glowing telegram - Episode 137
2024-11-18GPU batch job queue with AWS spot instances | glowing telegram - Episode 136
2024-11-16Making Our Rust/TS Web App Serverless with Pulumi and Python | glowing telegram - Episode 135
2024-11-14Serverless Audio Transcriber with Rust and Python Using Pulumi | glowing telegram - Episode 134
2024-11-12Optimizing ECS with Pulumi and Python | "glowing telegram" - Episode 133
2024-11-10"glowing telegram" serverless with pulumi and python - Episode 132
2024-10-23Optimizing AWS Batch Jobs for Efficient Video Ingestion | glowing-telegram project - Episode 131
2024-10-20Bringing in Pulumi and AWS properly, part 3 | glowing-telegram project - Episode 130
2024-10-18Bringing in Pulumi and AWS properly, part 2 | glowing-telegram project - Episode 129
2024-10-16Bringing in Pulumi and AWS properly, part 1 | glowing-telegram project - Episode 128
2024-10-10Wrapping up Pulumi experiments with AWS | glowing-telegram project - Episode 127
2024-10-08Transforming Rust Web App into a Serverless AWS Solution | glowing-telegram project - Episode 126